Repository navigation
Conversation
A reload paid for a fresh interpreter to import livekit.agents and the project's dependencies (~650 ms with common plugins), although only the user's code changed, and then waited for the old process's graceful shutdown (~200 ms) before starting. Keep one spare interpreter per dev session. It imports every installed module the last worker used (recorded per project in the user cache dir), then waits on stdin; a reload activates it, so only the user's modules are imported. The spare starts once the active worker connects to the dev channel, so it never competes with its imports. A spare is discarded if dependencies changed since it started, and modules the project itself provides are never preloaded. When the old process has no running jobs, it now shuts down while the new one starts. With jobs, the order is unchanged so a restored job never overlaps the original. Reload, save to registered worker (livekit-agents 1.8.10, macOS): 1315 ms to 512 ms, of which 500 ms is the debounce.
With a preloaded interpreter, the debounce was 97% of a reload. One save's events arrive within a few milliseconds (write, or write to a temporary file and rename), so 100 ms still coalesces them.
The spare must preload recorded installed modules but never one the project provides, hold its output and run only once activated; a run must record its installed modules for the next spare.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1102, which is stacked on #1094.
Why a reload was slow
I timed a
lk agent devreload (save →registered worker) on #1094 with the starter template. It takes ~1.2 s, more than a cold start:livekit.agentsand every dependency againThe last step is repeated work: only the user's own files changed.
What this changes
pyproject.toml, the lockfiles andsite-packagesare fingerprinted by mtime.Results
Median of 5–10 runs, against a local
livekit-server --devon macOS, livekit-agents 1.8.10. The PTY harness answers OSC 11/DA1 like a real terminal.agent-starter-python, reloadagent-starter-python, cold startChecked by hand
lk dispatch createruns in the worker started from the spare.TestWarmPythonSparePreloadsInstalledModulesAndWaitsForActivationcovers: preloading, never preloading the project's own module, holding output until activation, and recording the module list. It fails with the project-module guard removed.Still slow, outside lk
AgentServerinitializes the inference executor before it registers (worker.py,await self._inference_executor.initialize()). Prototyping that init alongside registration, with job launch waiting for it, took such an agent from 1463 to 528 ms cold, and 1091 to 123 ms per reload with this PR.livekit.agentsimportsopenai(viainference.llm),av(codecs) andrequests(telemetry) at import time, though registration needs none of them. Deferring them would take the starter template from 446 to 284 ms to registration.