daemon

Warm-process render daemon: re-run a scene script without paying startup.

Every fresh python scene.py pays several seconds of library import plus ~20 s of Taichi kernel preparation before the first pixel renders, even with a warm offline cache. This daemon pays them once: it keeps the process (and Taichi’s in-process kernel cache) alive and re-executes the scene script on demand, so from the second render on the only cost is the render itself.

Usage:

.venv/Scripts/python.exe -m algan.daemon               # general daemon
.venv/Scripts/python.exe -m algan.daemon scene.py [options] [-- script args]

Launching one by hand is optional. An ordinary python scene.py starts a general daemon itself when none is running, runs on it, and leaves it warm for the next script (see algan.daemon_client; ALGAN_AUTO_DAEMON=0 disables it, ALGAN_USE_DAEMON=0 disables the daemon entirely). Launch one by hand when you want it in a terminal you can watch, or want Enter-to-re-render.

General mode (no SCRIPT) is the one to leave running. The daemon publishes a state file at $ALGAN_HOME/daemon.json and then serves whatever scripts come to it: every subsequent python any_scene.py notices the state file during import algan and hands itself over, so scripts are launched exactly as they always were and simply start rendering in ~1 s.

Concurrent scripts are queued and run one at a time, in arrival order – which is also what this project needs on Windows, where two live render processes fight over the output mp4. A waiting client is told its position.

Triggers (a render is never interrupted; triggers arriving mid-render coalesce into at most one queued re-run):

  • Enter in the daemon terminal re-renders; q quits. This is the primary workflow: edit in your editor, save, switch to the daemon, Enter.

  • A localhost TCP socket (preferred port 46711; --port, or env ALGAN_DAEMON_PORT; --no-serve disables) accepts the line commands render / ping / quit, each of which must carry the token from the state file. If the preferred port is taken the daemon binds an ephemeral one instead of exiting, and publishes it in the state file – which is where clients look anyway. Bind an editor key to:

    algan daemon render     # also: algan daemon ping, algan daemon quit

    Those subcommands read $ALGAN_HOME/daemon.json for the port and the token and send the line for you.

  • --watch re-renders when the scene script or any of its sibling helper modules change on disk (polled; coalesced; never interrupts).

When a run ends the daemon restores a clean slate, before it goes idle rather than at the start of the next run – so what it holds while waiting is the warm process and nothing else:

  • SceneManager.reset() – fresh scene, camera, light and timeline.

  • SETTINGS.snapshot() / SETTINGS.restore() resets every public runtime settings section to its import-time value, so one run cannot leak configuration into the next. Private adaptive renderer state is retained.

  • User helper modules – modules imported from the script’s directory tree are evicted from sys.modules so the next run picks up their edits (the daemon prints what it evicted). Modules imported from elsewhere are NOT reloaded.

  • The render’s GPU memory goes back to the driver: one gc.collect() (the scene’s object graph is cyclic, so refcounting alone frees almost none of it) and one torch.cuda.empty_cache(). Measured on a 4 GB card, an idle daemon holding 1.6 GB after a 90-frame render now holds ~0.1 GB, at a cost of ~0.15 s and no measurable change to the next render. ALGAN_DAEMON_RELEASE_MEMORY=0 keeps the memory cached instead.

Edits to algan itself are handled, not merely warned about. The daemon fingerprints every algan source file at startup and re-checks at every run launch; if anything changed it refuses the run and shuts down, so the script executes in a fresh process that loads the edited code. A new daemon starts on the next run. This costs a cold start – that is what editing the library has always cost – but it can no longer render with stale modules or compile mixed-version kernels from a half-edited *_taichi.py. An edit that lands during a run is not caught, exactly as it is not caught for a plain python scene.py. Keep to one rendering process at a time on Windows.

A run served here is meant to be indistinguishable from one in its own process: sys.argv, the working directory, the caller’s full environment, stdout/stderr at the descriptor level (so ffmpeg and other subprocesses reach the caller) and their tty-ness are all reproduced. stdin is not – it is connected to os.devnull, because the daemon’s own stdin is its re-render trigger – and atexit handlers do not run, because runpy does not run them and a warm process never shuts down.

Three more limits that are specific to serving other processes:

  • Startup-only settings cannot be adopted from a client, with one exception. ALGAN_ANIMATION_DEVICE and friends are read while Torch/Taichi initialise, i.e. when the daemon started. A script that sets one to a different value is refused with an explanation and runs cold in its own process, rather than being silently rendered on the wrong device. ALGAN_RENDER_DEVICE is adopted (_adopt_render_device()): it only seeds SETTINGS.computing.render_device, which owns the value from then on, and every render re-selects Taichi’s arch from it. A script that wants the other device is served warm; if that crosses the CPU/GPU line its first render pays one kernel-preparation pass, which is still far less than the cold start refusing it used to cost.

  • Neither can settings read while algan is imported. The renderer’s toggles become module-level defaults during import algan, which in a daemon happened at its launch – so a script that sets one before its own import algan, the way every A/B script in benchmarks/ selects an arm, would otherwise be served by a process that never saw it and would render with the daemon’s values instead. Those are refused too (algan.daemon_client.describe_import_env_mismatch()); the swapped-in environment covers every variable read live, so flipping one during a run works here exactly as it does cold. The corollary is that a daemon started by a script with non-default toggles serves only scripts that set the same ones: stop it if you want one baked with the defaults.

  • Anything that can reach 127.0.0.1 can ask the daemon to execute a path. Every request – run, cancel, render, ping and quit alike – must carry the token from the state file, which lives in the user’s home directory (mode 0600 where the platform honours it). Do not forward the port off-host.

  • The daemon must be the same Algan. The state file records the interpreter, the prefix, the package directory and the version it was started with, and a client whose own differ is not served: it runs cold in its own process rather than executing against another virtualenv’s site-packages (algan.daemon_client.describe_interpreter_mismatch()).

Functions

default_port()[source]

The trigger socket’s default port.

Read when the command line is parsed rather than at import: it configures the client/daemon transport, not anything a script renders, so there is no reason a warm process could not honour a value set after its own import.

main(argv=None)[source]
strip_plumbing_frames(tb)[source]

Drop the daemon’s and runpy’s frames from the top of tb.

A script that raises under the daemon showed algan/daemon.py ... in execute and two runpy frames above its own first line – frames a plain python scene.py would never print and that say nothing about the error. Only the leading run of them is dropped, so a script that itself calls runpy still shows that call.

Returns tb unchanged if every frame is plumbing: an empty traceback would be worse than an honest one.