mirror of
https://github.com/alphaXiv/OpenResearch.git
synced 2026-11-12 10:05:24 +08:00
Daniel Kim
91b30e033a
* Keep run logs complete across supervisor respawns and run completion A respawned supervisor no longer truncates the local run log before its replay has delivered anything, so an unreachable backend leaves the log intact (OR-385). Terminal status is now recorded only after the log tail has drained, and the tail only stops after a successful pass that began once the job was terminal (OR-376). Local and SSH tails count only newline-terminated lines until the final drain, so a record flushed mid-line is no longer cut at the flush boundary (OR-365). * Fail streaming log passes on read errors and start the final drain promptly HF, Modal and Kubernetes log passes now return an error on read failures and unsuccessful launcher or kubectl exits, so a failed final pass is retried instead of counted as drained. The supervisor counts replayed lines itself so a pass that fails partway still dedups, cuts an in-flight pass short when the job ends, and gives the final pass a 5s idle wait so it fits inside the 20s drain bound. * Let the final streaming pass run until the stream closes A short idle wait could end the final Kubernetes or Modal replay before a slow launcher printed its first line, and count that as drained. The final pass now uses the normal idle wait, so only the watcher's 20s drain bound stops it.