Commit Graph
3 Commits
Author SHA1 Message Date
Mukkesh Ganesh 03e5ea0672 feat(server): report per-request prefill and decode timings in api responses (#814)
* feat(server): report per-request prefill and decode timings in api responses

* fix(server): take timings spans from the scheduler and gate them behind --enable-timings-report

* fix(scheduler): count accepted drafts as min(n, emitted) and stamp reqstats with perf_counter
2026-11-03 14:44:25 -08:00
Xiaoze Fan e0a7e23c9d feat(server): add /v1/cache/flush to drop the prefix cache when idle (#811)
* feat(server): add /v1/cache/flush to drop the prefix cache when idle

* refactor(scheduler): run every cache flush at the idle safe point

* test: cover cache flush on the overlap loop and on a live server
2026-10-22 23:43:14 -07:00
Xiaoze FanandShuo Yang 3af9d90ee5 feat: initial open-source release
Co-authored-by: Shuo Yang <andy_yang@berkeley.edu>
2026-08-11 22:53:25 +00:00