Files
Xiaoze Fan e328cf6381 refactor: add engine and scheduler seams for multi-row decode steps (#833)
* refactor: add engine and scheduler seams for multi-row decode steps

* refactor: rename the graph steps argument to rows_per_req

* refactor: drop VerifyInfo and require rows_per_req on verify batches

* refactor: fix stale seam comments and read decode_rows_per_req directly

* refactor: reword two GraphRunner comments for multi-row graphs

* fix(engine): scale the GPU-decode MoE pool limits by decode_rows_per_req

* refactor: drop the unread Context.moe_offload_cache field

* test: trim the multi-row seam tests
2026-10-27 16:03:01 -07:00
..
2026-08-11 22:53:25 +00:00