mirror of
https://github.com/FlashML-org/FreeToken.git
synced 2026-11-21 01:52:30 +08:00
* fix(models): find index-only checkpoints that keep shards in a subdirectory * fix(models): make the checkpoint index authoritative for shard discovery The index is read directly so safetensors_weight_map stays the only header-based reader, and _host_ram_fits_parallel sizes the shards the reader will actually open, so the low-RAM serial fallback fires again for subdirectory layouts. * fix(models): fall back to root shards when the index is unusable Share one index parser between iter_weight_files and safetensors_weight_map instead of a third copy; warn and glob the root when the index is broken, absent or empty; reject index keys that escape the model folder; let the hub download fetch the index itself; tell the truth in the diagnose docstring. * revert(models): back out the defensive index fallback, keep the no-index test Per review: a broken index is a broken checkpoint, so the loader keeps the plain one rule -- use the index if there is one, otherwise glob the root. The no-index fallback test stays as the only coverage of the root glob and the consolidated.safetensors rule.