checkpoint_diagnose
Find out why FreeToken does not load a checkpoint, sort the failure into one kind, and repair the checkpoints whose own files are non-standard.
Run everything from the repo root with PYTHONPATH=python. Nothing here needs a GPU or the checkpoint's weights.
Warning
The diagnosis is heuristic and can be wrong: the failure kind, the findings and the suggested fixes are a best guess from the headers and configs, not a proof. Check them against the problems it prints before acting on them.
Important
For agents: explain the diagnosis to the user in plain terms first: what fails, which kind it is, and what the fix would change. Ask the user before you run a fix script or open an issue. Do not change the user's checkpoint on your own.
| Script | Does |
|---|---|
diagnose.py |
checks a checkpoint dir or HF repo id, prints the failure kind and the next step |
fix_config.py |
level 1 fix: edits only config.json / hf_quant_config.json |
rename_tensors.py |
level 2 fix: renames tensors (shard headers, index, quant skip lists); tensor data is kept byte for byte |
fetch_headers.py |
header-only stub of an HF repo; diagnose.py calls it for a repo id |
ckpt_edit.py |
the edit primitives both fix scripts use, for a case they do not cover |
Step 1: which format
diagnose.py routes by itself, but know where each format goes:
| The checkpoint has | Go to |
|---|---|
*.gguf files |
scripts/gguf_check/ (its README), then the GGUF section below |
freetoken_weight.json (FTW, made by ft checkpoint) |
diagnose the HF checkpoint it was converted from; FTW repairs are in docs/ftw-hotfix.md |
root *.safetensors shards |
this directory, from step 2 |
only *.bin / *.pt, or MLX weights |
not served: FreeToken reads safetensors and GGUF |
GGUF: three kinds of failure
scripts/gguf_check/check.py prints the problems of each file; the first one tells the kind.
| Kind | check.py says | Do |
|---|---|---|
| a ggml type has no kernel | <layer kind> <type>: no <ops> kernel (needs --kernels from probe_kernels.py) |
the build or GPU lacks kernels for that quant type: pick a file with another type, or report it with the probe_kernels.py table |
| the model's loader does not read the file | unexpected tensor ..., N tensors never loaded, tensor ... yielded twice, <tensor>: got ..., model has ..., MoE layer N ... bank is ..., the model needs ..., or a NotImplementedError from the family's config parse |
a FreeToken gap for that layout: report it with the check.py record |
| the GGUF file itself | is not a GGUF file, duplicate GGUF key / tensor, runs past the end of the file, bad general.alignment, tokenizer: no EOS token, tokenizer: no chat template, chat template: <error> |
the file is broken or incomplete: download it again, or use another file of the repo |
A file whose architecture FreeToken does not serve shows as skip, not as a failure.
Step 2: diagnose a safetensors checkpoint
python scripts/checkpoint_diagnose/diagnose.py <ckpt dir | HF repo id[@revision]> --out report.jsonl
- A local dir is read as is: the headers and JSON files, so a 400 GB checkpoint takes seconds. A weight reader that needs tensor values gets the real bytes it asks for.
- A repo id is fetched as a header-only stub into
~/.cache/freetoken/checkpoint-stubs/(a few MB). Nothing is downloaded beyond that, and no.pyfile: a repo whose config is its own code (auto_map) is skipped, diagnose a local download of it instead. - A remote repo is only diagnosed: its record has no fixes, and the fix scripts refuse a stub. When it says
fixable, download the checkpoint and diagnose the local directory to get the fix commands. - Add
--text-model-onlywhen the checkpoint will be served with--text-model-only. --outappends one JSON record per checkpoint. The fix scripts read their fixes from it with--from-report.
It first checks the files themselves (data offsets cover each shard exactly, the index agrees with the shard headers), then runs the server's own load path on the CPU with meta tensors: config parse, model build, dense weight load through the model's load_state_dict, and expert-bank fill for MoE models with the reader the engine picks.
When that fails, it also checks the tensor names against transformers' own model class and its checkpoint conversion rules, and lints the files for the known non-standard forms.
Output, per checkpoint: PASS, or FAIL at <stage>: [<kind>] ... with the problems, what transformers makes of the names, the findings, and the next step.
Step 3: the kind of failure
Each failure gets one kind. diagnose.py tries, in this order: broken files (checkpoint), arch, quant_method, a broken prefix (checkpoint), quant_model, names transformers cannot map or another non-standard finding (checkpoint), packing; anything left is loader.
| Kind | Means | Do |
|---|---|---|
arch |
the architecture is not registered in python/freetoken/models/register.py |
not served; a new architecture starts as a feature request |
quant_method |
FreeToken has no reader for the quantization format (GPTQ, AWQ, MLX, bitsandbytes, an unsupported compressed-tensors scheme, ...) | not served; use a release in a format python/freetoken/layers/quantization/configs/ reads |
quant_model |
the format is read, but this model or layer kind has no method for it (no quant method for moe.<kind>), or the reader does not produce the scales the method needs |
a FreeToken gap: report it |
loader |
the model and format are served, and transformers loads these names, but FreeToken's reader does not | a FreeToken gap: report it. A finding marked loader may offer a fix that works around it |
packing |
the expert banks do not fill, and transformers repacks these expert tensors (for example one tensor per expert where the family's release stacks them) | a FreeToken gap: report it. Repacking tensor data is not offered here |
checkpoint |
the checkpoint's own files are broken or non-standard: an incomplete shard or index, names transformers would not load either, or a config that contradicts what the shards store | fix it, step 4 |
To report a FreeToken gap, use the "model checkpoint" issue template and paste the diagnose.py output.
How to tell loader / packing from checkpoint: the line transformers ... would not load it either means transformers' model class and conversion rules cannot map the names either, so the checkpoint is at fault. transformers loads these names means the names are a form transformers accepts, so FreeToken should read them too.
Step 4: fix a non-standard checkpoint
Fixes apply to a local copy of the checkpoint only, never to a remote repo or its stub.
Rules:
-
Fix into a new directory (
--out <new dir>). Unchanged files there are symlinks to the original, so it costs only the rewritten shards. Use--in-placeonly when the user asks for it. It keeps a backup, and--restoreundoes the latest in-place edit. -
One level per round. If the record has a
renamefix, apply that first, then rundiagnose.pyon the new directory, then apply the config fixes of that new record. A rename changes the names the config findings talk about. -
Read the plan before applying it. Every run prints the plan first and changes nothing until it is confirmed. Check that:
- each rule renames the tensors you expect, and the example
old -> newlines are right; - nothing is renamed that should not be. A
warning: ... matches are inside the nameline means the pattern is not anchored; - the config edits name the right modules;
- the shard costs are what you expect.
rewritten, copies N GiBneeds that much free disk.
If anything in the plan is not what you expected, do not apply it: change the rule or the edit and plan again.
- each rule renames the tensors you expect, and the example
-
Confirming: in a terminal the script asks
[y/N]. Without a terminal (an agent), pass--plan <file>. The first run saves the plan there and changes nothing. Read it, show it to the user, and run the exact same command again to apply it only after the user agrees. If the command or the checkpoint changed in between, the second run saves the new plan instead of applying anything. -
After every fix, run
diagnose.pyon the result, until it passes or nothing fixable is left.
The usual round, for an agent:
D=scripts/checkpoint_diagnose
python $D/diagnose.py /models/x --out r0.jsonl
python $D/rename_tensors.py /models/x --out /models/x-fixed --from-report r0.jsonl --plan plan.json # prints and saves the plan
python $D/rename_tensors.py /models/x --out /models/x-fixed --from-report r0.jsonl --plan plan.json # after the user agrees to the plan: applies it
python $D/diagnose.py /models/x-fixed --out r1.jsonl
python $D/fix_config.py /models/x-fixed --in-place --from-report r1.jsonl --plan plan.json # twice, as above
python $D/diagnose.py /models/x-fixed
--from-report applies the fixes diagnose.py found for that checkpoint; anything else is a manual edit, below.
Fix levels
| Level | Script | Changes | Typical use |
|---|---|---|---|
| 1 | fix_config.py |
config.json, hf_quant_config.json |
the quantization skip lists, compressed-tensors targets, a config key |
| 2 | rename_tensors.py |
shard headers, model.safetensors.index.json, and the skip-list entries / targets that are module names or globs (a re: entry is not rewritten; the plan warns about every entry left naming the old modules) |
broken prefixes, legacy tensor names |
| - | not offered | tensor data | repacking experts, dropping tensors, dequantizing: re-export the checkpoint, or report the FreeToken gap |
Manual edits
When no automatic fix fits, write the edit yourself. It goes through the same plan and confirmation.
# rename: re.sub(PATTERN, REPL, name), rules in order; anchor with ^ and escape dots
python $D/rename_tensors.py /models/x --out /models/x-fixed --rule '^model\.language_model\.visual\.' 'model.visual.'
# config key: dotted path, JSON value
python $D/fix_config.py /models/x --out /models/x-fixed --set text_config.tie_word_embeddings false
# quantization skip lists, in the dialect's syntax:
# ModelOpt: glob (model.language_model.layers.*.mlp.gate)
# compressed-tensors: module name or re:<regex> (re:^model\.visual\..*)
# fp8: module name (model.language_model.layers.0.linear_attn.in_proj_a)
python $D/fix_config.py /models/x --out /models/x-fixed --skip-add 'model.language_model.layers.*.mlp.gate'
python $D/fix_config.py /models/x --out /models/x-fixed --skip-remove 'lm_head'
A rule that renames nothing, and a rename that gives two tensors the same name, are refused. So is an --out directory that already has files in it.
Known non-standard forms
The findings diagnose.py prints, and what they fix. kind is the failure kind the finding points to.
| Finding | The checkpoint | Level | Fix |
|---|---|---|---|
prefix |
nests the text tower more than once (model.language_model.language_model.) |
2 | rename to model.language_model. |
prefix |
stores the vision tower under the text tower (model.language_model.visual.) |
2 | rename to model.visual. |
prefix |
uses vLLM's internal names (language_model.model., language_model.lm_head.) |
2 | rename to model.language_model. / lm_head. |
quant_declared_but_dense |
stores modules in bf16 that its quantization_config does not skip (vision tower, lm_head) | 1 | add them to the skip lists |
quant_skipped_but_quantized |
stores modules quantized that its skip list covers, often through a container entry (...linear_attn) |
1 | remove the entry, and list its dense children instead |
ct_target_vllm_names |
writes compressed-tensors targets with vLLM's names, so they match no tensor | 1 | rewrite the targets to model.language_model. |
quant_unlisted_dense_router (kind loader) |
stores in bf16, without listing them as skipped, modules the family serves in bf16 anyway (the Qwen3.5 routers) | 1 | list them; works around the reader |
kv_output_scale (kind loader) |
stores fp8 KV scales as k_proj.output_scale / v_proj.output_scale (only reported for the families whose reader drops k_scale) |
2 | rename to k_scale / v_scale, which the reader drops |
expert_tensors_incomplete |
lacks some expert's tensors in some layer | - | download it again, or ask its uploader |
(stage files) |
has a shard whose tensors do not cover the file, or an index that disagrees with the shards | - | download it again |
no_vision_tower |
is text-only under a multimodal architecture | runtime | serve with --text-model-only |
no_processor_config |
has a vision tower but no preprocessor_config.json (image requests fail with 400) |
file | copy that file from the base model |
fp8_without_config |
stores fp8 weights with no quantization_config | - | not fixable here; the scales and block size are unknown |
mlx (kind quant_method) |
is MLX-quantized (.scales / .biases) |
- | not served; use a non-MLX release |
Step 5: check the fixed checkpoint
diagnose.py passing means: the names, shapes, quantization methods and expert-bank layout all line up with what the model builds.
It does not read weight values. On a real checkpoint, finish with ft serve --model-path <fixed dir> and a short request. For a checkpoint with a vision tower, also send an image request.
Using the primitives
import sys; sys.path.insert(0, "scripts/checkpoint_diagnose")
from ckpt_edit import Checkpoint, confirm, plan_lines_for_save
ck = Checkpoint("/models/x")
ck.rename_regex([(r"^old\.", "new.")])
ck.quant_skip_add(["model.visual.*"])
print("\n".join(ck.log + plan_lines_for_save(ck, "/models/x-fixed")))
ck.save(out="/models/x-fixed") # ck.save() edits in place, with a backup
ck.tensors() gives every tensor's shard, dtype and shape from the headers. ck.quant_configs() gives every quantization config the checkpoint carries.