Tool Selection: Contrastive Filtering and Constrained Decoding
==============================================================

Supports main.txt sections 2.3, 2.5.
Two mechanisms that narrow the output space: filtering tools before
generation, constraining tokens during generation.


## 1. Problem

  (a) Context pressure: 1024 encoder tokens, real deployments have
      50-200 tools - they don't fit.
  (b) Structural validity: the model can hallucinate invalid function
      names or argument keys despite high JSON parse rates.


## 2. Contrastive Head (Tool Filtering)

Real deployments have too many tools for the context window. Rather
than adding a separate retrieval model, we reuse the encoder: a
lightweight MLP projects mean-pooled encoder output into a contrastive
space for tool ranking.

  encoder -> mean_pool -> Dense(d_model/4) -> ReLU -> Dense(128) -> L2-norm

CLIP-style training (0.1x weight, every 1000 steps, in-batch
negatives, learnable temperature). At inference: encode query and
candidate tools, rank by cosine similarity, pass only top-k to the
encoder for generation.

  - No separate retrieval model - same encoder, same parameters
  - Tool embeddings can be precomputed and cached
  - Contrastive head adds <0.1% parameters
  - For small catalogs (<=5 tools): skip retrieval, pass all directly


## 3. Why MLP Instead of Linear Projection

Our encoder has NO feedforward layers. The only nonlinearities are
softmax (attention) and sigmoid (gates). A linear contrastive
projection can only rotate and scale the mean-pooled representation.

In a standard transformer with FFN, the encoder output has been through
N layers of nonlinear per-position processing - a linear projection
often suffices. In our pure-attention encoder, the output has been mixed
extensively but never nonlinearly transformed per-position.

The MLP provides the per-position nonlinearity that the encoder lacks,
specifically for the retrieval task. Same argument applies to the
confidence head (cloud-fallback.txt section 3).


## 4. Constrained Decoding

After filtering selects top-k tools, the decoder generates tool calls.
We enforce structural validity via character-level trie-based logit
masking:

A JSON state machine tracks position in the output:
  - IN_NAME: constrain to valid tool names
  - IN_KEY: constrain to valid parameter names per selected tool
  - IN_VALUE: unconstrained

Character-level (not token-level) because BPE tokens don't align with
name boundaries. "get_weather" might tokenize as ["get", "_we", "ather"]
or ["get_", "weather"]. Character-level trie works regardless of
tokenization.

Guarantees syntactically valid output with zero accuracy cost.


## 5. The Funnel

Filtering and constrained decoding are complementary stages:

  200 tools -> contrastive filter top-5 -> encode 5 tools
  -> constrain to 5 names -> constrain to valid params per name
  -> free-form values

Each stage narrows the output space. The model's actual "choice" is
just: which of 5 tools to call, and what values to fill in.


## 6. vs. Alternatives

  vs. Separate retrieval model:
    One model, shared representations, no distribution shift.

  vs. Stuffing all tools in context:
    50 tools x ~50 tokens = 2500 tokens, exceeds 1024 context.
    Even if they fit, attention dilution over irrelevant tools.


## 7. Ablations (maps to main.txt 4.4-4.5)

  A. Retrieval Quality [MUST-HAVE]
     Recall@k, MRR. Stratify by tool count.

  B. With vs Without Contrastive [MUST-HAVE]
     lambda=0.1 vs lambda=0, generation metrics only (no retrieval
     at inference). Tests whether contrastive training helps the
     encoder build more discriminative tool representations.

  C. Retrieve-then-Generate vs Full Context [MUST-HAVE]
     Synthetic large catalogs (20, 50, 100 tools).
     All-in-context vs retrieved top-5 vs oracle top-5.

  D. Constrained vs Unconstrained Decoding [MUST-HAVE]
     Unconstrained vs name-only vs full constraint.

  E. Linear vs MLP Head [STRONG]
     Tests whether head nonlinearity matters more for attention-only
     encoder.

  F. Contrastive Weight Sweep [STRONG]
     lambda in {0, 0.01, 0.05, 0.1, 0.2, 0.5}.
