fix(phase-4/25): CMER — text_conf is a probability in [0,1]; flag directionality clarified; shape validation + PyTorch assumed

phases/04-computer-vision/25-vision-language-models/outputs/skill-cmer-monitor.md

@@ -20,7 +20,7 @@ Treat cross-modal alignment as a first-class production KPI.
 ## Inputs
 
 - `vlm_output`: generated text.
-- `text_confidence`: average per-token log-prob converted to probability (or mean logit).
+- `text_confidence`: mean per-token probability after softmax, in `[0, 1]`. Compute as `exp(mean(log_probs))`. Do not pass raw logits; raw logits are unbounded and `conf_threshold` assumes a probability.
 - `image_embedding`: CLIP-family embedding of the image (DINOv3, SigLIP, CLIP).
 - `text_embedding`: CLIP-family embedding of the generated text.
 - Optional `prompt_type`: label for grouping (vqa / ocr / captioning / agent).
@@ -28,14 +28,20 @@ Treat cross-modal alignment as a first-class production KPI.
 ## Per-request computation
 
 ```python
+import torch
+
 def cmer_flag(image_emb, text_emb, text_conf, sim_thr=0.25, conf_thr=0.8):
-    image_emb = image_emb / image_emb.norm()
-    text_emb = text_emb / text_emb.norm()
+    if image_emb.shape != text_emb.shape:
+        raise ValueError(f"emb shape mismatch: {image_emb.shape} vs {text_emb.shape}")
+    image_emb = image_emb / (image_emb.norm() + 1e-8)
+    text_emb = text_emb / (text_emb.norm() + 1e-8)
     sim = float((image_emb * text_emb).sum())
     flagged = (text_conf > conf_thr) and (sim < sim_thr)
     return {"sim": sim, "flagged": flagged}
 ```
 
+Embeddings are 1-D PyTorch tensors (`torch.float32`) from an independent CLIP-family encoder. If you use NumPy arrays, swap `.norm()` for `np.linalg.norm(...)` and cast the output accordingly.
+
 Store `sim`, `text_conf`, `flagged`, `prompt_type`, `timestamp`, `model_version`, `request_id` to your monitoring pipeline (Prometheus, DataDog, OpenTelemetry).
 
 ## Aggregate metric
@@ -72,4 +78,4 @@ Report per endpoint, per prompt_type, per model version.
 - Never compute CMER using the VLM's own embeddings; use an independent encoder (DINOv3, SigLIP, or CLIP-L/14). Otherwise you are measuring the model's self-consistency, not alignment.
 - Always log the raw `sim` value, not just the `flagged` bit; distribution shifts show up in the lower quartile before the flag rate changes.
 - Do not ship a VLM endpoint without CMER monitoring; hallucinations are the dominant production failure mode and silent without this metric.
-- For sensitive domains (medical, legal, financial), drop `sim_threshold` to 0.35 or higher; a stricter gate is the right default.
+- For sensitive domains (medical, legal, financial), raise `sim_threshold` to 0.35 or higher; the flag condition is `sim < sim_threshold`, so a higher threshold catches more outputs as potentially ungrounded — the right default for high-stakes use.