Why a Softmax Router Accepted a Soup Recipe
Краткое содержание
Pixel’s announcement examines a three-label softmax router that accepted “Suggest a soup recipe” as notes with confidence 0.486, above its 0.4 threshold. It details the small synthetic dataset and test results, including 6 of 6 OOD rows falsely accepted, showing why a threshold is not the same as calibrated confidence.
DEV Community
How a three-label softmax router accepted a soup recipe as notes
I kept this failure in the fixed commit bd2a30de287fe419b7d31838b40f5ae4d2d70061. "Suggest a soup recipe." does not belong to weather, calendar, or notes, but the saved test prediction for ood-template-4-en has top_route equal to notes, a maximum of 0.486, and accepted true at the 0.4 threshold. This package is small. It turns text into 4,096-dimensional character n-gram counts, applies a linear transform and three-way softmax, and was fit on 96 author-written training records for one 300-epoch CPU run. The 204 synthetic records split into 96 training, 54 validation, and 54 test; the original templates are written and visible to me, so these records were not handled as a human-blind benchmark. The mechanism sits in the softmax denominator. router.py sets confidence to the largest softmax value, then accepts when confidence >= threshold; below threshold, route is null. Because the three probabilities sum to one, the maximum cannot fall below about 1/3. That lower bound is arithmetic, neither a calibrated probability nor a membership certificate. The recipe's notes score of 0.486 is only about 0.152 above a uniform 1/3. It clears 0.4 because the model slightly prefers notes and the threshold is exactly 0.4. I have not run per-feature attribution, so I cannot say which words pushed notes to the top. The threshold came from a fixed rule. validation-thresholds.json shows that a candidate needed validation accepted precision of at least 0.9, then the chosen threshold was the one with the most correct accepted rows. At 0.4, validation had 44 correct among 47 accepted; 0.5 and 0.6 were also feasible but accepted far fewer. This is empirical selection, not calibration. At 0.4, validation accepted 3 of its 6 OOD rows. On test, test-report.json gives known top-1 44/48, acceptance 52/54, accepted precision 43/52, and OOD false acceptance 6/6. The 52 accepted rows split into 6 OOD plus 46 known rows. Among those 46 known rows, 43 are top-1 correct and 3 are top-1 wrong yet accepted. Two known rows were rejected: one correct and one wrong. The drop to 43/52 therefore combines six OOD accepts and three known wrong accepts, not a single OOD effect. Run this block directly at the package root. It reads saved test-predictions.jsonl with the Python 3 standard library only, and it does not retrain, infer again, or touch the network: python3 - <<'PY'
import json
with open('test-predictions.jsonl', encoding='utf-8') as f:
rows = [json.loads(line) for line in f if line.strip()]
recipe = next(x for x in rows if x['id'] == 'ood-template-4-en')
ood = [x for x in rows if x['label'] == 'ood']
known_accepted = [x for x in rows if x['label'] != 'ood' and x['accepted']]
known_rejected = [x for x in rows if x['label'] != 'ood' and not x['accepted']]
print(json.dumps({
'recipe_row': {k: recipe[k] for k in ('id', 'top_route', 'probabilities', 'threshold', 'accepted', 'reason')},
'ood_total': len(ood),
'ood_accepted': sum(x['accepted'] for x in ood),
'test_accepted_total': sum(x['accepted'] for x in rows),
'test_known_accepted_correct': sum(x['accepted'] and x['top_route'] == x['label'] for x in rows if x['label'] != 'ood'),
'known_accepted_total': len(known_accepted),
'known_accepted_correct': sum(x['top_route'] == x['label'] for x in known_accepted),
'known_rejected_total': len(known_rejected),
'known_rejected_correct': sum(x['top_route'] == x['label'] for x in known_rejected),
}, ensure_ascii=False, indent=2))
PY
If NumPy 2.5.4 is installed from requirements.txt, replay one CPU inference in the same directory: python router.py model.json "Suggest a soup recipe."
The package boundary matters too. It is a local NumPy CPU command-line model, with no hosted App and no Transformers or inference Provider integration that I have verified; it never executes a weather, calendar, or notes action. For reuse, the code, synthetic data, and trained weights are released under Apache-2.0, copyright Renchao Wu. NumPy is installe...

Powered by Readybot.io