smoke run 9d8d039821fa

Started 2026-07-30T11:39:10+00:00 · status done · 20/20 attempts passed · model cost $0.00 · judge cost $0.00

Offline pipeline demo — every scorer type, no API keys.

Model × prompt matrix

modelpromptpassmean score p50 msp95 mstokenscostjudge
echo-1 direct 100% 1.00 0 0 116 $0.00 $0.00
echo-1 json 100% 1.00 0 0 156 $0.00 $0.00
echo-2 direct 100% 1.00 0 0 116 $0.00 $0.00
echo-2 json 100% 1.00 0 0 156 $0.00 $0.00

Cases

caseattemptspassed
capital-france 8 8/8 drill down →
json-shape 4 4/4 drill down →
judged 8 8/8 drill down →