Code Llama 34B Instruct
Category breakdown
Score per category, normalized 0–100 against the v1 anchor.
Strengths
Same model, different rigs
Every submission of Code Llama 34B Instruct on the 0–100 scale. The spread is the point: where it runs changes what you get.
Best 69.5 on m3-max-64gb · lowest 63.6 on rtx-4080-16gb-offload · spread 5.8 pts across 4 runs. Hover a dot for its rig.
Sample tasks
A taste of what the test pack measures. Full prompts are private and rotated daily.
Fibonacci function
Write a Python `fib(n)` returning the nth Fibonacci number, O(n).
Train meeting time
Two trains, opposite directions, given speeds and start times — when do they meet?
Extract metrics to JSON
From the context, extract net sales, operating margin, and free cash flow as a JSON object. Numbers only.
OpenAPI param selection
Given an OpenAPI schema with limit/offset/sort, fill JSON for 'next 50, recent first.'
Refuses to fabricate
Context lacks the answer — does the model fabricate or correctly say it can't?