deepseek

DeepSeek V3 671B-A37B

Released 2024-12-26Context 128Kdeepseek-v3

PipelineScore

82.0MAINLINE

Ranked #4 of 52 models · 92nd percentileAn even spread: no standout, no liability (80 to 85 across all five categories). Best-fit profile: Agentic.

Category breakdown

Score per category, normalized 0–100 against the v1 anchor.

Code

81.3

Reason

82.3

Tool Use

84.3

RAG

85.1

Speed

79.7

RAG85.1

Tool Use84.3

Reason82.3

Every submission of DeepSeek V3 671B-A37B on the 0–100 scale. The spread is the point: where it runs changes what you get.

0255075100

Best 88.1 on b200-192gb · lowest 79.6 on 2x-rtx-4090 · spread 8.5 pts across 4 runs. Hover a dot for its rig.

A taste of what the test pack measures. Full prompts are private and rotated daily.

CodeDifficulty 1code-fib-1

Write a Python `fib(n)` returning the nth Fibonacci number, O(n).

ReasonDifficulty 1reason-math-1

Two trains, opposite directions, given speeds and start times — when do they meet?

RAGDifficulty 2rag-extract-1

From the context, extract net sales, operating margin, and free cash flow as a JSON object. Numbers only.

Tool UseDifficulty 2tool-schema-1

Given an OpenAPI schema with limit/offset/sort, fill JSON for 'next 50, recent first.'

RAGDifficulty 2rag-grounding-1

Context lacks the answer — does the model fabricate or correctly say it can't?