A designed one-page brief for a director or legislator, with the key figures charted.
- Deliverable
- A designed one-page PDF
Leadership one-pager · overall
Deliverable quality score
- Justinian3.2A93.3%
- ChatGPTAstra 6.2B+85%
- ClaudeFable 5.2B+85.7%
- Grok4.7D-45%
- Gemini4 CarbonD-44.3%
Checklist
What the judges checked
Written before any system ran.
- 01A PDF of exactly one page is delivered
- 02At least one real chart of the data is included and it is accurate
- 03All figures used match the data provided
- 04A clear recommendation (statewide online ordering) with cost and the unspent-funds argument is stated
- 05Design is clean and professional; text is readable at print size
Example tasks
What the systems were asked to do
Real tasks from this benchmark. Every system got the same prompt and data.
Program briefs
Leadership one-pager
Make a designed one-page PDF brief for our Secretary of Health, who meets the Governor tomorrow about the WIC program. Data: participants 2023 112,400; 2024…
Full task
Make a designed one-page PDF brief for our Secretary of Health, who meets the Governor tomorrow about the WIC program. Data: participants 2023 112,400; 2024 108,900; 2025 104,300; 2026 101,700. Eligible but not enrolled 2026: 61,000 (est.). Redemption rate of food benefits 2026: 71% (national 79%). Clinics: 96, down from 104 in 2023. Pilot of online ordering in 6 counties raised redemption from 68% to 80% in 9 months. Cost to expand online ordering statewide: $1.9 million one-time, $420,000 per year. Federal food funds unspent in FY2026 because of low redemption: $14.2 million. Include the key figures charted and a clear recommendation.
Task types
What this benchmark covers
- Program briefs
- What a program does, what it costs and what it achieved.
- Issue briefs
- A decision a leader faces, the options and a recommendation.
Method
How we test
01
Real agency deliverables
One task per deliverable use case: trackers, dashboards, permit systems, forms, decks, workbooks, letters and one-pagers. Each carries its own rules and data, with problems planted where a careful reviewer would catch them.
02
Every system can make files
ChatGPT Astra 6.2, Claude Fable 5.2, Grok 4.7 and Gemini 4 Carbon run on their makers' own services with web search, high reasoning effort and a code tool whose files are what the user receives. When their code crashed or sent no files, they got up to two follow-ups. Justinian ran as shipped, with none.
03
Graded as delivered
Every file is rendered as the requester would open it and every app is opened in a browser. Code or instructions the user would still have to run, deploy or schedule do not count. Bill numbers and rule sections are checked against the official record, and every regulatory call is checked against the statute.
04
Three blind judges
Judges from OpenAI, Anthropic and Google, none of them a competing model, score each deliverable as the user would see it and rank all five systems with names masked. Every system is graded with the same checklist and scoring guide.
More in Workforce and operations
All benchmarksBriefing
