Spending against appropriation by program and month from the agency's ledger, with variances flagged.
- Deliverable
- A shareable dashboard
Budget dashboard · overall
Deliverable quality score
- Justinian3.297%
- ChatGPTAstra 6.273.3%
- ClaudeFable 5.275.3%
- Grok4.774%
- Gemini4 Carbon69.3%
Checklist
What the judges checked
Written before any system ran.
- 01A dashboard is delivered that a person can open
- 02Year-to-date spending by program is computed correctly from the ledger
- 03Exactly the programs more than 10% off pace are flagged (Public health, Road maintenance, Senior services)
- 04Monthly trend by program is shown
- 05Program and fund filters work
- 06It is clear and presentable to elected officials
Example tasks
What the systems were asked to do
Real tasks from this benchmark. Every system got the same prompt and data.
Spend against budget
Budget dashboard
Build a budget dashboard for our county commissioners from the ledger below (January to September 2026). Show spending against budget by program and by month,…
Full task
Build a budget dashboard for our county commissioners from the ledger below (January to September 2026). Show spending against budget by program and by month, year to date, and flag any program whose year-to-date spending is more than 10% above or below its pro-rated budget (9/12 of the annual budget). Include filters for program and fund. Share it so anyone with the link can view it, and give me the link. program,fund,month,approved_budget_annual,spent Road maintenance,General,2026-01,900000,51904 Road maintenance,General,2026-02,900000,49257 Road maintenance,General,2026-03,900000,47592 Road maintenance,General,2026-04,900000,55369 Road maintenance,General,2026-05,900000,45583 Road maintenance,General,2026-06,900000,53941 Road maintenance,General,2026-07,900000,57735 Road maintenance,General,2026-08,900000,55247 Road maintenance,General,2026-09,900000,50504 Parks,Special Revenue,2026-01,900000,71035 Parks,Special Revenue,2026-02,900000,68932 Parks,Special Revenue,2026-03,900000,85642 Parks,Special Revenue,2026-04,900000,83768 Parks,Special Revenue,2026-05,900000,85024 Parks,Special Revenue,2026-06,900000,76681 Parks,Special Revenue,2026-07,900000,88386 Parks,Special Revenue,2026-08,900000,83459 Parks,Special Revenue,2026-09,900000,86573 Libraries,General,2026-01,900000,72748 Libraries,General,2026-02,900000,77004 Libraries,General,2026-03,900000,80462 Libraries,General,2026-04,900000,87097 Libraries,General,2026-05,900000,86354 Libraries,General,2026-06,900000,76949 Libraries,General,2026-07,900000,88468 Libraries,General,2026-08,900000,72316 Libraries,General,2026-09,900000,86848 Public health,General,2026-01,900000,49034 Public health,General,2026-02,900000,56813 Public health,General,2026-03,900000,47036 Public health,General,2026-04,900000,58187 Public health,General,2026-05,900000,51057 Public health,General,2026-06,900000,57372 Public health,General,2026-07,900000,54382 Public health,General,2026-08,900000,47399 Public health,General,2026-09,900000,55484 Code enforcement,General,2026-01,900000,69907 Code enforcement,General,2026-02,900000,68419 Code enforcement,General,2026-03,900000,68820 Code enforcement,General,2026-04,900000,84993 Code enforcement,General,2026-05,900000,79246 Code enforcement,General,2026-06,900000,64239 Code enforcement,General,2026-07,900000,69847 Code enforcement,General,2026-08,900000,77654 Code enforcement,General,2026-09,900000,85962 Senior services,Special Revenue,2026-01,3100000,354544 Senior services,Special Revenue,2026-02,3100000,358931 Senior services,Special Revenue,2026-03,3100000,315481 Senior services,Special Revenue,2026-04,3100000,354262 Senior services,Special Revenue,2026-05,3100000,320732 Senior services,Special Revenue,2026-06,3100000,336615 Senior services,Special Revenue,2026-07,3100000,294630 Senior services,Special Revenue,2026-08,3100000,280064 Senior services,Special Revenue,2026-09,3100000,298766
Task types
What this benchmark covers
- Spend against budget
- Year to date by program and fund.
- Variance flags
- Lines over or under plan beyond a set threshold.
Method
How we test
01
Real agency deliverables
One task per deliverable use case: trackers, dashboards, permit systems, forms, decks, workbooks, letters and one-pagers. Each carries its own rules and data, with problems planted where a careful reviewer would catch them.
02
Every system can make files
ChatGPT Astra 6.2, Claude Fable 5.2, Grok 4.7 and Gemini 4 Carbon run on their makers' own services with web search, high reasoning effort and a code tool whose files are what the user receives. When their code crashed or sent no files, they got up to two follow-ups. Justinian ran as shipped, with none.
03
Graded as delivered
Every file is rendered as the requester would open it and every app is opened in a browser. Code or instructions the user would still have to run, deploy or schedule do not count. Bill numbers and rule sections are checked against the official record, and every regulatory call is checked against the statute.
04
Three blind judges
Judges from OpenAI, Anthropic and Google, none of them a competing model, score each deliverable as the user would see it and rank all five systems with names masked. Every system is graded with the same checklist and scoring guide.
More in Grants, contracts and budgets
All benchmarksBriefing
