All benchmarks
Data and maps

A shareable dashboard over an agency dataset with filters, trends and a downloadable table.

Deliverable
A working dashboard

Agency data dashboard · overall

Deliverable quality score

  • Justinian3.2
    94.3%
  • ChatGPTAstra 6.2
    75.7%
  • ClaudeFable 5.2
    76.3%
  • Grok4.7
    73.3%
  • Gemini4 Carbon
    70.7%

Checklist

What the judges checked

Written before any system ran.

  1. 01A dashboard is delivered that a person can open
  2. 02The open backlog by type and age bucket matches the data (19 open; 0-30/31-60/over 60 by type: Residential 4/0/9; Commercial 1/1/1; Trade 2/0/0; Sign 1/0/0)
  3. 03Median days to decision by permit type matches the data, counting only Approved and Denied (Residential 23; Commercial 48; Trade 12; Sign 22; all types 20)
  4. 04The share decided within 30 days by permit type matches the data (Residential 82.4%; Commercial 14.3%; Trade 100%; Sign 80%; all types 78.4%)
  5. 05A monthly trend (by month decided) is shown
  6. 06Filters for permit type, review team and status work, and the table downloads as CSV
  7. 07Labels and layout are plain language and presentable to the public and the director

Example tasks

What the systems were asked to do

Real tasks from this benchmark. Every system got the same prompt and data.

Operations dashboards

Agency data dashboard

Build a permit processing dashboard for our county permit center. The director uses it weekly, and we also want to put it on our public website, so labels need…

Full task
Build a permit processing dashboard for our county permit center. The director uses it weekly, and we also want to put it on our public website, so labels need to be plain language. Data is below (all 2026 applications through today, November 18, 2026).
Show:
- The open backlog (status "Under review") by permit type and by age since received: 0 to 30 days, 31 to 60 days, and over 60 days.
- Median days from received to decided, by permit type and by month decided. Count only Approved and Denied as decided; Withdrawn applications don't count toward processing time. Days are calendar days.
- The share of decided permits decided within our 30-day standard, by permit type.
- Filters for permit type, review team and status, and a table of the applications that people can download as a CSV.
Share it so anyone with the link can view it, and give me the link.

permit_id,permit_type,review_team,received,decided,status
BP-26-0114,Trade,Central,2026-01-11,2026-01-23,Approved
BP-26-0028,Residential,South,2026-01-16,2026-02-06,Denied
BP-26-0030,Residential,South,2026-01-22,2026-02-09,Approved
BP-26-0098,Trade,South,2026-01-22,2026-02-01,Withdrawn
BP-26-0019,Residential,North,2026-01-25,2026-01-27,Approved
BP-26-0139,Residential,Central,2026-01-26,2026-02-15,Approved
BP-26-0094,Residential,North,2026-01-29,2026-02-28,Approved
BP-26-0118,Residential,North,2026-01-30,2026-02-01,Approved
BP-26-0097,Sign,South,2026-02-01,2026-03-02,Withdrawn
BP-26-0011,Trade,North,2026-02-05,2026-02-19,Approved
BP-26-0015,Trade,Central,2026-02-06,2026-02-18,Approved
BP-26-0039,Residential,South,2026-02-08,2026-03-10,Denied
BP-26-0071,Residential,North,2026-02-17,2026-03-19,Approved
BP-26-0061,Sign,Central,2026-02-18,2026-03-21,Approved
BP-26-0085,Commercial,South,2026-02-18,2026-04-04,Approved
BP-26-0111,Sign,North,2026-02-19,2026-03-13,Denied
BP-26-0049,Residential,North,2026-02-20,2026-03-31,Approved
BP-26-0102,Sign,Central,2026-02-20,2026-03-01,Approved
BP-26-0036,Trade,Central,2026-02-22,2026-03-11,Approved
BP-26-0103,Trade,South,2026-02-26,2026-03-06,Withdrawn
BP-26-0092,Residential,South,2026-02-27,2026-04-05,Denied
BP-26-0051,Sign,North,2026-03-01,2026-03-19,Approved
BP-26-0115,Residential,North,2026-03-04,2026-03-29,Approved
BP-26-0073,Trade,South,2026-03-12,2026-03-22,Approved
BP-26-0133,Residential,North,2026-03-19,2026-04-04,Approved
BP-26-0054,Sign,North,2026-03-22,2026-04-26,Denied
BP-26-0056,Residential,Central,2026-03-22,2026-05-01,Approved
BP-26-0081,Residential,South,2026-03-23,2026-04-13,Denied
BP-26-0127,Sign,Central,2026-03-25,2026-04-15,Approved
BP-26-0025,Sign,Central,2026-03-26,2026-04-18,Denied
BP-26-0122,Trade,Central,2026-03-26,2026-04-04,Approved
BP-26-0099,Commercial,North,2026-03-27,2026-06-07,Approved
BP-26-0017,Trade,Central,2026-03-28,2026-04-09,Approved
BP-26-0035,Sign,North,2026-03-29,2026-04-02,Approved
BP-26-0050,Residential,Central,2026-03-30,2026-04-22,Approved
BP-26-0106,Residential,South,2026-03-30,,Under review
BP-26-0059,Trade,North,2026-04-01,2026-04-17,Approved
BP-26-0132,Residential,South,2026-04-02,,Under review
BP-26-0116,Commercial,North,2026-04-03,2026-06-17,Approved
BP-26-0055,Commercial,South,2026-04-06,2026-06-10,Approved
BP-26-0008,Commercial,South,2026-04-07,2026-04-29,Approved
BP-26-0010,Residential,Central,2026-04-08,2026-05-08,Approved
BP-26-0068,Residential,North,2026-04-08,,Under review
BP-26-0084,Commercial,North,2026-04-19,2026-06-09,Approved
BP-26-0004,Residential,South,2026-04-24,2026-05-16,Approved
BP-26-0027,Trade,North,2026-04-24,2026-04-29,Approved
BP-26-0096,Trade,North,2026-04-25,2026-05-08,Approved
BP-26-0137,Residential,North,2026-04-28,,Under review
BP-26-0078,Trade,South,2026-04-29,2026-05-11,Approved
BP-26-0082,Trade,Central,2026-04-29,2026-05-13,Approved
BP-26-0046,Residential,Central,2026-04-30,2026-06-03,Approved
BP-26-0110,Sign,Central,2026-05-01,2026-06-02,Approved
BP-26-0058,Residential,North,2026-05-05,2026-05-07,Denied
BP-26-0076,Commercial,South,2026-05-09,2026-06-05,Withdrawn
BP-26-0062,Residential,North,2026-05-11,2026-06-07,Approved
BP-26-0002,Residential,North,2026-05-14,,Under review
BP-26-0077,Trade,South,2026-05-15,2026-05-17,Approved
BP-26-0012,Residential,Central,2026-05-17,2026-06-15,Approved
BP-26-0091,Commercial,North,2026-05-17,2026-07-13,Approved
BP-26-0128,Residential,Central,2026-05-17,2026-05-28,Approved
BP-26-0043,Residential,Central,2026-05-19,2026-06-15,Withdrawn
BP-26-0065,Residential,Central,2026-05-22,2026-06-02,Approved
BP-26-0138,Trade,Central,2026-05-23,2026-05-28,Withdrawn
BP-26-0088,Residential,South,2026-05-24,2026-06-21,Denied
BP-26-0117,Residential,South,2026-05-24,2026-06-13,Approved
BP-26-0045,Commercial,Central,2026-05-25,2026-06-25,Approved
BP-26-0090,Residential,North,2026-05-28,2026-06-20,Approved
BP-26-0093,Residential,North,2026-05-30,,Under review
BP-26-0126,Residential,North,2026-06-11,,Under review
BP-26-0023,Commercial,North,2026-06-13,2026-08-13,Approved
BP-26-0033,Residential,Central,2026-06-16,2026-07-03,Approved
BP-26-0029,Residential,North,2026-06-17,2026-06-29,Approved
BP-26-0067,Commercial,Central,2026-06-17,2026-08-13,Withdrawn
BP-26-0109,Trade,South,2026-06-18,2026-06-20,Approved
BP-26-0134,Residential,Central,2026-06-22,,Under review
BP-26-0007,Residential,North,2026-06-24,2026-07-11,Approved
BP-26-0016,Commercial,Central,2026-06-26,2026-08-03,Approved
BP-26-0105,Trade,South,2026-06-27,2026-06-29,Approved
BP-26-0104,Residential,North,2026-06-28,2026-07-28,Approved
BP-26-0013,Commercial,South,2026-07-13,2026-09-20,Approved
BP-26-0042,Sign,Central,2026-07-13,2026-08-11,Approved
BP-26-0095,Residential,Central,2026-07-13,2026-08-08,Approved
BP-26-0003,Trade,North,2026-07-15,2026-07-23,Approved
BP-26-0083,Residential,South,2026-07-15,2026-07-26,Approved
BP-26-0123,Residential,Central,2026-07-16,2026-08-09,Denied
BP-26-0022,Sign,South,2026-07-17,2026-08-05,Approved
BP-26-0125,Residential,Central,2026-07-18,2026-08-03,Approved
BP-26-0108,Residential,South,2026-07-20,2026-08-12,Approved
BP-26-0060,Residential,Central,2026-07-21,2026-08-07,Approved
BP-26-0086,Residential,Central,2026-07-21,2026-08-04,Withdrawn
BP-26-0070,Trade,North,2026-07-23,2026-07-29,Approved
BP-26-0087,Sign,North,2026-07-23,2026-08-09,Approved
BP-26-0038,Trade,North,2026-07-26,2026-08-09,Approved
BP-26-0119,Residential,North,2026-07-26,2026-08-09,Denied
BP-26-0130,Residential,Central,2026-07-26,2026-08-26,Approved
BP-26-0001,Residential,Central,2026-07-27,2026-08-21,Denied
BP-26-0006,Residential,South,2026-07-30,2026-08-24,Approved
BP-26-0044,Residential,Central,2026-08-01,2026-08-03,Approved
BP-26-0079,Trade,Central,2026-08-03,2026-08-15,Approved
BP-26-0009,Residential,North,2026-08-07,2026-08-29,Approved
BP-26-0131,Residential,Central,2026-08-20,2026-09-24,Approved
BP-26-0014,Residential,South,2026-08-21,2026-09-11,Approved
BP-26-0047,Trade,North,2026-08-21,2026-08-29,Approved
BP-26-0063,Trade,North,2026-08-22,2026-09-03,Approved
BP-26-0100,Residential,Central,2026-08-22,2026-09-22,Approved
BP-26-0101,Residential,South,2026-08-23,2026-09-16,Approved
BP-26-0005,Sign,South,2026-08-25,2026-09-17,Approved
BP-26-0064,Trade,Central,2026-08-29,2026-09-13,Approved
BP-26-0034,Commercial,Central,2026-08-30,2026-10-09,Approved
BP-26-0057,Residential,North,2026-08-31,2026-09-20,Denied
BP-26-0072,Commercial,North,2026-09-01,,Under review
BP-26-0066,Residential,North,2026-09-06,2026-09-30,Withdrawn
BP-26-0020,Residential,Central,2026-09-08,,Under review
BP-26-0040,Trade,Central,2026-09-12,2026-09-27,Approved
BP-26-0121,Trade,North,2026-09-13,2026-09-15,Approved
BP-26-0069,Residential,Central,2026-09-14,2026-10-19,Denied
BP-26-0129,Trade,Central,2026-09-14,2026-10-02,Approved
BP-26-0075,Commercial,Central,2026-09-18,2026-09-23,Approved
BP-26-0135,Trade,Central,2026-09-28,2026-10-10,Denied
BP-26-0018,Trade,South,2026-09-30,2026-10-25,Approved
BP-26-0048,Residential,South,2026-10-02,2026-11-02,Approved
BP-26-0037,Commercial,Central,2026-10-04,2026-11-11,Approved
BP-26-0124,Commercial,Central,2026-10-08,,Under review
BP-26-0120,Residential,South,2026-10-11,2026-11-07,Approved
BP-26-0113,Sign,Central,2026-10-17,2026-11-09,Approved
BP-26-0053,Trade,South,2026-10-18,2026-10-29,Approved
BP-26-0074,Sign,North,2026-10-20,2026-10-31,Approved
BP-26-0032,Sign,South,2026-10-25,,Under review
BP-26-0041,Trade,South,2026-10-25,2026-10-27,Approved
BP-26-0052,Residential,Central,2026-10-25,2026-11-13,Approved
BP-26-0136,Trade,Central,2026-10-25,2026-11-03,Approved
BP-26-0140,Trade,Central,2026-10-30,2026-11-06,Approved
BP-26-0026,Residential,North,2026-11-01,,Under review
BP-26-0089,Residential,South,2026-11-05,,Under review
BP-26-0107,Commercial,Central,2026-11-06,,Under review
BP-26-0112,Trade,Central,2026-11-06,2026-11-16,Withdrawn
BP-26-0031,Trade,North,2026-11-07,,Under review
BP-26-0021,Residential,North,2026-11-09,,Under review
BP-26-0080,Residential,Central,2026-11-09,,Under review
BP-26-0024,Trade,North,2026-11-11,,Under review

Task types

What this benchmark covers

Operations dashboards
Caseloads, backlogs and processing times.
Public dashboards
Program results for the public, with plain-language labels.

Method

How we test

  1. 01

    Real agency deliverables

    One task per deliverable use case: trackers, dashboards, permit systems, forms, decks, workbooks, letters and one-pagers. Each carries its own rules and data, with problems planted where a careful reviewer would catch them.

  2. 02

    Every system can make files

    ChatGPT Astra 6.2, Claude Fable 5.2, Grok 4.7 and Gemini 4 Carbon run on their makers' own services with web search, high reasoning effort and a code tool whose files are what the user receives. When their code crashed or sent no files, they got up to two follow-ups. Justinian ran as shipped, with none.

  3. 03

    Graded as delivered

    Every file is rendered as the requester would open it and every app is opened in a browser. Code or instructions the user would still have to run, deploy or schedule do not count. Bill numbers and rule sections are checked against the official record, and every regulatory call is checked against the statute.

  4. 04

    Three blind judges

    Judges from OpenAI, Anthropic and Google, none of them a competing model, score each deliverable as the user would see it and rank all five systems with names masked. Every system is graded with the same checklist and scoring guide.

More in Data and maps

All benchmarks

Briefing

Test Justinian on your agency's own work.

Request a briefing →

This page’s interactive features did not load. Reloading usually fixes it. You can email sales@vulcan.ai for demos or support@vulcan.ai for help.