Agency records joined to boundaries, facilities and permits, and turned into a map staff can share.
- Deliverable
- A working, shareable map
GIS mapping · overall
Deliverable quality score
- Justinian3.2A94.7%
- ChatGPTAstra 6.2B+85.3%
- ClaudeFable 5.2B78%
- Grok4.7B+85.7%
- Gemini4 CarbonF20.3%
Checklist
What the judges checked
Written before any system ran.
- 01A map is delivered that a person can open, with all 30 facilities plotted at their coordinates
- 02FEMA's flood hazard zones from the National Flood Hazard Layer are shown on the map, at least the Special Flood Hazard Area
- 03The city limits of Baton Rouge, Baker, Central and Zachary are shown on the map
- 04Each facility's Special Flood Hazard Area status matches FEMA's National Flood Hazard Layer (10 of 30 are in it: CARRIAGEWOOD ESTATES, COPPER MILL GOLF COMMUNITY, EXCLUSIVE HOMES BY CHUCK WALTERS LLC, M&J CIVIL CONSTRUCTION LLC - SPECIALTY SURGICAL HOSPITAL, OAK CLUSTER SUBDIVISION, SANTA MARIA, ST. PIERRE PROPERTY, TJC BUILDERS LLC- THE SETTLEMENT @ WILLOW GROVE, TRACY JORDAN LLC, UNIFIED CONSTRUCTION GROUP LLC) (automated check of the spreadsheet)
- 05Each facility's flood zone and zone subtype match the National Flood Hazard Layer (X (AREA OF MINIMAL FLOOD HAZARD) 10; X (AREA WITH REDUCED FLOOD RISK DUE TO LEVEE) 9; AE 9; A 1; X (0.2 PCT ANNUAL CHANCE FLOOD HAZARD) 1) (automated check of the spreadsheet)
- 06Each facility's jurisdiction matches the Census 2025 city boundaries (all 30: Baton Rouge 11; Central 5; Zachary 3; St. George 6; Unincorporated parish 5). Key correction before grading: the request listed four cities, but St. George (incorporated 2024) is also a city in the Census 2025 boundaries, and 6 facilities are inside it (automated check of the spreadsheet)
- 07The summary table of facilities in the Special Flood Hazard Area by jurisdiction is correct (Baton Rouge 2 of 11; Central 3 of 5; Zachary 1 of 3; St. George 4 of 6; Unincorporated parish 0 of 5)
- 08Facilities in the Special Flood Hazard Area are clearly marked, each facility has a popup with its name, flood zone and jurisdiction, and the map is clear enough for staff to use without training
Agencies that use it
- Transportation, Facilities & Infrastructure
In the field
What agencies did with it
Results agencies reported from their own work in Justinian.
Arkansas Department of Energy and Environment
ConsultingManaging agency data in-house
- database migration quote avoided
- $150K
- database migration quote avoided
- GIS analyst position no longer needed
- $77K/yr
- GIS analyst position no longer needed
Example tasks
What the systems were asked to do
Real tasks from this benchmark. Every system got the same prompt and data.
Record-to-boundary joins
GIS mapping
I'm with the East Baton Rouge Parish floodplain management office. We're checking which EPA-permitted wastewater and stormwater (NPDES) facilities in the…
Full task
I'm with the East Baton Rouge Parish floodplain management office. We're checking which EPA-permitted wastewater and stormwater (NPDES) facilities in the parish sit in FEMA's Special Flood Hazard Area, and whose floodplain rules apply to each one. The facility list below is from EPA's ECHO database (FRS registry ID, address and EPA's coordinates). For each facility, tell me its current FEMA flood zone from FEMA's National Flood Hazard Layer (for zone X, include the subtype, such as reduced risk due to levee or 0.2 percent annual chance), whether it is in the Special Flood Hazard Area, and whether it is inside the city limits of Baton Rouge, Baker, Central or Zachary or in the unincorporated parish, using the Census Bureau's 2025 city boundaries. Deliver: 1. A map our staff can open that shows FEMA's flood hazard zones, the four city limits, and the facilities, with facilities in the Special Flood Hazard Area clearly marked and a popup for each facility with its name, flood zone and jurisdiction. Share it so anyone with the link can view it, and give me the link. 2. A summary table of how many facilities in each jurisdiction are in the Special Flood Hazard Area. 3. A spreadsheet with one row per facility: registry ID, name, flood zone, zone subtype, in the Special Flood Hazard Area (yes or no) and jurisdiction. frs_registry_id,facility_name,street_address,mailing_city,zip,latitude,longitude 110011146937,"BREC'S BATON ROUGE ZOO","3601 THOMAS ROAD",BATON ROUGE,70807,30.563280,-91.158450 110037932146,"THE OAKS","TBD HIGHLAND OAKS DR.",ZACHARY,70000,30.662578,-91.158133 110056151122,"NOTO DEVELOPMENT LLC - UNIVERSITY CLUB PLANTATIONS","11959 NICHOLSON DR",BATON ROUGE,70810,30.345160,-91.139970 110041910260,"COPPER MILL GOLF COMMUNITY","2438 CARNOUSTIE DR.",ZACHARY,70791,30.634540,-91.182630 110071545181,"M&J CIVIL CONSTRUCTION LLC - SPECIALTY SURGICAL HOSPITAL","6330 MOURNING DOVE DR",BATON ROUGE,70808,30.392083,-91.037806 110071729438,"BLUE SKY PROPERTIES JLU, LLC - LION'S LAWN CARE & LANDSCAPE LLC","430 SOUTH STEVENDALE RD",BATON ROUGE,70819,30.463300,-91.003620 110038516629,"OAK CLUSTER SUBDIVISION","6442 OAK CLUSTER DR.",CENTRAL,70739,30.509810,-91.009760 110070235173,"TJC BUILDERS LLC- THE SETTLEMENT @ WILLOW GROVE","OFF PERKINS RD",BATON ROUGE,70817,30.375109,-90.980419 110071896602,"DOLLAR TREE STORES, INC. - FAMILY DOLLAR #28831","801 O'NEAL LANE",BATON ROUGE,70808,30.457220,-91.008410 110054107195,"THE SETTLEMENT AT WILLOW GROVE","BLUEBONNET BLVD & PERKINS RD",BATON ROUGE,70808,30.379250,-91.097970 110040438559,"TRACY JORDAN LLC","3549 FAIRWOOD",BATON ROUGE,70805,30.481210,-91.141660 110030757704,"BLD BUILDERS, INC.","AMBER LAKES SUBDIVISION",GREENWELL SPRINGS,70739,30.567770,-91.002990 110042382579,"SANTA MARIA","19634 FAIRWAY OAKS AVE.",BATON ROUGE,70817,30.337820,-91.001930 110030997205,"MENZIE STONE COMPANY, INC.","11145 DARRYL DRIVE",BATON ROUGE,70815,30.471010,-91.059890 110072201866,"ENGQUIST-ROUZAN COMMERCIAL DEVELOPMENT LLC - ROUZAN","SE CORNER OF PERKINS RD & GALSGOW AVE.",BATON ROUGE,70806,30.448797,-91.124704 110011176066,"COUNTRY AIRE MOBILE HOME PARK","21020 HWY 19",ZACHARY,70791,30.649627,-91.157635 110044892621,"GREENTRAILS","15554 PARKSIDE COURT",BATON ROUGE,70808,30.394540,-91.010860 110070626005,"THE LEMOINE COMPANY, LLC - GEO NEXT GENERATION HIGH SCHOOL","2121 N SHERWOOD FOREST BLVD",BATON ROUGE,70817,30.468180,-91.058090 110055631534,"UNIFIED CONSTRUCTION GROUP LLC","2136 SUGAR CANE LN",BATON ROUGE,70000,30.454590,-91.069520 110041247610,"DEMCO/THOMPSON CREEK SUBDIVISION","22520 SAMUELS RD (US HWY 61)",ZACHARY,70791,30.688281,-91.267845 110043742073,"ST. PIERRE PROPERTY","LIBERY & ALPHONSE FORBES RD",PRIDE,70000,30.597070,-91.020809 110022464135,"CVS PHARMACY, INC.","14190 GREENWELL SPRINGS RD",GREENWELL SPRINGS,70739,30.511660,-91.029710 110012671901,"EXTREME CLEAN CAR WASH","LEE DRIVE LLC",BATON ROUGE,70808,30.397136,-91.156217 110071444171,"ARKEL CONSTRUCTION LLC - ALL STAR AUTOMOTIVE USED CARS","10313 AIRLINE HWY",BATON ROUGE,70802,30.416820,-91.070020 110069423877,"EXCLUSIVE HOMES BY CHUCK WALTERS LLC","THE VILLAGE AT MAGNOLIA SQUARE",CENTRAL,70000,30.559305,-91.035507 110032945829,"ST.LOUIS POINTE RESIDECES LP","STAN AVE. &",BATON ROUGE,70805,30.470090,-91.054430 110070050269,"CIRCLE K STORES INC.","22015 S SHERWOOD FOREST BLVD",BATON ROUGE,70808,30.432500,-91.056944 110043729132,"CARRIAGEWOOD ESTATES","18515 SURREY COURT AVE. (LOT 39)",BATON ROUGE,70808,30.372040,-90.982380 110064429523,"THE HAMILTON FAMILY LLC","6575 AIRLINE HWY",BATON ROUGE,70808,30.492981,-91.124444 110070112331,"PLAIN CONSTRUCTION LLC - CARRIAGEWOOD ESTATES","OFF VIGNES RD",BATON ROUGE,70805,30.488923,-91.158261
Task types
What this benchmark covers
- Record-to-boundary joins1 task
- Placing records in the correct county, district or watershed.
- Facility and permit overlays2 tasks
- Facilities, permits and enforcement actions on one map, from public records.
- Area summaries1 task
- Counts and totals by county or district, matching the underlying records.
- Shareable output1 task
- A map a colleague can open, filter and cite without rebuilding it.
Method
How we test
01
Real agency deliverables
One task per deliverable use case: trackers, dashboards, permit systems, forms, decks, workbooks, letters and one-pagers. Each carries its own rules and data, with problems planted where a careful reviewer would catch them.
02
Every system can make files
ChatGPT Astra 6.2, Claude Fable 5.2, Grok 4.7 and Gemini 4 Carbon run on their makers' own services with web search, high reasoning effort and a code tool whose files are what the user receives. When their code crashed or sent no files, they got up to two follow-ups. Justinian ran as shipped, with none.
03
Graded as delivered
Every file is rendered as the requester would open it and every app is opened in a browser. Code or instructions the user would still have to run, deploy or schedule do not count. Bill numbers and rule sections are checked against the official record, and every regulatory call is checked against the statute.
04
Three blind judges
Judges from OpenAI, Anthropic and Google, none of them a competing model, score each deliverable as the user would see it and rank all five systems with names masked. Every system is graded with the same checklist and scoring guide.
More in Data and maps
All benchmarksBriefing
