Simulation + Code
Download everything as one zip ({size_mb:.1f} MB).
"""Public "Code" page -> build/site/code.html + build/site/code/ (raw files) + a zip of the package.
Every program, settings file and data file the pipeline uses, grouped by workflow step, each readable in the page
(collapsible, with line numbers), viewable raw, and all downloadable as one zip. The County PDFs are NOT published
(obtained by email); their parsed contents are.
"""
import html, pathlib, shutil, zipfile, datetime, sys
sys.path.insert(0, str(pathlib.Path(__file__).parent))
from common import ROOT, BUILD
SITE = BUILD/"site"; RAW = SITE/"code"
E = html.escape
GROUPS = [
("Run everything", "How the whole workflow is started, and what it needs.", [
("run_all.sh", "The one command: runs every step below in order and rebuilds all results."),
("README.md", "Instructions: how to rerun, how the pieces fit, how to change things."),
("requirements.txt", "Software the pipeline needs (SUMO, Python libraries), with exact versions."),
]),
("Settings", "Every choice in plain files: which lights, which hour, every assumption with its evidence.", [
("config/corridor.toml", "Which four lights, which count stations feed each approach, lanes, design hour, timing pattern, simulation options."),
("config/assumptions.toml", "Every value not measured (turn shares fallback, saturation flow, model defaults) with the reason for it."),
]),
("Step 1 · Read and check the County data", "Turns the County documents into data tables and checks every number.", [
("src/parse_county_counts.py", "Reads the hourly turning counts from the County count sheets (text-based sheets)."),
("src/parse_county_timing.py", "Reads the timing sheets: phases, yellow/all-red, cycle, splits, offsets for every pattern."),
("src/verify_counts.py", "Re-adds every count row and column and compares with the totals printed on the sheets."),
]),
("Step 2 · Build the master data file", "One file with every number, each labelled measured / derived / assumed.", [
("src/derive.py", "Combines County counts and timing, MoDOT and map data into corridor.json; computes lost time and capacity."),
]),
("Step 3 · Hand-off to the math team", "The data file the optimizer reads.", [
("src/to_optimizer.py", "Writes the optimizer input (volumes, capacities, travel times, parameters). Only file that knows their input format."),
]),
("Step 4 · Build the virtual street", "The SUMO road network.", [
("src/build_network.py", "Builds the network: the generated corridor (default) or the full OpenStreetMap network."),
("src/build_network_gen.py", "Generates the corridor from the data: 4 signals at real spacing, real lanes, left-turn bays, speeds."),
("src/common.py", "Shared helpers: file paths, loading the network, labelling each approach as NB/SB/EB/WB."),
]),
("Step 5 · Fill it with traffic", "Virtual cars that match the real counts.", [
("src/build_demand.py", "Entry flows, measured turn shares at each light, balancing flows between lights, routes."),
]),
("Step 6 · The timing plans", "Every light timing we test.", [
("src/plans.py", "Plan format and the baselines: County current, equal split, Webster, SUMO coordination, formula optimum."),
("src/formula.py", "The delay formula (Webster + platoon) and its optimizer: our stand-in for the math team's code."),
]),
("Step 7 · Run the simulations", "Each plan, 10 runs, delay measured.", [
("src/run_sumo.py", "Runs SUMO for each plan and seed in parallel; measures delay, stops, throughput."),
("src/plan_to_sumo.py", "Turns a plan into SUMO signal programs (County phase sequence or simple two-phase)."),
("src/nema.py", "Builds the County's real phase sequence (incl. left arrows) scaled to a plan's timing."),
]),
("Step 8 · Check against reality", "Simulated vs counted traffic.", [
("src/validate.py", "Compares simulated volumes with the counts (GEH score) at every approach."),
]),
("Study 1", "Formula model vs optimizing inside the simulation.", [
("src/study1.py", "Part A: 30 random plans, formula vs simulation ranking. Part B: Nelder–Mead search using the simulation."),
]),
("Browser mini-simulator", "The simplified simulation that runs on this page, and how it is checked against SUMO.", [
("sim/core.js", "The engine: cars follow the Krauss rule (SUMO's default), lights run the County sequence; runs in the browser and in Node."),
("sim/ui.js", "Drawing (corridor + intersection close-up), controls and live counters on this page."),
("src/sim_export.py", "Exports the corridor, County phases, timing plans and SUMO results the simulator reads (sim/data.json)."),
("sim/validate.js", "Runs every plan through the mini-simulator and compares delays and ranking with SUMO."),
]),
("Step 9 · Tables, charts and this website", "Everything you see on the site is generated by these.", [
("src/report_tables.py", "Report tables (provenance, volumes, timing, results, validation) and figures."),
("src/site_page.py", "Generates the main website page from the results."),
("src/site_figs.py", "Draws the map and charts for the website."),
("src/site_explain.py", "The plain-English 'Explain this section' texts."),
("src/code_page.py", "Generates this page (Simulation + Code)."),
("src/review_page.py", "A local review page with every table (REVIEW.html)."),
]),
("Data used (parsed from the County documents)", "The measured inputs, as the pipeline reads them.", [
("data/measured/county_timing.json", "All four timing sheets, parsed: phases, yellow/all-red, every pattern's cycle, splits, offset."),
("data/measured/county-counts/034-clayton-rd-2026-06-24.csv", "Clayton Rd turning counts, hourly (parsed from the sheet's text)."),
("data/measured/county-counts/063-wydown-2025-09-26.csv", "Wydown turning counts, hourly (parsed from the sheet's text)."),
("data/measured/county-counts/040-forsyth-2017-03-03.csv", "Forsyth turning counts, hourly (typed from the scanned sheet, verified)."),
("data/measured/county-counts/051-forest-park-pkwy-2021-06-30.csv", "Forest Park Pkwy counts, 5–6 PM by 15 min (typed from the scan, verified)."),
("data/raw/county/county-signals.csv", "St. Louis County signal inventory near the corridor (IDs, locations, owner)."),
("data/raw/modot/modot_aadt_2025_corridor.csv", "MoDOT 2025 road segments: daily traffic, lanes, speed limits."),
]),
]
ARC = "https://services2.arcgis.com/w657bnjzrjguNyOy/arcgis/rest/services/STLCO_Transportation_Assets_File_Geodatabase_All_Layers_View/FeatureServer"
ORIGINALS = [
("From St. Louis County Transportation & Public Works (Traffic Operations), by email, November 17, 2026",
"The four documents exactly as received: signal timing sheets plus turning-movement counts. Byte-for-byte the email attachments.", [
("data/measured/county-timing/051-forest-park-pkwy.pdf", "Big Bend Blvd & Forest Park Pkwy.pdf", "Signal #051: timing sheet (Oct 21, 2026) + counts (Feb 5 / Jun 30, 2021, scanned)", None),
("data/measured/county-timing/040-forsyth.pdf", "Big Bend Blvd & Forsyth Blvd.pdf", "Signal #040: timing sheet (Dec 10, 2024) + counts (Mar 3, 2017, scanned)", None),
("data/measured/county-timing/063-wydown.pdf", "Big Bend Blvd & Wydown Blvd.pdf", "Signal #063: timing sheet (Dec 10, 2024) + counts (Sep 26, 2025)", None),
("data/measured/county-timing/034-clayton-rd.pdf", "Big Bend Blvd & Clayton Rd.pdf", "Signal #034: timing sheet (May 12, 2026) + counts (Jun 24-25, 2026)", None),
]),
("Downloaded from public online sources, November 17, 2026",
"Exactly as saved from each service (JSON / XML as returned), with the address it came from.", [
("data/raw/county/county-signals-raw.json", None, "St. Louis County signal inventory near the corridor (Signals layer), raw JSON",
ARC+"/40/query?where=1%3D1&geometry=-90.340,38.626,-90.298,38.660&geometryType=esriGeometryEnvelope&inSR=4326&outSR=4326&spatialRel=esriSpatialRelIntersects&outFields=*&f=json"),
("data/raw/county/county-count-locations-raw.json", None, "St. Louis County traffic count stations near the corridor (latest counts), raw JSON",
ARC+"/13/query?where=1%3D1&geometry=-90.340,38.626,-90.298,38.660&geometryType=esriGeometryEnvelope&inSR=4326&outSR=4326&spatialRel=esriSpatialRelIntersects&outFields=*&f=json"),
("data/raw/county/county-count-history-raw.json", None, "St. Louis County count history for the corridor stations, raw JSON",
ARC+"/14/query?where=Location_ID+IN+(119,1063,132,123,1064,1062,117,118,388,387,296)&outFields=*&orderByFields=Location_ID,Count_Year&f=json"),
("data/raw/county/stlco_traffic_counts_corridor.csv", None, "Same County count stations, wider area, saved as CSV (converted from the service's JSON)",
"https://services2.arcgis.com/w657bnjzrjguNyOy/arcgis/rest/services/STLCO_Transportation_Assets_File_Geodatabase_Traffic_Counts_Public_Display/FeatureServer/13"),
("data/raw/modot/modot_aadt_2025_corridor.csv", None, "MoDOT 2025 traffic segments near the corridor (daily traffic, lanes, speed), saved as CSV from the service",
"https://mapping.modot.mo.gov/arcgis/rest/services/BusinessInt/TrafficInfoSegAADT/MapServer"),
("data/raw/osm/bigbend.osm", None, "OpenStreetMap data for the corridor (roads, lanes, signals), raw XML (© OpenStreetMap contributors, ODbL)",
"https://api.openstreetmap.org/api/0.6/map?bbox=-90.3235,38.6315,-90.3105,38.6545"),
("data/raw/osm/context/t1.osm", None, "OpenStreetMap, wider area for the map (tile 1 of 5), raw XML", "https://api.openstreetmap.org/api/0.6/map?bbox=-90.336,38.636,-90.318,38.6465"),
("data/raw/osm/context/t2.osm", None, "OpenStreetMap, wider area (tile 2 of 5)", "https://api.openstreetmap.org/api/0.6/map?bbox=-90.318,38.636,-90.300,38.6465"),
("data/raw/osm/context/t3.osm", None, "OpenStreetMap, wider area (tile 3 of 5)", "https://api.openstreetmap.org/api/0.6/map?bbox=-90.336,38.6465,-90.318,38.657"),
("data/raw/osm/context/t4.osm", None, "OpenStreetMap, wider area (tile 4 of 5)", "https://api.openstreetmap.org/api/0.6/map?bbox=-90.318,38.6465,-90.300,38.657"),
("data/raw/osm/context/t5.osm", None, "OpenStreetMap, wider area (tile 5 of 5)", "https://api.openstreetmap.org/api/0.6/map?bbox=-90.326,38.627,-90.306,38.636"),
]),
]
SOURCE_PAGES = [
("St. Louis County: Traffic Studies and Maps (counts map)", "https://stlouiscountymo.gov/st-louis-county-departments/transportation-and-public-works/operations-division/traffic-studies-and-maps/"),
("St. Louis County: Traffic Signal Network", "https://stlouiscountymo.gov/st-louis-county-departments/transportation-and-public-works/operations-division/traffic-signal-network/"),
("St. Louis County public GIS data (signals, count stations)", ARC),
("MoDOT traffic volume map service", "https://mapping.modot.mo.gov/arcgis/rest/services/BusinessInt/TrafficInfoSegAADT/MapServer"),
("OpenStreetMap", "https://www.openstreetmap.org/#map=16/38.6430/-90.3165"),
]
def sim_html():
import json
for f in ("core.js", "ui.js"):
shutil.copy(ROOT/"sim"/f, SITE/"sim"/f)
shots = sorted((ROOT/"data/screenshots").glob("sumo-*.png"))
(SITE/"shots").mkdir(exist_ok=True)
caps = {"sumo-1.png": "The whole simulated corridor in SUMO, north at the top: Forest Park Pkwy, Forsyth, Wydown, Clayton Rd (the short cross-bars between lights are the mid-block side-street access points).",
"sumo-2.png": "Close-up of one of the four intersections at rush hour: queues on Big Bend waiting at red, cross-street traffic moving.",
"sumo-3.png": "SUMO's own window (toolbar, clock, log) showing an intersection with queues on both streets.",
"sumo-4.png": "Another intersection close-up in SUMO: long queues on Big Bend and on the cross street.",
"sumo-5.png": "A quieter moment at one of the lights, cross-street traffic queued at the stop line."}
gal = ""
for s in shots:
shutil.copy(s, SITE/"shots"/s.name)
gal += f'
A simplified version of our SUMO model that runs right here in your browser: the same four lights, real County traffic counts and turn shares, the same timing plans.
A simplified version of our SUMO model, written for this page. Same four lights, real County counts and turn shares, same timing plans and signal programs as SUMO.
Cars: drive at the speed limit, keep a safe gap (SUMO's default rule), stop at red.
Use: pick a plan, press Play. Left: whole corridor (red = stopped). Right: one intersection up close. "Custom timing" sets your own cycle and Big Bend green share. "Run the full hour instantly" simulates 5–6 PM in ~1 s and compares with SUMO.
Limits: no lane-changing tactics or vehicle types, so delays are lower than SUMO's. It ranks plans similarly (table below).
| Plan | Simulated time | ||
|---|---|---|---|
| Cars on the road | Cars waiting to enter | ||
| Delay so far (veh-h per hour) | Cars through per hour |
| plan | cycle (s) | mini-sim delay | SUMO delay |
|---|
Screenshots of SUMO's own viewer running our simulated corridor at rush hour (County's current timing). Click to enlarge.
Every plan run through both, full rush hour ({V['seeds']} runs each for the mini-simulation, 10 for SUMO). Delay in vehicle-hours per hour; rank 1 = least delay.
| plan | mini-sim delay | mini-sim rank | SUMO delay | SUMO rank |
|---|
Rank agreement (Spearman): {V['spearman']:.2f} (1 = identical order). Both put the optimized plans first and equal split last; the mini-simulation is kinder to the County's timing and its delays are lower overall, because it leaves out lane-changing friction.
""" def originals_html(): import urllib.parse out = ["Before any processing: the County's emailed documents and " "the raw downloads from public websites. Everything else on this page is built from these.
"] for title, brief, files in ORIGINALS: rows = [] for rel, orig_name, desc, url in files: src = ROOT/rel if not src.exists(): continue name = orig_name or src.name dst = RAW/"originals"/name; dst.parent.mkdir(parents=True, exist_ok=True); shutil.copy(src, dst) href = "code/originals/" + urllib.parse.quote(name) size = src.stat().st_size sz = f"{size/1e6:.1f} MB" if size > 1e6 else f"{size/1e3:.0f} KB" srccell = f'source address' if url else "email attachment" rows.append(f'{E(name)}{E(brief)}
| file (opens / downloads) | " f"what it is | size | where it came from |
|---|
{E(brief)}
" + "".join(items)) # zip of the package (no County PDFs, no regenerated outputs, no local Python install) zp = RAW/"bigbend-pipeline-code.zip" with zipfile.ZipFile(zp, "w", zipfile.ZIP_DEFLATED) as z: done = set() _w = z.write def add(path, arc): if arc not in done: done.add(arc); _w(path, arc) z.write = add for _, _, files in GROUPS: for rel, _ in files: if (ROOT/rel).exists(): z.write(ROOT/rel, f"pipeline/{rel}") for extra in sorted((ROOT/"data/raw").rglob("*")): if extra.is_file() and "osm/context" not in str(extra): z.write(extra, f"pipeline/{extra.relative_to(ROOT)}") for extra in sorted((ROOT/"data/measured/county-counts").glob("*.totals.csv")): z.write(extra, f"pipeline/{extra.relative_to(ROOT)}") size_mb = zp.stat().st_size/1e6 css = """ :root{--bg:#fff;--fg:#000;--mut:#444;--rule:#000;--hair:#bbb;--shade:#f5f5f5;color-scheme:light} *{box-sizing:border-box} html{background:var(--bg)} body{margin:0;background:var(--bg);color:var(--fg);font-family:"Times New Roman",Times,serif;font-size:17px;line-height:1.5; padding-left:max(16px,env(safe-area-inset-left,0px));padding-right:max(16px,env(safe-area-inset-right,0px));padding-block:32px 64px} main{max-width:980px;margin:0 auto} nav{font-size:15px;margin-bottom:10px} nav a{color:var(--fg)} header{text-align:center;border-bottom:2px solid var(--rule);padding-bottom:14px;margin-bottom:18px} h1{font-size:26px;margin:0 0 6px} .meta{color:var(--mut);font-size:14px} h2{font-size:19px;margin:28px 0 4px;border-bottom:1px solid var(--rule);padding-bottom:3px} .brief{margin:2px 0 8px;color:var(--mut);font-style:italic} details.file{border:1px solid var(--hair);margin:6px 0} details.file>summary{cursor:pointer;padding:6px 10px;list-style:none} details.file>summary::-webkit-details-marker{display:none} details.file>summary::before{content:"▸ "} details.file[open]>summary::before{content:"▾ "} details.file[open]>summary{border-bottom:1px solid var(--hair);background:var(--shade)} summary code{font-family:"Courier New",Courier,monospace;font-size:15px;font-weight:700} .desc{display:block;font-size:15px;margin-left:18px} .raw{margin:6px 10px;font-size:14px} .raw a{color:var(--fg)} pre.src{margin:0;padding:8px 0;overflow-x:auto;max-height:70vh;font-family:"Courier New",Courier,monospace;font-size:13px;line-height:1.4;counter-reset:ln} pre.src .l{display:block;padding:0 12px 0 0;white-space:pre} pre.src .l::before{counter-increment:ln;content:counter(ln);display:inline-block;width:3.2em;padding-right:10px;text-align:right;color:#888;user-select:none} .box{border:1.2px solid var(--rule);padding:10px 14px;margin:10px 0} .box a{color:var(--fg);font-weight:700} h3{font-size:17px;margin:16px 0 2px;font-style:italic;font-weight:400} .tw{overflow-x:auto} table.orig{border-collapse:collapse;width:100%;font-size:14.5px} table.orig th{border-top:1.5px solid var(--rule);border-bottom:1px solid var(--rule);text-align:left;padding:5px 7px} table.orig td{border-bottom:1px solid var(--hair);padding:5px 7px;vertical-align:top} table.orig a{color:var(--fg)} ul a{color:var(--fg)} .ctrl{display:flex;flex-wrap:wrap;gap:8px 16px;align-items:center;margin:8px 0;font-size:15px} .ctrl button,.ctrl select,.ctrl input{font-family:inherit;font-size:15px} .ctrl button{border:1px solid #000;background:#fff;padding:2px 9px;cursor:pointer} .ctrl button.on{background:#000;color:#fff} .simgrid{display:grid;grid-template-columns:300px minmax(0,640px);gap:12px;margin:8px 0} .simgrid canvas{border:1px solid #000;max-width:100%;height:auto;background:#fff} @media (max-width:760px){.simgrid{grid-template-columns:1fr}} table.live{border-collapse:collapse;font-size:15px;margin:6px 0} table.live th{text-align:left;font-weight:400;color:#444;padding:2px 8px 2px 0} table.live td{padding:2px 18px 2px 0;font-variant-numeric:tabular-nums;font-weight:700} .err{color:#c0392b;margin:2px 0} .shots{display:grid;grid-template-columns:repeat(auto-fit,minmax(280px,1fr));gap:12px} .shots figure{margin:0} .shots img{width:100%;border:1px solid #bbb} .shots figcaption{font-size:14px;color:#444} details.explain{border:1px solid #000;margin:6px 0 10px} details.explain>summary{font-style:italic;padding:4px 10px;list-style:none;cursor:pointer} details.explain>summary::-webkit-details-marker{display:none} details.explain>summary::before{content:"▸ "} details.explain[open]>summary::before{content:"▾ "} .exp{padding:4px 14px 8px;font-size:16px} footer{margin-top:36px;border-top:1px solid var(--rule);padding-top:8px;font-size:13px;color:var(--mut)} """ today = datetime.date.today().strftime("%B %-d, %Y") page = f"""