Public data, kept honest · part 1 of 4
adopted13 September 2026cost: local CPU; runtime ≈ 2.5 minutes a run
Geography: where the job actually is
7 runs
The question
Did the change make the dataset more accurate without moving the headline numbers for a reason nobody can explain?
adopted
History fixes: date parsing (mixed date/datetime strings dropped FY2013/14/19), population-weighted ZCTA->county (Census 2010 rel file; fixes Boise/Bartlesville), city fallback share 0.8->0.6 — all 17 FYs now populated (national…
Runs7
Kept7
Every run
| # | what changed | verdict | KC filings (3 y) | geo resolved % | notes |
|---|---|---|---|---|---|
| 0 | Baseline: ZIP->CBSA geo with city/zip3 fallback; suffix-strip employer keys; FY2023-FY2026Q3 | kept | 5559 | 99.802 | runtime 150.7s; unresolved geo 4056 rows (mostly CT) |
| 1 | Geo: Connecticut fallback to 2020 CBSA delineations (legacy counties) in build_geo.py | kept | 5559 | 99.981 | geo_unresolved_rows 4056 -> 397; metros 919 -> 952; KC unchanged |
| 2 | Geo: ZIP/state conflict resolution (trust typed city+state when it is a known combo and city+ZIP-state is not; else trust ZIP and correct the state) | kept | 5554 | 99.98 | 789 states corrected from ZIP, 1431 ZIPs distrusted (e.g. 330 Dallas TX rows with NJ ZIPs); KC last3 5559->5554 (ZIP-typo rows moved out); kc_city_labels 96->76 |
| 9 | Secondary worksites: DOL LCA_Worksites files (all worksites per case) -> kc_metro.json any_worksite block (superset of primary-worksite headline); bug fix: FY rollups now carry… | kept | 5977 | 99.98 | KC 3-yr any-worksite 6817 filings / 1782 employers vs primary 5977 / 1549; worksite files cover 1.95M of 2.04M in-scope cases |
| 12 | KC-headquartered employers view (EMPLOYER address ZIP/city -> CBSA; all their LCAs nationwide) -> kc_metro.json kc_headquartered | kept | 5977 | 99.98 | KC-HQ 3-yr: 5202 filings / 556 employers; Black & Veatch 404, Burns & McDonnell 341 nationwide |
| 13 | Bug fix: pipeline wiped data/out (deleting uscis_petitions.json and crosscheck json) - now only rebuilds states/ and metros/ | kept | 5977 | 99.98 | USCIS + cross-check sections restored in README |
| 17 | History fixes: date parsing (mixed date/datetime strings dropped FY2013/14/19), population-weighted ZCTA->county (Census 2010 rel file; fixes Boise/Bartlesville), city fallback… | kept | 5912 | 99.537 | all 17 FYs now populated (national 342k-622k/yr); 8.65M cases in scope; KC last-3 5912; runtime 1352s |