[Data]: Systemic duplicate wikiDataId across contributions/cities/MX.json (~3,276 records, ~35% of file)

Author: rahulpawar-31Created Sep 18, 2026Updated Sep 18, 2026
Labelsbughelp wantedgonna take long time to solvedata-correctiondata:citieslarge-contribution

Summary

While fixing #1623 (Epitacio Huerta misclassified state), a 4-record run (ids 70310-70313) was found to share one incorrect wikiDataId — the result of what looks like a copy-forward bug during the original 2019 bulk import (see #1630, .github/fixes-docs/FIX_1623_SUMMARY.md). Pullfrog's automated review of that PR flagged that this pattern is not isolated to those 4 records — it's systemic across the whole file. I independently re-verified the scan; numbers below are mine, cross-checked against pullfrog's.

Scope

  • contributions/cities/MX.json: 9,321 total records.
  • 1,939 groups of records share an identical wikiDataId with at least one other record.
  • 5,215 records fall inside one of these groups; assuming (generously) one "rightful" owner per group, that leaves ~3,276 records (≈35% of the file) carrying a wikiDataId that very likely belongs to a different place.
  • 1,924 of the 1,939 groups (99%) are fully consecutive id runs — the same shape as the 70310-70313 case, strongly suggesting the same root cause (an import-time bug that failed to look up/assign a fresh Wikidata match per row and instead carried the previous row's value forward).
  • Of the duplicate groups: 1,784 contain records with different name values, and 1,665 span more than one state_code — both are strong signals these are genuinely different places wrongly sharing one ID, not legitimate multi-locality Wikidata entities.

Example groups

wikiDataId # records Sample
Q1144317 11 69819 Ejutla de Crespo (OAX), 69820 Ekmul (YUC), 69821 Ekpedz (YUC), 69822 El Abra (TAM)...
Q20219114 10 70256 El Vergel (DUR), 70257 El Vicarlo (GUA), 70258 El Volador (VER), 70259 El Walamo (SIN)...
Q20237772 9 69682 Detiña (MEX), 69683 Diamante de Echeverría (CHP), 69684 Dieciocho de Marzo (MIC)...
Q5654722 3 68017 Abasolo del Valle (VER), 68018 Abdenago C. García (CHH), 68019 Abelardo L. Rodríguez (CAM)

(Full group listing available on request — didn't want to dump 1,939 groups into this issue body.)

What's already fixed

PR #1630 corrects the 4-record case that surfaced this pattern (70310-70313 — Epazoyucan, Epigmenio González, Epitacio Huerta, Ermita de Guadalupe), each re-verified against Wikidata's raw entity JSON individually.

What this issue tracks

The other ~1,935 groups (~3,272 records) are not yet fixed. This is deliberately not being attempted in one pass — each record needs an individual Wikidata lookup/verification (by name, admin division, and coordinate match, the same method used in #1630) before its wikiDataId can be corrected; there's no safe bulk-automatable fix since the "correct" ID for each row has to be looked up per place, not derived from the existing (wrong) data.

Suggested approach for whoever picks this up

  1. Regenerate the full list of duplicate-wikiDataId groups programmatically (script above; happy to share it).
  2. For each group, verify each member's real-world identity independently (name + state + coordinates) via Wikidata, exactly as done in #1630/FIX_1623_SUMMARY.md.
  3. Batch fixes by state or by group size to keep individual PRs reviewable — 3,276 records in one PR would be unreviewable.
  4. Worth checking whether other country files have the same import-era bug — this scan only covered MX.json.

Also worth checking

Whether this same 2019-era import bug affected other large country files (US.json, IN.json, BR.json, etc.) — not yet scanned.

Source: dr5hn/countries-states-cities-database