Methodology
How the data flows
- Collect. A Python pipeline downloads the FBI agency directory for 51 jurisdictions, monthly NIBRS animal cruelty series, and the annual NIBRS tables, recording the URL, retrieval time and checksum of every file.
- Normalize. Spreadsheets from six publication years, in several layouts, are parsed into one consistent format and linked to agency ORIs.
- Validate. Every output is checked against a JSON Schema and unit-tested before it is published.
- Publish. This static site is rebuilt from the validated data. A scheduled job checks for new FBI data every month and proposes updates for review.
Linking agencies to ORIs
The FBI's agency tables name agencies but do not include their ORI codes. IROPO links each
row to the FBI agency directory within the same state, first by exact name, then after
removing generic words such as "Police Department", then by word order. Each ORI can be
claimed only once per state and year, and ambiguous matches are left unlinked. 99.0% of
64,306 agency-year rows are linked.
Rates and coverage
Rates divide offenses by the average population served by agencies that reported to NIBRS
that year, per 100,000 residents. Coverage is the share of all residents served by those
agencies. States with under 50% coverage, or with almost no reported cruelty despite broad
coverage, are excluded from state rankings.
The 1977 agency directory
IROPO's first agency spreadsheets for North Carolina, North Dakota and New York were
transcribed by OCR from a 1977 directory. The pipeline checks each legacy ORI against today's
FBI directory, flags look-alike character errors (such as I for 1),
and publishes the results as a crosswalk.
What is an ORI?
An ORI, or Originating Agency Identifier, is the nine-character code the FBI's National Crime
Information Center assigns to each criminal justice agency. The first two letters are the
state. For example, the New York City Police Department is NY0303000. Every
NIBRS incident report carries the ORI of the agency that submitted it, which makes ORIs the
natural key for joining agency data.