UrbaneOptics

NYPD Moving Violations v0.1

Written by Joe James-Rodriguez - November 13, 2019

The NYPD has a public data feed where they publish all of the moving violations they’ve written. The data itself is a general tally of moving violations, broken down by precinct and by month. The data has historically been poorly formatted, and only exists in Excel or PDF format, making it extremely difficult to parse historical information for any precinct.

An example of the invalid formatting that exists in the NYPD feed. A parser attempting to analyze this data would read this data as one row instead of two
An example of the invalid formatting that exists in the NYPD feed. A parser attempting to analyze this data would read this data as one row instead of two

Additionally each month is broken into a separate zip file that needs to be downloaded, and each precinct has it’s own Excel file. If you want to see all of the moving violations written by one precinct since 2011, you’d need to manually download 97 zip files. This is not scalable, and creates a huge barrier for analysts attempting to use this data.

Want to see historical moving violations for a precinct? Have fun downloading, extracting and opening 97 different zip files containing 100 files each
Want to see historical moving violations for a precinct? Have fun downloading, extracting and opening 97 different zip files containing 100 files each

I’ve worked on a couple of projects over the last few weeks to address these deficiencies, and am officially releasing the first public version today. The web-application can be viewed at nypd-moving-violations.urbaneoptics.com. This app allows you to generate charts which show the breakdown of moving violations written for any precinct. The data is being parsed from UrbaneOptics/nypd-moving-violation-data. I built that repository as a sort of ‘bandaid’ to correct all of the data issues that exist within the NYPD feed. That repository also aggregates all of the data in a format that can be easily parsed by any data analysis tool, so anyone can build off of the data.

v0.1 of the NYPD moving violation application
v0.1 of the NYPD moving violation application

My inspiration for this project was mostly stirred from my interest in trying to better understand the traffic violence occuring on Canal Street, a dangerous street that sees an average of 500 crashes per year, and whether any correlation could be drawn between enforcement and the contributing factors to crashes. While it was extremely easy to derive crash information for the street from Crashmapper.org, no such solution had yet to exist for analyzing enforcement, and I’m hoping this tool can help fill that gap.

Questions, comments or suggestions? Shoot me an e-mail at joe@urbaneoptics.com