Tag Archives: data science

Data Scientists vs. Data Engineers: Facts vs. Interpretation

Some of the things we build at work are closed-loop, Internet-scale machine learning micro-services. We have created algorithms that run in milliseconds that we can invoke via REST calls, thousands of times per second. We also have created data pipeline processes that process new (mostly sensor) data and build and publish new models when critical thresholds are reached. This work requires the collaboration of two very in-demand specialists: Data Scientists and Data Engineers.

Contrary to the classic Math vs. Coding vs. Domain Expertise Venn diagram, Data Scientists and Data Engineers share many similarities. Both love data. Both have domain expertise. Both are great functional programmers. Both are good at solving complicate mathematical problems—both discrete and continuous. Both use many similar tools and languages (in our case, Spark, Hadoop, Python and Scala).

However, over the past two years, as we have improved the collaboration between each to build better machine learning services, we have some key differences between each role. These differences are not just based on skill set or disposition. They also include differences areas of responsibility that are essential to creating fast, scalable, and accurate machine learning services.

It is easy to muddle raw data from fully deterministic derived data from algorithmically derived data. Raw data never changes. Rules may change but are easy to manage with clean version controls. However, even the same deterministic algorithms can produce different results (one example: whenever you refit or rebuild a model using new data, your results can change). If you are building algorithmic services you need to keep everything clean and separate. If not, you cannot cleanly “learn” from new data and continuously improve your services.

We have found a very nice separation of responsibility that prevents muddling things:

  • Our Data Engineers are responsible for determinist facts
  • Our Data Scientists are responsible for interpretation of these

This boils down to this: determinist rules are the purview of engineers while algorithmic guesses come from scientists. This is a gross simplification (as both engineers deal in many, many complexities). However, this separate keeps it very clear, not only in determining “who does what” but also preventing errors, guesses, and other unintended consequences that pollute data driven decision-making.

Let’s take Google Now’s “Where you parked” service as an example. Data Engineers are responsible for processing the streaming sensor updates from your phone, combining this with past data, determining motion vs. at rest, factoring out duplicate transmission, geospatial drift, etc. Data Scientists are responsible for coming up with the algorithm to determine whether your detected stop state is a place where you parked (vs. simply being at work, at home, or at a really bad stop light). Essentially, Data Engineers capture and process the data to extract required model features while Data Scientists come up with the algorithm to interpret these features and provide an answer.

Once you have separation down, both teams can collaborate cleanly. Data Scientists experiment and test algorithms while Engineers design how to apply at scale, with sub-second execution. Data Scientists determine what approach is used to build models (and what triggers model optimization, build and re-fitting). Data Engineers build seamless implementation of this. Data Scientists build algorithm prototypes and MVPs; Data Engineers scale these into fast, reliable, services. Data Scientists worry about (and define rules) to exclude outliers that would wreak havoc on F-tests; Data Engineers implement defensive programming and automated test coverage to ensure unplanned data does not wreak havoc on production operation.

Data Analysis of USAToday’s NFL Arrest database: 15 Surprising Insights

As soon as I learned that USA Today had released on open database of NFL player arrests (2000 to present), the data scientist in me thought, “I imagine there are some interesting patterns in there.” Rather than wondering, I downloaded it and dived right in.

The arrest data is easily readable, but lacks some important items (such as the age of the player at the time of arrest). As such, I decided to mash-up the data with two other sources: DOB, Height and Weight data from NFL.com and the strength and speed data from the NFL Combine. This would let me explore some of the more interesting (and potentially controversial) claims I heard in many TV interviews about the effect of increases in player size and strength had on aggression and crime.

My findings

Here are my findings from analyzing the data:

  1. Arrest frequency is NOT increasing. It is actually down from a really bad spate from 2006-2008
  2. NFL players, in general, have a one-third less likelihood of being arrested than everyday US residents. They have 15x the median US income and 3x the college graduation rate.
  3. However, many of those who are arrested are arrested many times throughout their career. 124 people were arrested more than once. One player was arrested 9 times. Sixty-five arrests were for multiple counts, across multiple criminal charges.
  4. Guilty verdicts (conviction, plea, or plea agreement) are the most common legal outcome. They occur almost 7x more frequently than Acquittals
  5. Nevertheless the most common action taken by NFL teams in response to arrests is “No Response.” This occurs 84% of the time
  6. Two-thirds of arrests occur off-season. However over 99% are arrest of players under contract. Free agent arrests are rare (although all of them later signed onto teams)
  7. Three teams (Minnesota, Cincinnati and Denver) have seen double the “normal” number of arrests per team
  8. Four criminal charges (DUI, Drugs, Domestic Violence and Assault) represent 60% of all arrests.
  9. Six charges (DUI, Drugs, Domestic Violence, Assault, Gun Charges and Disorderly Conduct) represent 80% of all arrests. Each of these has a single team with more arrests than any other.
  10. Of the most frequent charges, conviction rate varied enormously. DUIs had the highest conviction rate; Domestic Violence the lowest. While Domestic Violence pleas + convictions outcomes outnumbered acquittals 10:1, the vast majority of these cases were dropped or resolved in Diversion Programs
  11. The median arrested NFL player is: 25 years, 6 months old; is 6’2” tall, weighs 230 lbs., can run the 40-yd dash in 4.61 seconds and can bench press 225 lbs. 21times.
  12. While 88% of the arrests were of players under 30, age was not a factor (in arrest or criminal charge). The distribution of age at time of arrest virtually matched the distribution of ages across the NFL.
  13. There has been much talk in the media about the size of players and the potential impact on aggression. However, contrary to the opinions, neither height nor weight was a factor in likelihood of arrest or type of criminal charge.
  14. Unsurprisingly,  player speed was not a factor as well.
  15. However, analysis of player strength did show a pattern–not one about the strongest players, but about the least-strong. players It turns out those arrested for Sexual Assault stood out as the group with the lowest distribution of strength scores in the NFL Combine.

The data

  • 730 arrests between 2000 and the present (the database actually expanded by one entry a few days after launch to account for the arrest of Jonathan Dwyer)
  • These 730 arrests spanned 544 players (more on that below). Of these 544 players, 330 had publicly-available NFL Combine results
  • The arrests spanned 51 separate criminal charges (with some interesting concentrations, see below)

Sankey NFL Arrest “Flow”

The diagram at the top of this post is called a Sankey Diagram. This allows exploration of which teams had players arrested for each type of criminal charge then what were the distribution of outcomes for these charges. You can explore this chart, for all teams and all criminal charges in full at this page.

Deeper Dive

Sankeys allow exploration of broad pattern (such as which teams have the most Domestic Violence arrests) and interesting outliers (e.g., which team had a player arrested for Pimping; and what was the result). However, they do not make it easy to explore other dimensions of this data. The rest of this post takes a deeper dive into the data, exploring each of the 15 findings with different summaries and visualizations.

   Next: Arrest Frequency (and who was arrested nine times)