Analytics Crafting a Way Forward

There is a developing train of thought in the analytics hive mind. It has been rolling us towards approaching our work as a craft - not a sterile set of empirical technical tools. At first glance, this idea doesn’t fit the narrative. Analysts are hard on the numbers and laser-focused on the truth. Look closer. In every analysis, somewhere between the source data and enlightened insight, there is a squishy middle that leaves room for creativity.

Acknowledging analytics as a profession would provide a win-win-win. Individual contributors could stay in their rewarding career longer, without the need “to go into management”. On-ramps for new talent would be developed, clearly setting expectations before they start the job. Finally, professionalizing will help deliver consistent value to our companies.

The community has established the why analytics is a craft but is looking for the how. This conversation is too important not to push forward.

If you still need to internalize the idea of analytics as a craft, then you’d be best served reading a series of posts. They will provide more clarity than I can rehashing their thoughts.

  • Benn Stancil notices our industry is at a crossroads and encourages us to benchmark analysts by their puzzle-solving skills, not purely their technical ones.
  • Bobby Pinero reflects on who analysts are, putting the emphasis on soft skills first and training for technical skills as the need arises.
  • Then Tristan Handy and Randy Au make the case to recognize analytics as a profession and craft, respectively.
    • Quick aside - I believe Tristan and Randy are driving at the same point. Therefore I’ll be using their terms - profession and craft - interchangeably.

For brevity, here are a couple of quotes that won me over (emphasis mine).

Jobs that are hard tend to follow one of three paths:

  1. Specialize Sometimes it’s possible to decompose a job into many component parts, assign those each to different specialists, and create an assembly line model of production.
  2. Automate Sometimes you can externalize the knowledge of a small group of experts into some artifact of technology, which then reduces the expertise needed by individual practitioners.
  3. Professionalize If the job is irreducibly hard (like medicine or law or engineering), create the supports (training / credentialing / etc.) to support humans doing that job well even though it is hard.

I think we have to admit that creating and disseminating knowledge in organizations is really, irreducibly hard. I think the answer is actually #3

Tristan takes a top-down approach. Analysts have a difficult task with three ways to move forward. If you agree that an analyst’s task is irreducibly hard, then the only path is to recognize that work as a profession.

Randy Au makes a similar argument, but builds from the bottom-up, looking at the plethora of skills needed for a career in data. At some point, you will need to apply math, write code, persuade others, demonstrate domain knowledge, and more. His point is these loosely coupled skills are all packaged up in the analytics craft.

What’s it mean to be a good woodworker? It’s an endless list of skills about how wood works, how to use certain tools, how to design and put pieces together. For a writer it’s skill with the use of language, research, storytelling, planning. To become good at the craft, you need to build up all those interrelated skills to the point where they function as a distinguishable whole.

Personally, this parallel to woodworking struck a chord. My grandpa was a great woodworker. He crafted everything from Noah’s Ark toys to the table my family gathers around for Thanksgiving. Each project, regardless of size, was a way for him to practice his craft.

Grandpa crafting a bowl on his lathe
Grandpa crafting a bowl on his lathe

I believe my grandpa’s approach holds our key to professionalizing our craft. That key is practice.

Practice - yes, we are talking about practice. I am not talking about Titanic decision trees, 4 Cs of diamonds scatter plots, or even jaffle shops. These toy data sets drill a specific skill. We need the interrelated skills to function as a whole. We need to practice end-to-end analyses.

Thankfully we can look at other professions for a guide.

David Perrell - the online writing guy - coined the term “imitate then innovate”. Imitation requires using your entire skillset. Pay attention when you feel the pull to go a different direction, that is actually your personal style showing up. That is the point when imitation morphs to innovation.

The “imitate then innovate” approach has been used by artists, authors, and actors. It shows up in medical residencies, shadowing as a teacher’s assistant, and electrician’s reaching journeyman status. So why do we not see this for analysts?

Analysts are missing their medium to imitate because real analyses are locked behind NDAs or hidden from the public as proprietary information. If we are to continue developing the field of analytics into a profession, we need public examples to imitate.

The blueprints are being drawn up. Projects like Hex’s community contributed analyses provide self-guided examples to imitate. Bootcamps like the Analytics Engineer Club pair hands-on practice with experienced oversight to overcome hurdles. Our profession needs more of this. I encourage you to replicate others' analyses, build your own gallery, and share the work you can. Soon your work will be imitated too - and that’s the sincerest form of flattery.