How much data science do you really need to know as a PM?

AI + ML are radically transforming the world we live in, but for you as a product manager - what does this mean?

We sat down with Scott Jones, Cloud Platform Senior Product Manager at Lenovo to dive deep into what's necessary and what's not before his session at the Product-Led Festival in November.

To kick off, could you share a bit about your background and how you joined the team at Lenovo?

I studied economics and film in college, which set up the dichotomy of analytical pursuits and creativity/art that has persisted for the rest of my life. After college I spent a chunk of time as an analyst, in management consulting, economic consulting in higher education and public policy, and as an internal consulting role inside a $1 billion non-profit health care provider in NYC.  

I relocated to Los Angeles and joined a technology company, still as an analyst in a non-technical domain, but after a year I transitioned into Search Engine Marketing for an internet yellow pages property. I got deep into SQL as an autodidact, writing and managing logic to govern ~$3-$4 million monthly spend on Google, Yahoo and Bing across hundreds of thousands of keywords, and ultimately reached a level of knowledge and technical proficiency such that leadership ordained I would be a good product manager.  

That (amazing, career-defining) moment then set me on the path of the last ten years, driving production strategy and innovation across a variety of domains including adtech, martech, IoT, e-commerce, mobile, and the cloud. I was hired at Lenovo in early March of 2020 to drive innovation in their cloud software org.

What’s it been like starting in a new role during the remote working period?

The timing of starting my new role with Lenovo was fascinating. My new employee orientation was on March 9th, I met my hiring manager and a small fraction of my team March 10th and 11th, and then by March 12th I was fully remote and have been ever since.  

At most of my previous employers I think this situation would've been a train-wreck due to poor remote work tools, processes and practices. However, Lenovo was already largely a remote work powerhouse, given that they have 50k+ employees distributed across almost every continent (though I have yet to interface with someone in Antarctica, anything is possible right?).

Lenovo's suite of tools for network access and remote collaboration have truly been amazing and a natural fit for me, and the team has marvelled at my ability to fit in and drive progress without ever actually meeting 98% of my colleagues. I count myself very lucky!

How have you leveraged your data analysis and statistical research skills as a product manager?

It's an interesting question in that in the first product manager role I had, described above, I still had access to SQL and could fish for my own data and insights. Then I joined an adtech company where the CTO was very uncomfortable with a product manager having that level of access (i.e. the fear that I would randomly start deleting tables).  

The skillsets I had developed during my career stretch as an analyst concerned with making sense of what was often chaotic data and surfacing insights to decision makers, and that mindset lends itself very well to product management. As a product leader, you either are that decision maker, or it's incumbent on you to be able to tell the story of the signals present data, the "so what," and proposed path forward so that the decision maker is informed and aligned.  

At this point the analytical skills tend to manifest in at least a few key ways -- being able to "see" complex systems or flows in my mind's eye and distill them down to critical touch points where KPIs could be measured; being able to quickly react to data and call out takeaways that others might not see; more generally being able to connect dots on seeming disparate signals, and so on.

Whilst being deeply analytical, you’re also a creative, how do you foster creativity and innovation with your team at Lenovo?

As background context for this question, generally my spiritual pursuits of the past 10 years --  I will spare a deep dive into this, but will be more than happy to discuss with folks who want to reach out directly to converse about it -- has led to realization of what many call "the flow state," and how there is "muscle" we can develop such that this state is always accessible, regardless of the activity at hand.

As such, I'm now at a point where I am channeling or "riding" the flow all the time -- whether speaking publicly, playing music, making art, writing requirements in Jira, writing a PPT presentation to tell a story to sway an audience, hanging out with my family or friends -- whatever the activity, I get my conscious mind out of the way and let the flow through.  This leads to an effortless state where I don't really have to "do" anything -- I simply listen to the inner voice and follow my intuition.

In practice, this informs the energy and "vibe" I bring to all aspects of work. So just as you might see me playfully sitting at a piano or a set of drums conjuring up a new idea, you will similarly find me documenting ideas for business innovation, socializing them with stakeholders and iterating and developing where they can go to drive business and customer value.  

From the perspective of the team of Lenovo, or other teams I've worked with, I think this comes across in variety of ways -- a tendency to ask "why," and to pull investigative threads as far as I can take them to find kernels of insight and opportunity; connecting dots on opportunities that others might not see, either due to differences in experiences or skills, or perhaps simply from being locked into a particular point of view for a while; and the entrepreneurial / lean mindset of having a vision of what an opportunity looks like in future state, and how I could fashion iterative incremental steps to walk and test in that direction without wasting time or money.  

In more practical terms, I think I foster creativity and innovation by presenting lumps of clay that I have fashioned as mental models of opportunities, and I invite my colleagues to react to it and help me shape what it can become. We all tend to thrive off of this process because I like to make it fun, playful and very collaborative, where I make it clear that I know I don't know everything (unfortunately a unique leadership style at times, ha!) and that I want their input in how to realize these opportunities.

AI and ML continue to radically transform the world we live in, and the products we are building - but how much of Data Science do you really need to know as a PM?

You can get by at an abstract level of understanding built up from work experience and intellectual curiosity (reading, asking questions, etc).  I don't think you need to get a masters degree in these domains, for example.  

It's incumbent on the PM to be able to describe the "what" and the "why" -- and the nitty gritty of the "how," meaning the actual algorithms, models, tools and techniques that are employed are really up to the data scientists and engineers.

You can provide the typical PM reality checks -- e.g. that sounds great but it will also take 2 years before we realize any value... what's the "crawl" version that we could phase up to "walking" and "running" after we demonstrate tangible results?

For example, years ago a product manager at Netflix might have said "we should have a recommendation engine that looks at the engagement of a single user, compares that to all other similar users, and can then provide useful/high value recommendations to the user on what else they would like." The data science team could then say "hey, that's a great use case for collaborative filtering, here's how we can approach it." The product manager didn't need to say "hey, we should use collaborative filtering to drive recommendations."  

Prescribing solutions -- the domain of the "how" -- can generally be dangerous territory for a PM as it can build resentment from engineers and data scientists.

What have you found to be the biggest difference between product in enterprise vs product in startups?

At a startup you can wear as many hats as you are comfortable with, or as many as the business requires you to wear. You can also move extremely fast from idea to production.

In contrast, enterprises move slowly with separation of roles/responsibilities and an aversion to risk. What might take a month end-to-end in a startup could take years in an enterprise depending on the constraints.

What resources would you recommend for PMs looking to improve their AI/ML knowledge and abilities?

Asking lots of questions. Subscribing to AI/ML subject matter on Quora and similar sites.  

Reading about the industry and applications across verticals. What is relevant in one area will have tangential applications in another.  

For example, a few years ago I was interviewing for a role with a tech company that builds software for running clinical trials. You are trying to capture data form users about how they are feeling, but there's always a risk that you aren't getting at reality with self-reported data.  

In the interview I mentioned something I'd read about how Uber recently obtained a patent for a model that can look at the motion of the gyroscope on a smartphone to guess that a user is drunk. Why not use something similar to ascertain whether a clinical trial subject is adversely affected by a medication as it impacts their motor skills?

What’s the biggest change you’ve seen in 10 years of AI product management?

One of the coolest tangible examples came while working in adtech, and concerns the processing power of big data computations such that real-time use cases are attainable.

I worked with a data scientist on a POC of new product for measuring foot traffic associated with adtech mobile campaigns, and this hadoop job he fashioned took a few hours to run. A wizard data scientist / full stack engineer (a unicorn archetype I will discuss in my #PLGFest presentation) was on the leading edge of experimenting with Spark at this company.  The wizard refactored the hadoop job to run in Spark -- the run time was reduced from hours to something like 45 seconds.

You’re a mentor and advisor, what do you wish you’d been told at the start of your career?

Speak up and let go of worry.

That's not specific to product management but is more of a fundamental thing for all of life, and it informs being a great product manager.

Be sure to check out Scott's session and more, at the Product-Led Festival, November 10-12.