Data Science Product Manager

PM Roles — part 1

This is part of a series of articles on different types of product management roles. The post below was written in response to a question on Quora.

Typically data science product teams have incremental goals around financial metrics, customer behavior and customer experience/engagement metrics. PMs own defining and aligning the team around these metrics and put proper measurement programs in place. You cannot improve what you cannot measure.

The data science product development lifecycle includes the following steps:

Defining the business case: In fields such as Search and Recommendations, this requires understanding domain metrics (eg: NDCG, MRR), analysing customer sessions, aggregating unstructured customer feedback to understand customer problems. PMs may also run human judgment programs and other analytics programs. Basically, PMs do what it takes to build a business case using all available data analyses methods or go build new tools if they don’t exist. Good data science PMs can make convincing data-driven arguments to engineering about building new data products, articulate problems with existing products, create a backlog, prioritize and get buy-in for their ideas.

UX: Data science products have a front-end component. PMs will define the customer facing aspects of the product. Depending on org structure, data science PMs have to collaborate with ‘front-end’ PMs to integrate their products.

Data collection: This is probably the most under-appreciated function of a data science PM. PMs define tracking specs and work with dev teams to get good coverage and accuracy in logs. For example, as a Video Search PM, you have to work with TV App developers, Web and Mobile developers in your company to get a 360 degree view of the customer experience.

PMs must figure out how to unify data from multiple sources. If you work in fields such as Ads or Marketing technologies, this can involve partner management such as working with SSPs, DSPs, DMPs.

Feature engineering: Your role is typically more consultative. Bringing your domain knowledge & articulating the customer/business problem well can help engineering be smart about feature design. Depending on your analytical nous, you may even build/suggest features as you gain more experience in the problem domain.

Algorithm development: You should understand the evaluation methods for classification and regression algorithms and serve as a filter for any garbage in-garbage out implementation. What targets are you optimizing for, what precision/recall tradeoffs are you making, what is the feature importance. Does the model make sense? Influence these decisions by keeping the team aligned on the business case. Also, develop or understand existing offline validation tools to evaluate algorithms without expensive testing. Be the gatekeeper for when a data science product is ready for prime-time.

Data engineering: Integrating models online can be a separate function from data science. Refer Combining Data Science/Machine Learning. You should be the glue across the entire process.

Experimentation: Most data science products will be tested against a baseline to demonstrate incremental benefit. You will drive online experimentation, interpret results & make recommendations to business & engineering teams for future iterations of the product.

Monitoring: Post roll-out of a data science product, you should put monitoring programs in place to keep track of data pipelines, data quality & financial, behavioral metrics.

The golden rule in data science teams is “In God we trust, everyone else brings data”. So, remember to back your arguments with data and you will do fine. Otherwise, you will be ineffective as a Product Manager and end up project managing engineering teams.