Building AI products the right way, the expert take

“With great AI product power comes great responsibility...”

Okay, that’s not quite how the ancient adage goes but it’s very much applicable to AI product management today. 2012 was the breakthrough year for deep learning with the advent of Alex, the 62 million convolutional neural network. However, the discussion around the responsible usage of AI has been a very recent one.

We have yet to see the true operational use of a lot of responsibility principles and frameworks in AI product development. AI product managers must operate with some level of self-awareness and have the ability to ask the right type of questions when building AI products and driving product execution.

But how do you ensure the right quality of data? What are the potential biases? How can you build trust? And what strategies can you use for data privacy?

We posed these questions and more to a handful of experts in the field to get their advice and tactics. But if you want access to all the insights we gathered on AI product management get the full report here.👇

Discover insights from:

Alessandro Festa, Senior Product Manager at SmartCow, and Co-Author of K3ai
Deepak Paramanand, Director of Artificial Intelligence at JPMorgan Chase & Co
Megha Rastogi, Group Product Manager at Okta
Bruke Kifle, AI Product Manager at Microsoft

Main talking points:

Q: How would you ensure the right quality of data is being used when building AI products?

Alessandro Festa: Not only are there different kinds of data but there are also different kinds of AI. And different tasks that AI can do:

Sometimes you want to predict,
Sometimes you want to classify,
And sometimes you want to segment.

So if I put together this cross tabular thing where I have different types of data, AI technologies, and tasks, then you begin to understand this is the universal space I operate in. Now I can talk about data and how hard it is or how tough it is.

For example, say you’ve just started a website, you’re five months in and want to predict which customer is going to give you the next benefit. In that case, anybody will tell you five months of data is not enough. Now the question is, what can you do? The first option is you just stop, you don’t ask the question, because it doesn’t make sense.

Option B, you can go with synthetic data; which is when you’re saying - okay, I have five minutes of data right now representing a hundred customers; three genders, five ethnicities, and three age groups. Now, do I want to predict success for the same kind of customers? Or do I want some variability in that data?

You can then go and create synthetic data for the unseen age, ethnicity, and group combinations. Then say I have hypothecated that I’ll have more 40 year old caucasian males, using the site going forward for the next six months. And if that happens, I’m ready. So the right data there is hypothecated about the future.

Now, in this case, it’s synthetic - I’ve created it myself, based on the constraints I have. But is that right? Maybe it’s because I’m not going to discriminate against other people based on this data.

If I say the same thing about facial data, voice data, or speech data, where I’m going to personalize it and individualize it to someone, then perhaps the bar is different. It’s a very multi-layered question. I hate to say it depends, but it really does depend, because the application differs. If I take the same facial data, all I want to know is did I wink or not? Then you’re not as interested in whether I’m a man or a woman. You’re asking if you can reliably figure out if you can wink or not.

But in case I want to know more, like who winked? Male or female? What ethnicity? What age group? Then it starts getting complicated. It’s up to the individual who’s creating the product and the people designing it to understand what the nuances are and how to get to the right answer for themselves.

Q: What potential biases pose the biggest threat to modern AI products? And what are the best ways to address them?

Deepak Paramanand: When I was building a facial expression recognition product at Microsoft, there were a lot of reviews I had to do. The first thing that came out was, that it’s good to know a minority person is doing this because it’ll be a little more inclusive of your needs and other people’s needs.

But then I looked at what’s called the privilege chart. This ranks people by age, ethnicity, gender, and religious affiliation, and tells them how much privilege they have. It told me I have far more privilege than I thought. The privilege of being a man, speaking English, being in a western country, etc.

The moment I saw that I wasn’t as disenfranchised or as disempowered as I thought I was, I then looked at where I fitted in with the team I had. I realized we didn’t have enough women, we didn’t have enough money, the right ethnicity, and we definitely didn’t have people in the right age group. So look at yourself, look at your team, and then look at who you are building the product for.

After those things, I came to know that we had an ethnicity bias, gender, and age bias. So how could I ensure the product doesn’t have those things? How does it look for a 60 year old man or a 10 year old boy? It’s easy to think like that. But being a man I couldn’t know what it looked like to a woman. So you needed that person in the room to sit and tell you that’s a problem.

If you want to build products for the world, then the whole world must be involved in building them.

Map out the world of the customer; their behaviors, and the habits they’re going to have.
Map out your world, and then look at what the gap is.

Knowing that gap itself is going to be very humbling. It’s going to show up your biases, your blind spots, and tell you that, for example, the model is not representative of women because it’s all men building it. Yes, we got data for women. But we didn’t put the thought and the due diligence behind it to think like a woman.

Understand your own biases and your motivations
Map to your world and who you’re representing
Look at the gaps and then do your best to fill those gaps

Whether it be consultants, HR, family members, or colleagues, ask people to help you out. It’s also good to know how bad it is. But remember to document the journey, because in my experience, that journey of getting from bad to good, was also important, not only for us but for other teams. It allowed us to say, here are the five things you must do before building an AI product. It helped them understand their biases, and as a result, it took far less time to figure things out.

Bruke Kifle: The best way to solve biases in AI product development is to first be able to identify them. Beyond the traditional user-centered design approach to product development, this requires an awareness of different sensitive uses at the individual, organizational and societal levels.

Sensitive uses are simply the potential impacts of AI-powered products or services on people’s lives. In addition to an understanding of different sensitive uses, this requires ensuring products adhere to important responsible AI principles including fairness, ethics, transparency, and privacy.

These are important principles to consider and ensure throughout all stages of the AI development lifecycle – from product design and data collection/preparation to model development and deployment.

The OECD AI Policy Observatory is a great resource for monitoring the shared AI principles and policy initiatives in over 60 countries and territories. Ultimately, addressing biases in AI product development requires product leaders to consider the socio-technical implications of their work.

In addition to the considerations above, product leaders can work towards solving biases in AI product development by ensuring diverse teams (backgrounds, SMEs, etc.) and fostering more inclusive practices in the design and development of AI products.

Megha Rastogi: You should always think about the potential bias that might have crept into your algorithm. There are a few ways you can minimize this.

Sample bias - Make sure you have the right representation of data. That means having a large enough sample size, when you have this, you’re probably going to cover all the data representations and demographics.
Measurement bias - Ensure there is no missing data, or that everything is normalized correctly. So if you’re expecting results in inches, and some of your data is in inches and some in centimeters, you want to make sure it’s all normalized so it can give the most accurate result.
Prejudice - Take steps to avoid prejudice creeping into your algorithms. You need to have diversity in the data sets. This kind of goes hand in hand with sample bias. If you have a large enough sample size, you would probably decrease some of this prejudice as well.
Algorithm bias - Test and validate different algorithms. What happens sometimes is an algorithm might be leaning heavily on a certain feature or certain feature sets, and you want to make sure that it’s the right result you’re looking for.
Confirmation bias - You may have a hypothesis and you really expect that to be true, but you should ensure an open mind and perhaps do a lot of AV testing to make sure it’s the right representation of what you’re looking for.

AI systems will always contain a degree of human error. So the best thing we can do is identify accurate representative data and make sure there’s enough of that data.

Q: How pivotal is data preparation for ensuring user privacy?

Deepak Paramanand: It cannot be understated. Say I give you data for the expression recognition product, fully trusting that you’ll use it for that reason. But you then take the data to say - how do men engage on Zoom calls? That’s not what I signed up for. So now, how do I replicate that data to tell the people you’re going to do the work for that I need data from you, about you, for you, so that I can build it correctly?

It’s much harder for startups or smaller companies to do this because they don’t have a lot of resources, time, or people to do this. They may have to cut corners. But imagine being found out later on. Big companies like Microsoft, Amazon and Google can heavily course correct. For most startups, any bad news about them could spell the end. So it’s about being open and transparent.

Also, the right task is important. For example, in my first product, we were told to predict telecom churn, and we were able to predict whether person A is going to leave or not. But then if person A is going to leave, how do I retain them?

You have to think ahead and write about the problem. Maybe person A behaves a little differently than person B or C. And that’s the reason why they’re leaving. Maybe we caused it or perhaps it’s an innate behavior. The question is - what do I do uniquely for Person A to retain them that I shouldn’t do for someone else?

We had to say not only what the problem was but who was impacted and how to solve it. The what, who, and why are very important. When you prepare data, think about the end journey, when you give a task to someone and the task is completed, how are they going to use it? How could they operationalize it? If you’re able to think through to that level, your data preparation will make sense.

For example, in our case, predicting churn was important, but the segmentation of those customers who are going to churn was also important because you can enable personalized treatment plans to say - person A is leaving because in their area the network is a huge problem. They are a unique customer who’s suffering because of poor network quality.

In terms of ROI, should you put up more towers to retain them? Maybe not because perhaps it’s okay to let them go. But now I know why they’re going, and if I know that’s a demographic, customer base or locality we want to grow in, maybe person A is the first symptom, but the root cause is even worse.

So data preparation, in this case, is also about knowing how the AI task is operationalized. And being able to think through and create the data for that reason.

Q: How are you building trust and transparency with users?

Megha Rastogi: You want to work on building trust with the user by providing some sort of transparency in your wordings, your content and how you show the results.

One way that Amazon does this is when I see the recommendations on the top - customers who viewed this item also viewed, etc. This gives me a sense of what the model is looking for and how it got to that recommendation.

You also want to handle your errors gracefully. This can be even more important in an ML use case. So if you see that there’s a cohort of users that are always crossing out your recommendations, perhaps gently ask them for feedback or show an error and re-adjust the algorithm to adjust for their data.

Bruke Kifle: AI is built on a community of trust. Organizations must maintain compliance with data privacy regulations while ensuring inclusive practices to allow for the design and development of products that work well for all users.

It’s also important that organizations adhere to the responsible AI principle of transparency to ensure the explainability of AI-powered systems that users can trust. Ultimately, by developing AI products that improve users’ experiences while still putting users first by ensuring data privacy, more inclusive products, and adhering to key RAI principles, we can establish much greater trust and transparency.

Deepak Paramanand: The product manager acts as the middleman. If customers could tell developers exactly what they wanted, and the cost, developers could build it. My job wouldn’t be needed. The only reason why my job exists is that customers know their problems, but they don’t know their solutions.

If you ask the first person who rode on horseback, what do you need? They would say a faster horse. They’d never say a car. Somebody has got to figure it out. Even if they could conjure up solutions, developers will build up to something that’s uniquely required for that customer, not for other customers.

So you need somebody in between to think about how it scales. How does this repeat? Can I tell the whole story like this? Most of my job is explaining and simplifying things to point out that the customer is saying this, but they mean this. But I have opinions and customers have facts. They have problems. They don’t have solutions. So it’s up to me to say - when you say you need a faster horse - did you mean faster means of transport? Okay, let’s solve that!

What do we need faster transport to do? Does it need to be fair? Does it need to be clean? Does it need to be taken care of? You’re beginning to move from this animate world to the inanimate world, then you can hypothesize things.

Let’s iterate and experiment from there until we combine parts and put a car together. Being able to explain this whole journey of testing for opinions and testing opinions for facts is crucial. The more you’re able to tell that story and be the customer advocate, the better.

You have to analyze what you’re hearing. Here’s what I think they’re saying, here’s how I’m going to test it and build trust because you shouldn’t go into the room thinking I have all the answers. All you have are opinions.

The best thing is having a way to test your opinions to see which of them are facts. That’s how you build trust. So if you’re grounded, you will know a certain fact is true now because of this customer situation, but maybe it won’t be true afterward. If you say I know this because of this reason, you’ll show you have flexibility in doing things and people will innately trust you for that reason.

Q: What are the common strategies you can use for data privacy and anonymization?

Bruke Kifle: Data has been key to powering the rich and personalized experiences that AI has enabled. However, the use of such data must be done responsibly. This means ensuring data privacy to protect sensitive personally and organizationally identifiable information (PII & OII).

This is achieved through a data anonymization process that can preserve the key features of the data while keeping the source anonymous. There are many common techniques for anonymizing user data and ensuring data privacy such as:

Data encryption
Data masking
Generalization
And differential privacy

We are also seeing the popularity of newer techniques such as federated learning (FL), where we can decentralize machine learning and the training of algorithms directly on users’ devices, ensuring personal data doesn’t leave one’s device.