Product Management Event at #ProductCon Seattle on Building a Data Driven Culture by Ria Sankar, Senior Manager, Analytics for Ethics & Compliance at Microsoft.
15. The Placebo
effect
The importance of
hypothesis-driven
testing
Further Viewing: https://www.ted.com/talks/ben_goldacre_battling_bad_science?language=en#t-839994
Picture Source: https://upload.wikimedia.org/wikipedia/commons/5/55/VariousPills.jpg
Source: Wikimedia Commons
17. Left-handed
Bias
Statistics can be
sinister without
reliable data
Further Reading:
https://marginalrevolution.com/marginalrevolution/2013/09/sinister-statistics-do-left-handed-people-die-young.html
https://www.bbc.com/news/magazine-23988352
https://www.ncbi.nlm.nih.gov/pubmed/2006231
Source: http://en.wikipedia.org/wiki/Handedness
If this were true, being left-handed
~= smoking 120 cigarettes a day
20. 2. Customer Empathy
Understand your
customer’s
functional, social
and emotional
needs
Social
“Give volunteers a
way to connect
with like-minded
friends”
Emotional
“Instill in veterans a
new mission and
sense of purpose”
Functional
“Find the best
volunteers for
disaster response”
Source: TeamRubiconUSA.org
22. • Time to appointment
• Time to care
• Cost of treatment
• Effective treatment
• Provider expertise
• Complexity of care
Case Study: Immediate Clinic
From: Yelp.com
WHEN my child is hurt, I NEED immediate care SO I CAN relax knowing that she is pain-free
26. Qual & Quant effective in understanding new or sparse use cases
Further reading: https://www.linkedin.com/pulse/user-research-turning-data-delightful-experiences-ria-sankar/
28. Invest in data
quality and
tiered data
access
Data
Ingestion
Data
Collection
Data
Storage
Data
Access
Data
Insights
Further Reading: https://www.go-
fair.org/fair-principles/
30. Law of the Vital
Few
80% of effects come
from 20% of causes
Further Reading: The 7 basic quality tools: https://asq.org/quality-resources/seven-basic-quality-tools
Source: https://en.wikipedia.org/wiki/Pareto_chart
32. Belief Offline Test
No Change
Online
Experiment
No Change
Significant
Difference
Ship!
Significant
Difference
Offline metrics, Surveys, Sample
data, Panel Studies
Online Experimentation with AB
or Multi-variate tests
Lifecycle of a Belief
Customer Interaction, Problem
Definition, Data Collection