☆☆➹⁀☆ 3.5 stars☆➹⁀☆☆
What It’s about:
A former Wall Street quant sounds an alarm on mathematical modeling—a pervasive new force in society that threatens to undermine democracy and widen inequality.
We live in the age of the algorithm. Increasingly, the decisions that affect our lives—where we go to school, whether we get a car loan, how much we pay for health insurance—are being made not by humans, but by mathematical models. In theory, this should lead to greater fairness: Everyone is judged according to the same rules, and bias is eliminated. But as Cathy O’Neil reveals in this shocking book, the opposite is true. The models being used today are opaque, unregulated, and uncontestable, even when they’re wrong. Most troubling, they reinforce discrimination: If a poor student can’t get a loan because a lending model deems him too risky (by virtue of his race or neighborhood), he’s then cut off from the kind of education that could pull him out of poverty, and a vicious spiral ensues. Models are propping up the lucky and punishing the downtrodden, creating a “toxic cocktail for democracy.” Welcome to the dark side of Big Data.
Tracing the arc of a person’s life, from college to retirement, O’Neil exposes the black box models that shape our future, both as individuals and as a society. Models that score teachers and students, sort resumes, grant (or deny) loans, evaluate workers, target voters, set parole, and monitor our health—all have pernicious feedback loops. They don’t simply describe reality, as proponents claim, they change reality, by expanding or limiting the opportunities people have. O’Neil calls on modelers to take more responsibility for how their algorithms are being used. But in the end, it’s up to us to become more savvy about the models that govern our lives. This important book empowers us to ask the tough questions, uncover the truth, and demand change.
Weapons of Math Destruction (WMD) is written by a mathematician whose experience as a university math professor, a quant for a hedge fund, and a data scientist makes her uniquely qualified to study and assess the wide-spread use of mathematical models in a multitude of situations. O’Neil discusses a variety of data issues in her book, including how models are developed and maintained, whether or not models can or should be used to assess certain situations, and the potential negative impacts of misuse of mathematical models.
O’Neil presents strong arguments for transparency in any algorithms used. The models need to be fully understood, frequently reassessed, and results questioned so that results are useful and objective. O’Neil also emphasizes that good modeling tools should take to care in the exclusion or inclusion of outlying data points in order to be effective. She provides ample examples of how we are unknowingly, and in her opinion unfairly, affected by data collection tools/modeling. From the ads we see in social media feeds and the results of an internet search to online insurance applications, we are all impacted by pervasive data collection and modeling.
“Models are opinions embedded in mathematics.”
Some of O’Neil’s fairness arguments are not unbiasedly explored. Many of her arguments felt one-sided. Her exploration of the auto-insurance industry is a good example. She discusses insurance rate determination based on crime statistics where one resides. She makes the argument that fair means everyone pays the same rate. Does that mean that someone in a low crime area should pay a higher insurance rate just to achieve equality/fairness with someone who lives in a high crime neighborhood? Or is she arguing that everyone should pay the lower rate (which in my mind is more a social or business question rather than a mathematical modeling question)?
It was interesting to learn of the extensive use of modeling in the criminal justice system and teacher evaluations; O’Neil has some valid points regarding the trade-offs of using algorithms in those situations. I found her discussion of modeling in college admissions to be particularly interesting.
“Ill-conceived mathematical models now micromanage the economy, from advertising to prisons.”
O’Neil’s indignation at certain situations is more than apparent throughout her book. I would have appreciated Weapons of Math Destruction more had she stuck to a straightforward discussion of the mathematical models instead of using her book as a soapbox for her opinions on politics, society and on the “evils” of the affluent.
Weapons of Math Destruction will surely appeal to math geeks, conspiracy freaks and anyone who questions the veracity of statistical analysis. While not always an unbiased presentation, O’Neil does give her readers plenty food for thought.