Book Review: "Noise" by Daniel Kahneman, Olivier Sibony, and Cass Sunstein

I recently read the book Noise by Daniel Kahneman, Olivier Sibony, and Cass Sunstein. I will refer to it as the current book, because it was written after the book Thinking, Fast and Slow by Kahneman (one of the authors of the current book); I will refer to the latter book as the previous book because many concepts from the previous book are briefly reviewed in the current book, and as I reviewed the previous book in the post just before this one on this blog [LINK], I will sometimes compare some aspects of the current book to the previous book.

The current book introduces the concepts of statistical noise & statistical bias in human judgments, discusses the psychological biases that can lead to statistical biases & noise (of which statistical noise can be clearly seen even in the absence of clear information about statistical biases), demonstrates how statistical noise can lead to uncontrolled & large variations in human judgments in fields like criminal justice, medicine, forensic science, insurance claims adjustment, corporate hiring, and college admissions, explains the sorts of systematic techniques at individual & organizational levels that can be used to reduce noise in judgments, and discusses some tradeoffs that may be encountered when implementing these noise reduction strategies. The authors' discussion of many of the psychological biases that lead to statistical noise in judgments reviews concepts from the previous book, especially Systems 1 & 2.

When reading the current book, I found myself generally agreeing with the discussions of techniques to reduce noise in domains where the presence of significant statistical noise in judgments is broadly recognized as a severe problem. These techniques include aggregating predictions or evaluations that are made independently, structuring/sequencing discussions among people judging things so that their decisions don't affect each other through emergent group-based social dynamics, carefully accounting for base rates from external information when assessing various internal probabilities, and breaking up decision processes into smaller steps that are more clearly defined in their intent and in example decisions/anchors. It helped a lot that I had read the previous book first, such that even if I didn't remember every detail of every psychological bias presented in both the previous book and the current book, those things looked familiar upon reading them in the current book.

There were also a few new things that I learned from the current book. I learned about how the process of judgment feels so satisfying and infuses confidence into the person making the judgment specifically from the psychological signal of having completed the judgment, which explains why so many people who make professional judgments in many domains are so reluctant to turn their discretion over to more systematic rules or algorithms. I also learned about how simple models of human predictive judgments, when those predictive judgments are about specific outcomes, may do a better job at predicting the outcomes that are the objects of judgment than at predicting the judgments that humans would make, simply because those models lack within-person noise pervasive in human judgments.

However, my overall opinion of the current book was shaped more by the many major and minor (the latter to an appropriately lesser extent) criticisms of it. These minor and major criticisms as well as my concluding remarks will be presented in separate sections as follows after the jump; the spoiling of my concluding remarks is simply that I do not recommend this book to others.

Random Thoughts about the Affordable Care Act

Unless you have been living under a rock these days, you know that this week, the United States Supreme Court ruled (5-4) in favor of the Affordable Care Act (ACA), more popularly known as "Obamacare". What shocked everybody was that Chief Justice John Roberts was in the majority of that ruling. Basically, he and other Justices reinterpreted the clause about the ACA mandating the purchase of health insurance as being a tax rather than forced commerce. In addition, the only major setback to the ACA was the striking down of the clause saying (if I understand this correctly) that states must expand Medicaid coverage to cover low-income people who would otherwise be unable to afford the mandated insurance or else forgo funding for health care entirely. (Please note that I am not anything close to a legal scholar. The following constitutes little more than random mumbling, so you don't have to take it for more than that.)

So ACTA's Getting Passed After All

The title says most of it. It's supposed to be officially signed and enacted this weekend in a special ceremony. For new readers, you might notice in the sidebar a banner asking people to take action against the Anti-Counterfeiting Trade Agreement (also known as ACTA). Clicking on the banner will explain it to you a lot better than I ever could, but basically, there are so many problems with it, like the way it places liability on different people, how it allows for punishment just through accusation (without a fair trial), and stuff like that, just because the big recording and film industries can't adapt to the Interwebz; plus, the whole thing was negotiated in secret and repeated requests to reveal more information were turned down repeatedly for totally bogus reasons, which is completely against the openness and transparency the US is supposed to stand for.
It's official: President Barack Obama's stances on copyrights and patents when it comes to the recent bills he's signed disgust me, more than any other recent president's actions.

On Political Correctness and Retard[suffix]

I've wanted to talk about this for a while, but this (Brent Dykes, Yahoo! News) provides a good starting point. It's about how Obama Administration Chief of Staff Rahm Emanuel called liberal lobbies planning to air attack advertisements on conservative Democrats opposing the health care plan "[expletive] retarded".
I do not at all condone this. He should have attacked the plan on its merits rather than mindlessly foulmouthing it. I also think it would be best if he sincerely apologized for this, though knowing him, it probably won't happen.
Sarah Palin, who has a son with Down Syndrome, has responded by calling for his resignation/firing, saying that this will affect her son and other similarly affected people the same way African Americans would be affected by someone using the 'n'-word to insult them. While I don't agree with this extreme reaction, I think it is entirely reasonable to condemn him as she did.
That said, I wanted to use this opportunity to bring up something bigger about the word "retard"/"retarded" and political correctness.
I don't think it is at all right to use "retard"/"retarded" as an insult against mentally handicapped people. In our schools, we are taught to use words like "mentally handicapped", "mentally disabled", or "mentally challenged", among other words; "retard"/"retarded" aren't even part of this vocabulary (i.e. they are completely ignored altogether). Hence, I have never once met someone who used the word "retard"/"retarded" to [insultingly] refer to a mentally disabled person. This probably isn't true in other parts of the country, but from my own experiences, the word "retard"/"retarded" has solely taken on the meaning of a synonym for "idiot"/"imbecile".
That is probably what Emanuel meant; I still don't support him on this, because as it is, it is a pretty vicious insult to use against someone else.

2010 January 3: National Security and Book Piracy

The first thing I would like to talk about is national security - specifically the recent failed bombing of a Northwest Airlines flight bound for Detroit and the aftermath.
I think the way President Obama handled the issue showed that with respect to national security, he is as out of touch as George W. Bush was. Obama was vacationing in Hawaii when this happened and didn't even make an official response until 72 hours later.
More seriously, the government just couldn't connect the dots with respect to this terrorist. I have chastised Bush for not connecting the obvious dots before 9/11, and I am chastising Obama even more for not connecting the even more obvious dots.
For goodness sake, the terrorist's dad, after the terrorist dissolved ties with his family, went to the US Embassy to warn the ambassadors/other officials of his son's radicalization and to ensure that appropriate action is taken.
Appropriate action, anyone?
Umm, can anyone say oops?
To make matters worse, Secretary of Homeland Security Janet Napolitano said that regarding how the terrorist managed to slip through standard security, "the system worked".
What?
The system worked?
I don't even want to know what it would be like if it didn't work. Apocalypse now, anyone (pun definitely intended)?
Then, Napolitano clarified her previous statement, saying that "in this case, the system didn't work".
That didn't do her any favors (in my eyes).
Then, finally, Obama comes on the scene.
I definitely agree with Newsweek's analysis that his acknowledgement of the heroism of the passengers who subdued the terrorist was lacking, and that he should have been more full-on in the praise. To turn this into action, he should have increased the Citizen Corps's (the volunteer body designed for citizens to fight terrorism like this) budget and not the DHS's.
Furthermore, he said he would [and I'm paraphrasing here] "work with the Yemeni government". (The terrorist received training in Arabic and bombmaking in Yemen.)
What?
His "no-drama" approach has reached its physical limit. He really needs to take a hard stand against this sort of action. What could possibly happen through negotiation with the Yemeni government?
Obama has really dropped the ball on national security. He had better pick it up soon.

The second thing I want to talk about is book piracy - specifically, Native American poet and author Sherman Alexie's take on it.
Basically, he believes it is bad, which I agree with. However, he goes on to say that the open-source culture is fostering this, and that the whole culture is bad. He has also said that e-book readers are "elitist", on the grounds that poor rural people will not be able to buy them.
I think that last statement alone throws out his credibility on talking about new technologies and how they will shape the way we read.
Seriously? "Elitist"?
It's of course not going to be as expensive when it becomes more widespread. Furthermore, a lot of people even now don't have the money to purchase physical books, let alone e-books. That's why they go to the library to read/borrow their books. I foresee a similar thing happening in the future, at which point the readers themselves will be cheap enough for the people referred to by Alexie to publish.
I'm not going to pick on Alexie's misrepresentation of open-source culture, as it was a mix-up of terminology. That would just be mean and unfair of me.
I won't focus on Alexie anymore, as it pretty clear that he is pretty out of touch on the benefits and power such new technologies bestow. Of course, he also believes in going in person around the country to promote his book, especially in small towns; this is admirable, so I will leave him alone (his works seem to be pretty good on their own (i.e. discounting his views on technology)).
The larger point is about the future of reading, writing, and publishing.
With the very low prices of computers and word processing programs (heck, OpenOffice.org is free, as is AbiWord), it is very easy to physically type a book. The problem comes with then distributing it, ensuring that the author derives a fair return for this distribution. Now, it is very easy and cheap (though not totally free) to scan a published paper book and upload it to the Internet.
While reading through the comments on said article in Slashdot, I came across an idea which I thought was genius and should define future publishing; ironically, the basic concept is actually very old.
Charles Dickens used to publish his books in magazines in parts: basically, he would publish chapter 1 in one week's publication of the magazine, chapter 2 in the next week's publication, etc. He made huge profits from this, after which he published the entire book as a whole.
Stephen King has employed a similar system online; he can capitalize on his already-earned fame as a writer to do this. For some of his books, he has published chapter 1 online and would only publish the next chapter if he gets a certain number of legitimate sales for chapter 1 (or the current chapter). The vast majority of readers are willing to pay for the books and would rather acquire the book legally and not illegally, meaning that he has earned record profits from this model. After each chapter's sales target is met, the book is then published as a whole.
I think it's a model that can work for everyone, and it's a great model for its total reliance on the free market to determine whether the author floats or sinks, while creating an (not legally-binding) artificial scarcity that can only be circumvented through further purchase.
Granted, Stephen King can do this due to his already-earned fame, but other authors can and are doing it too. I see a new model in which the first 1 or 2 books by an author are distributed online for free (while donations are accepted), and if a sufficiently large fanbase is gathered, the aforementioned model can be employed.
Libraries could still operate; they could buy the fully-published physical book once it does become published.
Publishers and the WGA, your modus operandi is horribly outdated. People have been complaining about the current system, but until now, no suitable alternative has been proposed. Here it is.
I know there are probably a lot of flaws I haven't thought through. Please post those and any other suggestions/thoughts in the comments!

Obama: Nobel Peace Laureate?

Yes folks, the Nobel Peace Prize committee has awarded President Barack Obama the Nobel Peace Prize (Karl Ritter and Matt Moore, AP).
It's a strange choice, if you ask me.
I like President Obama, and while there are certainly a lot of things he could be doing better, I think he's going in the right general direction.
That said, what has Obama done to deserve this? The nomination was sent in less than a month into his presidency!
Though Obama has taken a lot of steps to try to make the world a better place, such as reductions in nuclear weapons and calls for settlement freezes and peace treaties between Israel and the Palestinian territories, the people he has called on around the world to work with him have not reciprocated. Russia has put pressure on the US on the nuclear issue, Israel refuses to freeze settlements, and China is threatening the US with economic harm if it meets with one of the most recognizable messengers of peace in the world - the Dalaï Lama.
Some of the people who have supported the prize, like Archbishop Desmond Tutu, have done so in the hopes that it would encourage further peace-building.
What?
Shouldn't the Peace Prize be given to people who have already done a plethora of work towards global peace rather than be an encouragement for such actions?
Furthermore, there's a lot of domestic opposition to his attempts to further the cause of peace, arguing that he is an appeaser (he is to the Israelis on the issues of settlement freezes (versus his own previous tough stance) and to the Chinese on meeting with the Dalaï Lama, but otherwise, he is only an appeaser when compared to the super-hard-line Bush).
So why did he get the prize again? I'm sure that there were plenty more people who have already done a lot of work to further the cause of global peace.

Socialist Education! Oh noes!

The big news this week was President Obama's address to students of all ages and their parents about education.
The bigger news was that prominent Republicans were lambasting it before they even knew what it was going to say. They claimed that Obama was going to "indoctrinate" schoolchildren in "leftist socialist ideologies".
Then, of course, Obama gave his speech. It was a speech that made sense - it was about students taking responsibility for their own education and for them to keep trying at it even when the going gets tough.
There was no hint of liberal ideology. In fact, personal responsibility is something many (now deceased) rational conservatives used to espouse. They should be proud.
Furthermore, the same conservatives who blasted the speech prior to its delivery without any basis changed course and praised it afterwards without any apology for bad prior judgment.
Why is this?

Health Care Reform as of 25 August 2009

Things I heard on the radio and read online yesterday about health care reform included how Taiwan and Australia got through health care reform and made health care available to everyone at a low cost.
In Taiwan, apparently, doctors don't get paid nearly as much as they do here. There, they are ordinary middle class citizens (like many would otherwise want to be here) who just want to make a decent living for them and their family and help their patients out. They don't require driving a Mercedes-Benz S-Class or owning a villa in the countryside as a summer home. At the same time, medical school there has only a nominal fee, compared to the exorbitant (>$50000) yearly prices here. They modeled their system after Canada's and the liberal party pushed it as a moral obligation as Taiwan has become a rich country and thus must bring up its poor people too. The conservative party initially resisted it (though not as strongly as Republicans do here) but saw resistance as futile due to increasing push from the liberal party; in fact, the plan got so popular that the conservative party later on tried to take credit for the plan by fully supporting it. Essentially the same thing happened in Australia. Both countries have national health care plans to provide for all citizens decent care.
I wish it could happen that way in the US as well, but it probably won't. The Republican Party is too vocal and too united for it to happen; it appalls me how powerful the party is and how it can make us look like such a backwards country at times. Furthermore, all of the planted protesters at town hall meetings are supposed to gain some sort of publicity for the GOP and thus make it look like that level of dissent is present in the entire country.
It saddens me that Obama is willing to do not the right thing by forcing the issue of a public option but the expedient thing by brushing off the idea to placate his political opponents. Hopefully once some groundwork is lain for a national health care system a public option can again come to the forefront of the debate.

Torture as of 25 August 2009

The biggest news out there now seems to be about the recent decision by the Obama Administration to investigate the actions from the previous Administration regarding "enhanced interrogation techniques" (i.e. torture). There has been a litany of articles out there both for and against investigation. I'm sure many of the articles against it are because investigations will taint the author, but some have legitimate concerns about how it will affect the country if it is done at the same time as the economic recovery, health care reform, and energy reform. My take follows.
President Obama pledged in his Inaugural Address that the government would do the right thing, even (or especially) if it wasn't the easiest or most expedient thing to do. I would say health care reform and the investigation of the previous Administration's actions regarding torture are the best examples of this that exist right now. Most opinions that argue against investigation argue for expedience and moving on. I beg to differ. This is a nation built upon the rule of law, so if we don't at least investigate the actions in question if not prosecute the people in question, we are essentially letting off the previous Administration for actions that could have more easily occurred in a dictatorship. They smashed the law to pieces and we can't let that slide as a nation.
Furthermore, many against investigation, some within the previous Administration (most notably Dick Cheney), have said that the memos about torture would vindicate the means by showing the successful ends (i.e. that torture really did provide valuable information about future terrorist threats). They also add that torture would be invaluable in the ticking time bomb scenario, in which a terrorist suspect who is caught must spill all the beans if a terrorist attack were to happen a few hours later. Both are patently false.
Look at the case of FBI interrogator Ali Soufan (the torture was carried out by the CIA, who essentially dumped FBI interrogators and ignored the information they got though the FBI interrogators were far more experienced with this than their CIA counterparts). He was tasked with interrogating terror suspect Abu Zubaydah; he did not torture once, and he even managed to get the respect of Zubaydah through kinder (that's relative) but persistent questioning. When Zubaydah kept denying his identity, Soufan searched through Zubaydah's files and found that his [Zubaydah's] mother called him "Hani" as a pet name; when Soufan used this against him, he was stunned and gave out a whole lot of information. Soufan managed to gain his respect by giving him sugar-free cookies at a meeting once (due to Zubaydah's diabetes) and they managed to share a laugh at Zubaydah's request for a Coke after he railed against capitalism. Later, when Soufan showed a bunch of photographs of other terror suspects to Zubaydah, the latter identified that of 9/11 mastermind Khalid Sheikh Mohammed by the alias "Mukhtar"; the FBI knew who K. S. Mohammed was but was stumped by the persistent appearence of the alias "Mukhtar" in al-Qaeda's internal communications and could not make the connection until then - no torture required. Then, CIA interrogators essentially kicked out Soufan and began waterboarding Zubaydah; they got 0.0 useful information from him.
Then consider the idea of the ticking time bomb scenario. The idea is that under such constraints of time, torture is the easiest, fastest, and most effective method of interrogation. This is also false, in that rarely does the ticking time bomb scenario occur (it's more of a thought experiment, really), and when it has occurred, regular interrogation has proved as useful if not more so than torture in the same scenario. Look at the case of when Saddam Hussein was captured. Soldiers in Iraq did a few on-the-spot interrogations (on the streets or in homes) of terror suspects and ordinary citizens, using classic psychological methods such as threatening shame upon the person and his family if they didn't talk (among other methods). Successful interrogation led to the capture of Saddam 6 hours later - no torture required.
I'm glad that Obama is pursuing investigations of these actions, as that should bring up our moral standing in the world again.