The Illusion of Transparency
Thursday, October 25th, 2007
In searching for something about overcoming bias in the selection of experimental subjects, I came across this:
http://www.overcomingbias.com/2007/10/illusion-of-tra.html
The illusion of transparancy
October 20, 2007
Illusion of Transparency: Why No One Understands You
In hindsight bias, people who know the outcome of a situation believe the outcome should have been easy to predict in advance. Knowing the outcome, we reinterpret the situation in light of that outcome. Even when warned, we can’t de-interpret to empathize with someone who doesn’t know what we know.
Closely related is the illusion of transparency: We always know what we mean by our words, and so we expect others to know it too. Reading our own writing, the intended interpretation falls easily into place, guided by our knowledge of what we really meant. It’s hard to empathize with someone who must interpret blindly, guided only by the words.
June recommends a restaurant to Mark; Mark dines there and discovers (a) unimpressive food and mediocre service (b) delicious food and impeccable service. Then Mark leaves the following message on June’s answering machine: “June, I just finished dinner at the restaurant you recommended, and I must say, it was marvelous, just marvelous.” Keysar (1994) presented a group of subjects with scenario (a), and 59% thought that Mark’s message was sarcastic and that Jane would perceive the sarcasm. Among other subjects, told scenario (b), only 3% thought that Jane would perceive Mark’s message as sarcastic. Keysar and Barr (2002) seem to indicate that an actual voice message was played back to the subjects.
Keysar (1998) showed that if subjects were told that the restaurant was horrible but that Mark wanted to conceal his response, they believed June would not perceive sarcasm in the (same) message:
“They were just as likely to predict that she would perceive sarcasm when he attempted to conceal his negative experience as when he had a positive experience and was truly sincere. So participants took Mark’s communicative intention as transparent. It was as if they assumed that June would perceive whatever intention Mark wanted her to perceive.”
(The above wording is from Keysar and Barr 2002.)
“The goose hangs high” is an archaic English idiom which has passed out of use in modern language. Keysar and Bly (1995) told one group of subjects that “the goose hangs high” meant that the future looks good; another group of subjects learned that “the goose hangs high” meant the future looks gloomy. Subjects were then asked which of these two meanings an uninformed listener would be more likely to attribute to the idiom. Each group thought that listeners would perceive the meaning presented as “standard”.
(Other idioms tested included “come the uncle over someone”, “to go by the board”, and “to lay out in lavender”. Ah, English, such a lovely language.)
Keysar and Henly (2002) tested the calibration of speakers: Would speakers underestimate, overestimate, or correctly estimate how often listeners understood them? Speakers were given ambiguous sentences (”The man is chasing a woman on a bicycle”) and disambiguating pictures (a man running after a cycling woman), then asked the speakers to utter the words in front of addressees, then asked speakers to estimate how many addressees understood the intended meaning. Speakers thought that they were understood in 72% of cases and were actually understood in 61% of cases. When addressees did not understand, speakers thought they did in 46% of cases; when addressees did understand, speakers thought they did not in only 12% of cases.
Additional subjects who overheard the explanation showed no such bias, expecting listeners to understand in only 56% of cases.
As Keysar and Barr (2002) note, two days before Germany’s attack on Poland, Chamberlain sent a letter intended to make it clear that Britain would fight if any invasion occurred. The letter, phrased in polite dipliomatese, was heard by Hitler as conciliatory - and the tanks rolled.
Be not too quick to blame those who misunderstand your perfectly clear sentences, spoken or written. Chances are, your words are more ambiguous than you think.
Keysar, B. (1994). The illusory transparency of intention: Linguistic perspective taking in text. Cognitive Psychology, 26, 165-208.
Keysar, B. (1998). Language users as problem solvers: Just what ambiguity problem do they solve? In S. R. Fussell and R. J. Kreuz (Eds.), Social and Cognitive Psychological Approaches to Interpersonal Communication, 175-200. Hillsdale, NJ: Erlbaum.
Keysar, B., & Barr, D. J. (2002). Self anchoring in conversation: Why language users do not do what they “should”. (2002). In T. Gilovich, D. W. Griffin, & D. Kahneman (Eds.), Heuristics and biases: The psychology of intuitive judgment. (pp. 150-166). Cambridge University Press.
Keysar, B., & Bly, B. (1995.) Intuitions of the transparency of idioms: Can one keep a secret by spilling the beans? Journal of Memory and Language, 34, pp. 89-109.
Keysar, B., & Henly, A. S. (2002). Speakers’ overestimation of their effectiveness. Psychological Science, 13, 207-212.
Posted by Eliezer Yudkowsky at 07:49 PM in Standard Biases Permalink
TrackBack
TrackBack URL for this entry:http://www.typepad.com/t/trackback/475590/22620244
Listed below are links to weblogs that reference Illusion of Transparency: Why No One Understands You:
Comments
Google found the actual meanings of “the goose hangs high” (the future looks good), and the reason it means that (there is a goose in the larder, therefor you will not starve), and “to go by the board” (to be completely destroyed or overthrown), but not the reason behind it. What do “to come an uncle over someone” and “to lay out in lavender” mean?
Posted by: chesh October 20, 2007 at 08:05 PM
I’ve heard things like this before. This is one reason why it’s a good idea to run your writing by someone else before you “publish”; you can get a better idea of whether what you wrote actually says what you wanted it to.
Posted by: Doug S. October 20, 2007 at 08:18 PM
1) That’s why Deidre (then Donald) McCloskey wrote in (I believe), “The Rhetoric of Economics” (JEL 1983), that, “If the reader says something is unclear, it is unclear.”2) The recent Rush Limbaugh imbroglio is consistent with the point about inferring meaning. The political leanings of the commentors guided their inerpretation of his words.
Posted by: Acad Ronin October 20, 2007 at 11:27 PM
Humor seems the most difficult. Once I taught a guy from Sweden English. One day he read an article in the newspaper to me and could tell me what it meant– no problem.When we got to the comics page… the humor didn’t translate at all.I still don’t know why the difference was so marked.
Posted by: douglas October 21, 2007 at 12:36 AM
Certain kinds of academics use this to justify a rhetorical technique: they simply say they do not understand what you say, and they keep repeating this for any statements that do not fit with their standard method. For example, in a community uses formal statistical techniques, they say they cannot understand any argument for a probabilistic conclusion that is not expressed as a formal statistical test.
Posted by: Robin Hanson October 21, 2007 at 04:51 AM
Robin, could you give a bit more detail?
Posted by: Scholar October 21, 2007 at 09:33 AM
I’d have to hear the voice mail message to believe what you’re saying. In my daily life, when one of my friends says, “That’s great. Just great” it’s totally clear whether he’s being sarcastic or not. Because he intends to sound sarcastic, and changes his inflection.
Now in this case the message was the same, so the inflection didn’t carry any content. But the subjects were told Mark was leaving a message for his friend. So it’s a reasonable assumption that Mark is using an inflection that his friend will recognize as sarcasm. I would assume that unless, as in the second case, they told me he was trying not to show sarcasm.
Posted by: Noumenon October 21, 2007 at 11:53 PM
Noumenon, one of the experiments (I think Keysar 1994) showed that, when people read in text that Mark left a voice message (that is, the experimental subjects themselves did not hear a spoken message, only read about Mark leaving one); or alternatively read in text that Mark left a written note; then they were equally likely to judge that June would identify sarcasm. In other words, the modality of text versus speech made no difference to their expectations.
Posted by: Eliezer Yudkowsky October 22, 2007 at 12:19 AM
Okay, then, that’s just weird. To me there’s obviously not enough information in a text message to tell sarcasm that way. Even a voice message wouldn’t be enough for someone who didn’t know Mark, like the restaurant manager.
I can’t really imagine myself in this scenario any more. My takeaway is that some people are dumb about text messages (or were in 1994 before the smiley).
Posted by: Noumenon October 22, 2007 at 03:03 AM
If they were told Mark left a voice message, it would be rational to assume June would identify sarcasm, because it would be rational to assume that Mark left it in a sarcastic tone (or not) if he intended sarcasm (or not). Though the written note case shows that’s not what’s really going on, unfortunately.
Posted by: Nick Tarleton October 22, 2007 at 09:37 AM
Hi Marty hereBack in the Bush we have many words to describe our environment that is mostly sand. The words are always the same but how we say the word describes what we mean example one click then word Danger under the sand maybe snake!Loud click then soft click then word slow rolling sand wind from the south face not good for hunting now. Click pop hand over our mouth then word means to hot for traveling best to go around. The word never changes or the meaning will be lost in enemy? Environment it is very important meaning is always understood life depends on meaning so I am confused about phone message Why not just say food was not good why leave meaning hanging to interpretation? This is not honest say the truth the person listening did not make the food so I say he should be honest. Or I, am I misunderstanding something in the article?
Posted by: Marty October 23, 2007 at 11:58 AM
I believe the reason for saying the opposite of what you mean is so that you and your friend can say, “We’re clever. Others would not perceive our nuanced tone and would stupidly think we meant what we said.” In the bush, don’t people sometimes make the sound for “snake” when there is no snake, just so they can laugh and mock the person who foolishly believed there was a snake?
Posted by: Noumenon October 23, 2007 at 01:35 PM
Chesh, lay out in lavender means “displayed in the best possible light” according to http://www.phrases.org.uk/meanings/222725.html. If you google “out in lavender” or “laid out in lavender”, you get more hits.
Posted by: Whit October 23, 2007 at 02:21 PM
Dear Whit, yes,when there is no real danger like at a party the youngsters may play everyone knowns it is harmless. Never in the bush this would be very bad manners to frighten for a laugh beside what pleasure is gained by such unkind act to me that is a foolish game and being mean is not showing cleverness. Marty
Posted by: Marty October 23, 2007 at 03:58 PM
Marty, where specifically are you from?
Posted by: TGGP October 23, 2007 at 09:46 PM
It should have been perfectly obvious to Julie that I was being (a) sarcastic, (b) sincere.
Posted by: Mark October 24, 2007 at 12:30 AM
Talk about clever — Mark illustrates this entire post in one sentence! He points out all these readings:
It should have been perfectly obvious to Julie that I was being sarcastic. (sarcastic)It should have been perfectly obvious to Julie that I was being sarcastic. (sincere)It should have been perfectly obvious to Julie that I was being sincere. (sarcastic)It should have been perfectly obvious to Julie that I was being sincere. (sincere)
and the point of the post is that you really can’t tell from the text which way he means it or how he really felt.
Thursday, October 18, 2007
Of course there is such a …
Thursday, October 18th, 2007
http://www.ethicalhacker.net/content/view/160/8/
Thursday, October 18th, 2007
http://www.ethicalhacker.net/content/view/160/8/
Sunday, October 14, 2007
game theory in political science …
Sunday, October 14th, 2007
first we have to buy in on the idea that science can apply to politics, which is already going well beyond the idea that science can apply to economics … in any case, here is one guy trying -
—-
http://www.goodmagazine.com/section/Features/the_new_nostradamus
Technology
The New Nostradamus
Words By Michael A.M. LernerPhotos By Ethan Hill
Can a fringe branch of mathematics forecast the future? A special adviser to the CIA, Fortune 500 companies, and the U.S. Department of Defense certainly thinks so.
If you listen to Bruce Bueno de Mesquita, and a lot of people don’t, he’ll claim that mathematics can tell you the future. In fact, the professor says that a computer model he built and has perfected over the last 25 years can predict the outcome of virtually any international conflict, provided the basic input is accurate. What’s more, his predictions are alarmingly specific. His fans include at least one current presidential hopeful, a gaggle of Fortune 500 companies, the CIA, and the Department of Defense. Naturally, there is also no shortage of people less fond of his work. “Some people think Bruce is the most brilliant foreign policy analyst there is,” says one colleague. “Others think he’s a quack.”
Today, on a rare sunny summer day in San Francisco, Bueno de Mesquita appears to be neither. He’s relaxing in his stately home, answering my questions with exceeding politesse. Sunlight streams through the tall windows, the melodic sound of a French horn echoing from somewhere upstairs; his daughter, a musician in a symphony orchestra, is practicing for an upcoming recital. It’s all so complacent and genteel, which is exactly what Bueno de Mesquita isn’t. As if on cue, a question sets him off. “I found it to be offensive,” he says about a colleague’s critique of his work. “This is absolutely, totally, and utterly false,” he says about the attack of another.
The criticism rankles him, because, to his mind, the proof is right there on the page. “I’ve published a lot of forecasting papers over the years,” he says. “Papers that are about things that had not yet happened when the paper was published but would happen within some reasonable amount of time. There’s a track record that I can point to.” And indeed there is. Bueno de Mesquita has made a slew of uncannily accurate predictions—more than 2,000, on subjects ranging from the terrorist threat to America to the peace process in Northern Ireland—that would seem to prove him right.
“The days of the digital watch are numbered,” quipped Tom Stoppard. After spending a few hours with Bueno de Mesquita, you might come to believe that so is everything else. Numbered as in “mathematics”—more precisely, game theory, an esoteric branch of mathematics used to analyze interaction. “Game theory is math for how people behave strategically,” Bueno de Mesquita says.
Bueno de Mesquita has big ideas, and he’s more than happy to put his career on the line for them. Back in March 2004, when al-Qaeda bombed a Madrid train station, influencing the course of Spain’s general election three days later, a lot of U.S. security folks were nervous. Worried that al-Qaeda might try something similar here in the run-up to the November, 2004, presidential elections, the Pentagon hired Bueno de Mesquita to run some data through his forecasting model to tell them what to expect. The results were unequivocal. “I said there would be no homeland attack. I also indicated that bin Laden’s second-in-command, Ayman al-Zawahiri, would resurface around Thanksgiving, 2004,” he says. Just after the elections in November that year, Zawahiri released a new videotape. Bueno de Mesquita was right on both counts. “One of the things government needs most is advice that’s not wishy-washy. I try to be as precise as I can.”
For the record, this man is not some lunatic soothsayer sequestered in a musty, forgotten basement office. He is the chairman of New York University’s Department of Politics, a senior fellow at the Hoover Institution at Stanford, and the author of many weighty academic tomes. He regularly consults with the CIA and the Department of Defense—most recently on such hot-button topics as Iran and North Korea—and has a new book coming out in the fall that he cowrote with his pal Secretary of State Condoleezza Rice. His curriculum vitae, which details his various Ph.Ds, academic appointments, editorial-board memberships, writings, honors, awards, and grants, runs 17 small-font pages long.
He is wildly controversial, though. As one of the foremost scholars of game theory—or “rational choice,” as its political-science practitioners prefer to call it—Bueno de Mesquita is at the center of a raging hullabaloo that has taken over some of the most prestigious halls of learning in this country. Exclusive, highly complex mathematically, and messianic in its certainty of universal truths, rational-choice theory is not only changing the way political science is taught, but the way it’s defined.
To verify the accuracy of his model, the CIA set up a kind of forecasting face-off that pit predictions from his model against those of Langley’s more traditional in-house intelligence analysts and area specialists. “We tested Bueno de Mesquita’s model on scores of issues that were conducted in real time—that is, the forecasts were made before the events actually happened,” says Stanley Feder, a former high-level CIA analyst. “We found the model to be accurate 90 percent of the time,” he wrote. Another study evaluating Bueno de Mesquita’s real-time forecasts of 21 policy decisions in the European community concluded that “the probability that the predicted outcome was what indeed occurred was an astounding 97 percent.” What’s more, Bueno de Mesquita’s forecasts were much more detailed than those of the more traditional analysts. “The real issue is the specificity of the accuracy,” says Feder. “We found that DI (Directorate of National Intelligence) analyses, even when they were right, were vague compared to the model’s forecasts. To use an archery metaphor, if you hit the target, that’s great. But if you hit the bull’s eye—that’s amazing.”
How does Bueno de Mesquita do this? With mathematics. “You start with a set of assumptions, as you do with anything, but you do it in a formal, mathematical way,” he says. “You break them down as equations and work from there to see what follows logically from those assumptions.” The assumptions he’s talking about concern each actor’s motives. You configure those motives into equations that are, essentially, statements of logic based on a predictive theory of how people with those motives will behave. From there, you start building your mathematical model. You determine whether the predictive theory holds true by plugging in data, which are numbers derived from scales of preferences that you ascribe to each actor based on the various choices they face.
The Prisoner’s Dilemma, a basic in game theory, explains it well: Two burglars are apprehended near the scene of a crime and are interrogated separately by the police. The police know these two goons did it, but they don’t know how, so they offer each one a deal. If they both confess and cooperate, they’ll both get a minor sentence of five years. If neither man confesses, they’ll both only get one year (for having been caught with some of the stolen loot on them). But, and here’s where it gets interesting, if one confesses and the other doesn’t, the one who confesses walks out scot-free while the other will do 10 years. What will they do? Will they trust each other and do what’s obviously in their best interest, which is not confess? Based on game theory’s assumptions about human nature, the math derived from this dilemma tells you squarely that the two goons will turn each other in.
“In the foreboding world view of rational choice, everyone is a raging dirtbag.”
Which illustrates the next incontrovertible fact about game theory: In the foreboding world view of rational choice, everyone is a raging dirtbag. Bueno de Mesquita points to dictatorships to prove his point: “If you liberate people from the constraint of having to satisfy other people in order to advance themselves, people don’t do good things.” When analyzing a problem in international relations, Bueno de Mesquita doesn’t give a whit about the local culture, history, economy, or any of the other considerations that more traditional political scientists weigh. In fact, rational choicers like Bueno de Mesquita tend to view such traditional approaches with a condescension bordering on disdain. “One is the study of politics as an expression of personal opinion as opposed to political science,” he says dryly. His only concern is with what the political actors want, what they say they want (often two very different things), and how each of their various options will affect their career advancement. He feeds this data into his computer model and out pop the answers.
Though controversial in the academic world, Bueno de Mesquita and his model have proven quite popular in the private sector. In addition to his teaching responsibilities and consulting for the government, he also runs a successful private business, Mesquita & Roundell, with offices in Rockefeller Center. Advising some of the top companies in the country, he earns a tidy sum: Mesquita & Roundell’s minimum fee is $50,000 for a project that includes two issues. Most projects involve multiple issues. “I’m not selling my wisdom,” he says. “I’m selling a tool that can help them get better results. That tool is the model.”
“In the private sector, we deal with three areas: litigation, mergers and acquisitions, and regulation,” he says. “On average in litigation, we produce a settlement that is 40 percent better than what the attorneys think is the best that can be achieved.” While Bueno de Mesquita’s present client list is confidential, past clients include Union Carbide, which needed a little help in structuring its defense after its 1984 chemical-plant disaster in Bhopal, India, claimed the lives of an estimated 22,000 people; the giant accounting firm Arthur Andersen; and British Aerospace during its merger with GEC-Marconi.
But there are limits to what his company will do. For example, Bueno de Mesquita may already know, but he won’t say who’ll succeed George W. Bush in the White House. “We have a corporate policy that we will not, on a commercial basis, use the model in campaigns,” he says. “We don’t think it’s appropriate to manipulate the democratic process. We won’t take a client who wants to manipulate U.S. government policy, even if we agree with the manipulation. And we won’t take a foreign client whose objectives are contrary to the objectives of the United States government.”
There’s also the book he’s written with Condoleezza Rice and two other authors, The Strategy of Campaigning, which comes out in the fall. Given the Bush administration’s heavy ideological bent—which would seem to represent everything a rationalist like Bueno de Mesquita opposes—how does he justify putting his name on the same dust jacket as Rice’s? Bueno de Mesquita repositions himself in his chair. “The central question in this book is a question that Condi raised before she came to Washington,” he says. (So is her name there just to sell books? “We are making a concerted effort not to play up the fact that the Secretary of State is a co-author,” he later adds.)
Meanwhile, he has just launched and is the director of NYU’s Alexander Hamilton Center. “The mission for the center is the application of logic and evidence to solving fundamental policy problems. Not to a bipartisan solution, but to a nonpartisan solution.” In his continuing work for the CIA and the Defense Department, one of his most recent assignments has been North Korea and its nuclear program. His analysis starts from the premise that what Kim Jong Il cares most about is his political survival. As Bueno de Mesquita sees it, the principal reason for his nuclear program is to deter the United States from taking him out, by raising the costs of doing so. “The solution, then, lies in a mechanism that guarantees us that he not use these weapons and guarantees him that we not interfere with his political survival,” he says.
“They said my work was evil, offensive, that it should be suppressed. It was a very difficult time in my career.—Bruce Bueno de Mesquita”
Perhaps not coincidentally, the recent agreement that the United States reached with the government of Pyongyang closely resembles the one that Bueno de Mesquita’s model suggested: Kim agrees to dismantle his existing nuclear weapons but not his existing nuclear capability. “He puts it in mothballs with IAEA (International Atomic Energy Agency) inspectors on site 24 hours a day, 365 days a year. And in exchange, we provide him with $1.2 billion a year, which we label ‘foreign aid,’ of course.” The “foreign-aid” figure published in the newspapers was $400 million, which concerns Bueno de Mesquita. “I read that and I said, I hope that’s not the deal because it’s not enough money. He needs $1.2 billion, approximately, to sustain the loyalty of his cronies in the military and so forth. It’s unpleasant, this is a nasty man, but we’re stuck with it. The nice part of the deal is that it’s self-enforcing. Each side has a reason to credibly commit to their part of the deal.”
Recently, he’s applied his science to come up with some novel ideas on how to resolve the Israeli-Palestinian conflict. “In my view, it is a mistake to look for strategies that build mutual trust because it ain’t going to happen. Neither side has any reason to trust the other, for good reason,” he says. “Land for peace is an inherently flawed concept because it has a fundamental commitment problem. If I give you land on your promise of peace in the future, after you have the land, as the Israelis well know, it is very costly to take it back if you renege. You have an incentive to say, ‘You made a good step, it’s a gesture in the right direction, but I thought you were giving me more than this. I can’t give you peace just for this, it’s not enough.’ Conversely, if we have peace for land—you disarm, put down your weapons, and get rid of the threats to me and I will then give you the land—the reverse is true: I have no commitment to follow through. Once you’ve laid down your weapons, you have no threat.”
Bueno de Mesquita’s answer to this dilemma, which he discussed with the former Israeli prime minister and recently elected Labor leader Ehud Barak, is a formula that guarantees mutual incentives to cooperate. “In a peaceful world, what do the Palestinians anticipate will be their main source of economic viability? Tourism. This is what their own documents say. And, of course, the Israelis make a lot of money from tourism, and that revenue is very easy to track. As a starting point requiring no trust, no mutual cooperation, I would suggest that all tourist revenue be [divided by] a fixed formula based on the current population of the region, which is roughly 40 percent Palestinian, 60 percent Israeli. The money would go automatically to each side. Now, when there is violence, tourists don’t come. So the tourist revenue is automatically responsive to the level of violence on either side for both sides. You have an accounting firm that both sides agree to, you let the U.N. do it, whatever. It’s completely self-enforcing, it requires no cooperation except the initial agreement by the Israelis that they are going to turn this part of the revenue over, on a fixed formula based on population, to some international agency, and that’s that.”
His first foray into forecasting controversy took place in 1984, when he published an article in PS, the flagship journal of the American Political Science Association, predicting who would succeed Iran’s ruling Ayatollah Khomeini upon his death. He had developed a rudimentary forecasting model that was different from anything anyone had seen before in that it was not designed around one particular foreign-policy problem, but could be applied to any international conflict. “It was the first attempt at a general mathematical model of international conflict,” he says. His model predicted that upon Khomeini’s death, an ayatollah named Hojatolislam Khamenei and an obscure junior cleric named Akbar Hashemi Rafsanjani would emerge to lead the country together. At the time, Rafsanjani was so little known that his name had yet to appear in the New York Times.
Even more improbably, Khomeini had already designated his successor, and it was neither Ayatollah Khamenei nor Rafsanjani. Khomeini’s stature among Iran’s ruling clerics made it inconceivable that they would defy their leader’s choice. At the APSA meeting subsequent to the article’s publication, Bueno de Mesquita was roundly denounced as a quack by the Iran experts—a charlatan peddling voodoo mathematics. “They said I was an idiot, basically. They said my work was evil, offensive, that it should be suppressed,” he recalls. “It was a very difficult time in my career.” Five years later, when Khomeini died, lo and behold, Iran’s fractious ruling clerics chose Ayatollah Khamenei and Hashemi Rafsanjani to jointly lead the country. At the next APSA meeting, the man who had been Bueno de Mesquita’s most vocal detractor raised his hand and publicly apologized to him.
Bueno de Mesquita had arrived, and so, too, had rational-choice theory. Rational choicers began sprouting up in political-science departments around the country and, say their critics, strangling anyone and anything in their way. By 2000, according to one estimate, some 40 percent of all articles published in the prestigious American Political Science Review were rational-choice themed. Increasingly, graduate students in political science viewed a fluency in formal mathematic modeling as a prerequisite for career advancement. And the leaps in technology taking place only fueled rational choice’s advance: faster, more powerful computers allowed rational choicers to build bigger, ever more complex models that could be applied to ever more complex situations. And, naturally enough, an intellectual counteroffensive was launched.
It began in 1994 when two Yale political-science professors, Donald Green and Ian Shapiro, published their book, Pathologies of Rational Choice Theory, which disputed much of the scientific underpinnings that rational choice claimed for itself. In essence, the authors said that when rational choice was actually put to the practical test, much of it simply didn’t work. This was followed by a 1999 (lightning speed in academia) article by Stephen M. Walt in the journal International Security called “Rigor or Rigor Mortis?” Walt, a political-science professor at Harvard’s John F. Kennedy School of Government, conceded some value to formal modeling but ultimately likened rational choice to a “cult of irrelevance” that stifled creativity and had little practical value in actual policy formulation. Most vexing, Walt accused rational choicers of regarding nonrational choice theorists such as himself as “methodological Luddites whose opposition rests largely on ignorance.”
“We found that [national intelligence] analyses, even when they were right, were vague compared to [Bueno de Mesquita’s] forecasts. If you hit the target, that’s great. But if you hit the bull’s eye—that’s amazing.—Stanley Feder, former CIA analyst”
Since no one snaps a towel back harder than a scorned academic, Bueno de Mesquita and several of his rational-choice cohorts immediately mounted a blistering counter-counteroffensive, firing off a series of lengthy rebuttals to Walt’s piece that deconstructed his criticism, questioned his facts, and cited what was in their view Walt’s muddled logic as a prime example of why rational choice was so desperately needed in the field. “In the piece that Steve Walt wrote, in which he acknowledged that logical consistency was important, he also argued that it was overrated, that it stifled creativity. To me this is a bizarre idea,” says Bueno de Mesquita, “because really what that statement means to me is, if you relax logical consistency, you can say whatever you feel like and therefore you are back to a world in which the study of politics is the expression of personal opinion instead of being political science. It’s the art of politics or the articulation of beliefs, which is what dominates much of advising to government. It’s rhetoric.”
The brouhaha culminated at a raucous APSA meeting in 2001 at San Francisco’s Hilton Hotel with the open revolt of a group of major-league political scientists who, one by one, took to the podium to rail against rational choice and its encroaching methodological orthodoxy. Dubbed the “Perestroika Movement” by its anonymous founder (apparently, rational-choice folk are a powerful and vindictive lot), the dissident group vowed to take a stand against “the domination of mathematical approaches to the discipline.” There is a “hegemonic threat out there,” warned John J. Mearsheimer, a noted professor of international relations at the University of Chicago. “This is about the mathematicization of political science,” he said. “I’m in favor of filling the zoo with all kinds of animals. But I’m concerned about them running us out of the business or making us marginal.” Ultimately, the Perestroikans did win some concessions: a new editor of the APSR who vowed to make the flagship journal more hospitable to mathematics-free articles and a pledge from the APSA to open up its method of appointing officers. “The APSA had become dominated by those practicing so-called rigorous analyses,” says Walt. “Now the pendulum has swung back a bit.”
For Bueno de Mesquita, getting his methodology accepted by the policy-making establishment remains somewhat of an uphill slog. The most pointed criticism of rational choice has been that, unlike with more traditional political scientists, very little cross-pollination takes place between rational-choice academics and government policy-makers. Bueno de Mesquita says it’s just a matter of time before that changes. “Because people who are in a position to appoint people weren’t trained in this way, they don’t feel as comfortable as with people who were trained in what I would describe as a less rigorous form of study of politics. And, so, the folks who do more rigorous work typically don’t get invited in,” he says. Of course, the same was true of economics 40 years ago when nontechnical types like John Kenneth Galbraith dominated the field. Paul Samuelson and Milton Friedman changed all that, and Bueno de Mesquita sees himself playing the same role for politics.
Bueno de Mesquita remains unfazed, ever certain that rational choice will ultimately prevail. “When I moved to Rochester in 1973, if you wanted to be trained in this kind of political science, you could go to Rochester, period,” he says. “Ten years later, you could go to Rochester, Caltech, and Washington University in St. Louis. If you asked me today, you could go to the places I just mentioned, and you could go to NYU, you could go to Stanford—there’s a long list of places you could go. Except, of course, Harvard. But it will happen there, too. I’m on their syllabus.”
Back to the Future
A sample of Bruce Bueno de Mesquita’s wilder—and most accurate—predictions
Forecasted the second Intifada and the death of the Mideast peace process, two years before it happened.
Defied Russia specialists by predicting who would succeed Brezhnev. “The model identified Andropov, who nobody at the time even considered a possibility,” he says.
Predicted that Daniel Ortega and the Sandanistas would be voted out of office in Nicaragua, two years before it happened.
Four months before Tiananmen Square, said China’s hardliners would crack down harshly on dissidents.
Predicted France’s hair’s-breadth passage of the European Union’s Maastricht Treaty.
Predicted the exact implementation of the 1998 Good Friday Agreement between Britain and the IRA.
Predicted China’s reclaiming of Hong Kong and the exact manner the handover would take place, 12 years before it happened.
Sunday, October 14th, 2007
first we have to buy in on the idea that science can apply to politics, which is already going well beyond the idea that science can apply to economics … in any case, here is one guy trying -
—-
http://www.goodmagazine.com/section/Features/the_new_nostradamus
Technology
The New Nostradamus
Words By Michael A.M. LernerPhotos By Ethan Hill
Can a fringe branch of mathematics forecast the future? A special adviser to the CIA, Fortune 500 companies, and the U.S. Department of Defense certainly thinks so.
If you listen to Bruce Bueno de Mesquita, and a lot of people don’t, he’ll claim that mathematics can tell you the future. In fact, the professor says that a computer model he built and has perfected over the last 25 years can predict the outcome of virtually any international conflict, provided the basic input is accurate. What’s more, his predictions are alarmingly specific. His fans include at least one current presidential hopeful, a gaggle of Fortune 500 companies, the CIA, and the Department of Defense. Naturally, there is also no shortage of people less fond of his work. “Some people think Bruce is the most brilliant foreign policy analyst there is,” says one colleague. “Others think he’s a quack.”
Today, on a rare sunny summer day in San Francisco, Bueno de Mesquita appears to be neither. He’s relaxing in his stately home, answering my questions with exceeding politesse. Sunlight streams through the tall windows, the melodic sound of a French horn echoing from somewhere upstairs; his daughter, a musician in a symphony orchestra, is practicing for an upcoming recital. It’s all so complacent and genteel, which is exactly what Bueno de Mesquita isn’t. As if on cue, a question sets him off. “I found it to be offensive,” he says about a colleague’s critique of his work. “This is absolutely, totally, and utterly false,” he says about the attack of another.
The criticism rankles him, because, to his mind, the proof is right there on the page. “I’ve published a lot of forecasting papers over the years,” he says. “Papers that are about things that had not yet happened when the paper was published but would happen within some reasonable amount of time. There’s a track record that I can point to.” And indeed there is. Bueno de Mesquita has made a slew of uncannily accurate predictions—more than 2,000, on subjects ranging from the terrorist threat to America to the peace process in Northern Ireland—that would seem to prove him right.
“The days of the digital watch are numbered,” quipped Tom Stoppard. After spending a few hours with Bueno de Mesquita, you might come to believe that so is everything else. Numbered as in “mathematics”—more precisely, game theory, an esoteric branch of mathematics used to analyze interaction. “Game theory is math for how people behave strategically,” Bueno de Mesquita says.
Bueno de Mesquita has big ideas, and he’s more than happy to put his career on the line for them. Back in March 2004, when al-Qaeda bombed a Madrid train station, influencing the course of Spain’s general election three days later, a lot of U.S. security folks were nervous. Worried that al-Qaeda might try something similar here in the run-up to the November, 2004, presidential elections, the Pentagon hired Bueno de Mesquita to run some data through his forecasting model to tell them what to expect. The results were unequivocal. “I said there would be no homeland attack. I also indicated that bin Laden’s second-in-command, Ayman al-Zawahiri, would resurface around Thanksgiving, 2004,” he says. Just after the elections in November that year, Zawahiri released a new videotape. Bueno de Mesquita was right on both counts. “One of the things government needs most is advice that’s not wishy-washy. I try to be as precise as I can.”
For the record, this man is not some lunatic soothsayer sequestered in a musty, forgotten basement office. He is the chairman of New York University’s Department of Politics, a senior fellow at the Hoover Institution at Stanford, and the author of many weighty academic tomes. He regularly consults with the CIA and the Department of Defense—most recently on such hot-button topics as Iran and North Korea—and has a new book coming out in the fall that he cowrote with his pal Secretary of State Condoleezza Rice. His curriculum vitae, which details his various Ph.Ds, academic appointments, editorial-board memberships, writings, honors, awards, and grants, runs 17 small-font pages long.
He is wildly controversial, though. As one of the foremost scholars of game theory—or “rational choice,” as its political-science practitioners prefer to call it—Bueno de Mesquita is at the center of a raging hullabaloo that has taken over some of the most prestigious halls of learning in this country. Exclusive, highly complex mathematically, and messianic in its certainty of universal truths, rational-choice theory is not only changing the way political science is taught, but the way it’s defined.
To verify the accuracy of his model, the CIA set up a kind of forecasting face-off that pit predictions from his model against those of Langley’s more traditional in-house intelligence analysts and area specialists. “We tested Bueno de Mesquita’s model on scores of issues that were conducted in real time—that is, the forecasts were made before the events actually happened,” says Stanley Feder, a former high-level CIA analyst. “We found the model to be accurate 90 percent of the time,” he wrote. Another study evaluating Bueno de Mesquita’s real-time forecasts of 21 policy decisions in the European community concluded that “the probability that the predicted outcome was what indeed occurred was an astounding 97 percent.” What’s more, Bueno de Mesquita’s forecasts were much more detailed than those of the more traditional analysts. “The real issue is the specificity of the accuracy,” says Feder. “We found that DI (Directorate of National Intelligence) analyses, even when they were right, were vague compared to the model’s forecasts. To use an archery metaphor, if you hit the target, that’s great. But if you hit the bull’s eye—that’s amazing.”
How does Bueno de Mesquita do this? With mathematics. “You start with a set of assumptions, as you do with anything, but you do it in a formal, mathematical way,” he says. “You break them down as equations and work from there to see what follows logically from those assumptions.” The assumptions he’s talking about concern each actor’s motives. You configure those motives into equations that are, essentially, statements of logic based on a predictive theory of how people with those motives will behave. From there, you start building your mathematical model. You determine whether the predictive theory holds true by plugging in data, which are numbers derived from scales of preferences that you ascribe to each actor based on the various choices they face.
The Prisoner’s Dilemma, a basic in game theory, explains it well: Two burglars are apprehended near the scene of a crime and are interrogated separately by the police. The police know these two goons did it, but they don’t know how, so they offer each one a deal. If they both confess and cooperate, they’ll both get a minor sentence of five years. If neither man confesses, they’ll both only get one year (for having been caught with some of the stolen loot on them). But, and here’s where it gets interesting, if one confesses and the other doesn’t, the one who confesses walks out scot-free while the other will do 10 years. What will they do? Will they trust each other and do what’s obviously in their best interest, which is not confess? Based on game theory’s assumptions about human nature, the math derived from this dilemma tells you squarely that the two goons will turn each other in.
“In the foreboding world view of rational choice, everyone is a raging dirtbag.”
Which illustrates the next incontrovertible fact about game theory: In the foreboding world view of rational choice, everyone is a raging dirtbag. Bueno de Mesquita points to dictatorships to prove his point: “If you liberate people from the constraint of having to satisfy other people in order to advance themselves, people don’t do good things.” When analyzing a problem in international relations, Bueno de Mesquita doesn’t give a whit about the local culture, history, economy, or any of the other considerations that more traditional political scientists weigh. In fact, rational choicers like Bueno de Mesquita tend to view such traditional approaches with a condescension bordering on disdain. “One is the study of politics as an expression of personal opinion as opposed to political science,” he says dryly. His only concern is with what the political actors want, what they say they want (often two very different things), and how each of their various options will affect their career advancement. He feeds this data into his computer model and out pop the answers.
Though controversial in the academic world, Bueno de Mesquita and his model have proven quite popular in the private sector. In addition to his teaching responsibilities and consulting for the government, he also runs a successful private business, Mesquita & Roundell, with offices in Rockefeller Center. Advising some of the top companies in the country, he earns a tidy sum: Mesquita & Roundell’s minimum fee is $50,000 for a project that includes two issues. Most projects involve multiple issues. “I’m not selling my wisdom,” he says. “I’m selling a tool that can help them get better results. That tool is the model.”
“In the private sector, we deal with three areas: litigation, mergers and acquisitions, and regulation,” he says. “On average in litigation, we produce a settlement that is 40 percent better than what the attorneys think is the best that can be achieved.” While Bueno de Mesquita’s present client list is confidential, past clients include Union Carbide, which needed a little help in structuring its defense after its 1984 chemical-plant disaster in Bhopal, India, claimed the lives of an estimated 22,000 people; the giant accounting firm Arthur Andersen; and British Aerospace during its merger with GEC-Marconi.
But there are limits to what his company will do. For example, Bueno de Mesquita may already know, but he won’t say who’ll succeed George W. Bush in the White House. “We have a corporate policy that we will not, on a commercial basis, use the model in campaigns,” he says. “We don’t think it’s appropriate to manipulate the democratic process. We won’t take a client who wants to manipulate U.S. government policy, even if we agree with the manipulation. And we won’t take a foreign client whose objectives are contrary to the objectives of the United States government.”
There’s also the book he’s written with Condoleezza Rice and two other authors, The Strategy of Campaigning, which comes out in the fall. Given the Bush administration’s heavy ideological bent—which would seem to represent everything a rationalist like Bueno de Mesquita opposes—how does he justify putting his name on the same dust jacket as Rice’s? Bueno de Mesquita repositions himself in his chair. “The central question in this book is a question that Condi raised before she came to Washington,” he says. (So is her name there just to sell books? “We are making a concerted effort not to play up the fact that the Secretary of State is a co-author,” he later adds.)
Meanwhile, he has just launched and is the director of NYU’s Alexander Hamilton Center. “The mission for the center is the application of logic and evidence to solving fundamental policy problems. Not to a bipartisan solution, but to a nonpartisan solution.” In his continuing work for the CIA and the Defense Department, one of his most recent assignments has been North Korea and its nuclear program. His analysis starts from the premise that what Kim Jong Il cares most about is his political survival. As Bueno de Mesquita sees it, the principal reason for his nuclear program is to deter the United States from taking him out, by raising the costs of doing so. “The solution, then, lies in a mechanism that guarantees us that he not use these weapons and guarantees him that we not interfere with his political survival,” he says.
“They said my work was evil, offensive, that it should be suppressed. It was a very difficult time in my career.—Bruce Bueno de Mesquita”
Perhaps not coincidentally, the recent agreement that the United States reached with the government of Pyongyang closely resembles the one that Bueno de Mesquita’s model suggested: Kim agrees to dismantle his existing nuclear weapons but not his existing nuclear capability. “He puts it in mothballs with IAEA (International Atomic Energy Agency) inspectors on site 24 hours a day, 365 days a year. And in exchange, we provide him with $1.2 billion a year, which we label ‘foreign aid,’ of course.” The “foreign-aid” figure published in the newspapers was $400 million, which concerns Bueno de Mesquita. “I read that and I said, I hope that’s not the deal because it’s not enough money. He needs $1.2 billion, approximately, to sustain the loyalty of his cronies in the military and so forth. It’s unpleasant, this is a nasty man, but we’re stuck with it. The nice part of the deal is that it’s self-enforcing. Each side has a reason to credibly commit to their part of the deal.”
Recently, he’s applied his science to come up with some novel ideas on how to resolve the Israeli-Palestinian conflict. “In my view, it is a mistake to look for strategies that build mutual trust because it ain’t going to happen. Neither side has any reason to trust the other, for good reason,” he says. “Land for peace is an inherently flawed concept because it has a fundamental commitment problem. If I give you land on your promise of peace in the future, after you have the land, as the Israelis well know, it is very costly to take it back if you renege. You have an incentive to say, ‘You made a good step, it’s a gesture in the right direction, but I thought you were giving me more than this. I can’t give you peace just for this, it’s not enough.’ Conversely, if we have peace for land—you disarm, put down your weapons, and get rid of the threats to me and I will then give you the land—the reverse is true: I have no commitment to follow through. Once you’ve laid down your weapons, you have no threat.”
Bueno de Mesquita’s answer to this dilemma, which he discussed with the former Israeli prime minister and recently elected Labor leader Ehud Barak, is a formula that guarantees mutual incentives to cooperate. “In a peaceful world, what do the Palestinians anticipate will be their main source of economic viability? Tourism. This is what their own documents say. And, of course, the Israelis make a lot of money from tourism, and that revenue is very easy to track. As a starting point requiring no trust, no mutual cooperation, I would suggest that all tourist revenue be [divided by] a fixed formula based on the current population of the region, which is roughly 40 percent Palestinian, 60 percent Israeli. The money would go automatically to each side. Now, when there is violence, tourists don’t come. So the tourist revenue is automatically responsive to the level of violence on either side for both sides. You have an accounting firm that both sides agree to, you let the U.N. do it, whatever. It’s completely self-enforcing, it requires no cooperation except the initial agreement by the Israelis that they are going to turn this part of the revenue over, on a fixed formula based on population, to some international agency, and that’s that.”
His first foray into forecasting controversy took place in 1984, when he published an article in PS, the flagship journal of the American Political Science Association, predicting who would succeed Iran’s ruling Ayatollah Khomeini upon his death. He had developed a rudimentary forecasting model that was different from anything anyone had seen before in that it was not designed around one particular foreign-policy problem, but could be applied to any international conflict. “It was the first attempt at a general mathematical model of international conflict,” he says. His model predicted that upon Khomeini’s death, an ayatollah named Hojatolislam Khamenei and an obscure junior cleric named Akbar Hashemi Rafsanjani would emerge to lead the country together. At the time, Rafsanjani was so little known that his name had yet to appear in the New York Times.
Even more improbably, Khomeini had already designated his successor, and it was neither Ayatollah Khamenei nor Rafsanjani. Khomeini’s stature among Iran’s ruling clerics made it inconceivable that they would defy their leader’s choice. At the APSA meeting subsequent to the article’s publication, Bueno de Mesquita was roundly denounced as a quack by the Iran experts—a charlatan peddling voodoo mathematics. “They said I was an idiot, basically. They said my work was evil, offensive, that it should be suppressed,” he recalls. “It was a very difficult time in my career.” Five years later, when Khomeini died, lo and behold, Iran’s fractious ruling clerics chose Ayatollah Khamenei and Hashemi Rafsanjani to jointly lead the country. At the next APSA meeting, the man who had been Bueno de Mesquita’s most vocal detractor raised his hand and publicly apologized to him.
Bueno de Mesquita had arrived, and so, too, had rational-choice theory. Rational choicers began sprouting up in political-science departments around the country and, say their critics, strangling anyone and anything in their way. By 2000, according to one estimate, some 40 percent of all articles published in the prestigious American Political Science Review were rational-choice themed. Increasingly, graduate students in political science viewed a fluency in formal mathematic modeling as a prerequisite for career advancement. And the leaps in technology taking place only fueled rational choice’s advance: faster, more powerful computers allowed rational choicers to build bigger, ever more complex models that could be applied to ever more complex situations. And, naturally enough, an intellectual counteroffensive was launched.
It began in 1994 when two Yale political-science professors, Donald Green and Ian Shapiro, published their book, Pathologies of Rational Choice Theory, which disputed much of the scientific underpinnings that rational choice claimed for itself. In essence, the authors said that when rational choice was actually put to the practical test, much of it simply didn’t work. This was followed by a 1999 (lightning speed in academia) article by Stephen M. Walt in the journal International Security called “Rigor or Rigor Mortis?” Walt, a political-science professor at Harvard’s John F. Kennedy School of Government, conceded some value to formal modeling but ultimately likened rational choice to a “cult of irrelevance” that stifled creativity and had little practical value in actual policy formulation. Most vexing, Walt accused rational choicers of regarding nonrational choice theorists such as himself as “methodological Luddites whose opposition rests largely on ignorance.”
“We found that [national intelligence] analyses, even when they were right, were vague compared to [Bueno de Mesquita’s] forecasts. If you hit the target, that’s great. But if you hit the bull’s eye—that’s amazing.—Stanley Feder, former CIA analyst”
Since no one snaps a towel back harder than a scorned academic, Bueno de Mesquita and several of his rational-choice cohorts immediately mounted a blistering counter-counteroffensive, firing off a series of lengthy rebuttals to Walt’s piece that deconstructed his criticism, questioned his facts, and cited what was in their view Walt’s muddled logic as a prime example of why rational choice was so desperately needed in the field. “In the piece that Steve Walt wrote, in which he acknowledged that logical consistency was important, he also argued that it was overrated, that it stifled creativity. To me this is a bizarre idea,” says Bueno de Mesquita, “because really what that statement means to me is, if you relax logical consistency, you can say whatever you feel like and therefore you are back to a world in which the study of politics is the expression of personal opinion instead of being political science. It’s the art of politics or the articulation of beliefs, which is what dominates much of advising to government. It’s rhetoric.”
The brouhaha culminated at a raucous APSA meeting in 2001 at San Francisco’s Hilton Hotel with the open revolt of a group of major-league political scientists who, one by one, took to the podium to rail against rational choice and its encroaching methodological orthodoxy. Dubbed the “Perestroika Movement” by its anonymous founder (apparently, rational-choice folk are a powerful and vindictive lot), the dissident group vowed to take a stand against “the domination of mathematical approaches to the discipline.” There is a “hegemonic threat out there,” warned John J. Mearsheimer, a noted professor of international relations at the University of Chicago. “This is about the mathematicization of political science,” he said. “I’m in favor of filling the zoo with all kinds of animals. But I’m concerned about them running us out of the business or making us marginal.” Ultimately, the Perestroikans did win some concessions: a new editor of the APSR who vowed to make the flagship journal more hospitable to mathematics-free articles and a pledge from the APSA to open up its method of appointing officers. “The APSA had become dominated by those practicing so-called rigorous analyses,” says Walt. “Now the pendulum has swung back a bit.”
For Bueno de Mesquita, getting his methodology accepted by the policy-making establishment remains somewhat of an uphill slog. The most pointed criticism of rational choice has been that, unlike with more traditional political scientists, very little cross-pollination takes place between rational-choice academics and government policy-makers. Bueno de Mesquita says it’s just a matter of time before that changes. “Because people who are in a position to appoint people weren’t trained in this way, they don’t feel as comfortable as with people who were trained in what I would describe as a less rigorous form of study of politics. And, so, the folks who do more rigorous work typically don’t get invited in,” he says. Of course, the same was true of economics 40 years ago when nontechnical types like John Kenneth Galbraith dominated the field. Paul Samuelson and Milton Friedman changed all that, and Bueno de Mesquita sees himself playing the same role for politics.
Bueno de Mesquita remains unfazed, ever certain that rational choice will ultimately prevail. “When I moved to Rochester in 1973, if you wanted to be trained in this kind of political science, you could go to Rochester, period,” he says. “Ten years later, you could go to Rochester, Caltech, and Washington University in St. Louis. If you asked me today, you could go to the places I just mentioned, and you could go to NYU, you could go to Stanford—there’s a long list of places you could go. Except, of course, Harvard. But it will happen there, too. I’m on their syllabus.”
Back to the Future
A sample of Bruce Bueno de Mesquita’s wilder—and most accurate—predictions
Forecasted the second Intifada and the death of the Mideast peace process, two years before it happened.
Defied Russia specialists by predicting who would succeed Brezhnev. “The model identified Andropov, who nobody at the time even considered a possibility,” he says.
Predicted that Daniel Ortega and the Sandanistas would be voted out of office in Nicaragua, two years before it happened.
Four months before Tiananmen Square, said China’s hardliners would crack down harshly on dissidents.
Predicted France’s hair’s-breadth passage of the European Union’s Maastricht Treaty.
Predicted the exact implementation of the 1998 Good Friday Agreement between Britain and the IRA.
Predicted China’s reclaiming of Hong Kong and the exact manner the handover would take place, 12 years before it happened.
Labels:
2007,
game theory
Subscribe to:
Posts (Atom)