## Archive for the 'probability' Category

### When the Future Is Unlike the Past

First, pick an arbitrary point in time. Pick a year, any year A.D. up till your birthday or when you were twelve—this is your year. If you can, conjure up some idealized image of your year in your mind’s eye—nothing terribly analytic for now, just something holistic.

via kottke.org

Second, pick a year in time a few years back from your year, and pick a year a few years forward from your year. And, if you would, conjure up the same kind of images for these two years. Again, these need not be exacting—think of them as blurry, semi-liminal collection of facts and themes and truths and so on. You don’t need to memorize all the details of the goat, in other words—just some vague but substantive handle on it.

Third, get ready to perform some mildly unusual comparisons—not impossible, just a little strange, a little odd, but highly interesting, I promise. Try to imagine the difference between your year and the year a few years back. And while you’re at it, try to imagine the difference between your year and the year a few years forward too.

It’s a funny thing to try to imagine differences in the world over time. But that’s why we’re starting with this pretty easy exercise. It’s pretty easy to imagine the differences between things that are pretty similar—whether those things are different kinds of four-legged mammalian farm animals or different states of the world a few years apart.

Still, it may not be obvious to you what the differences are (1) between the world in your year and the world a few years before that or (2) between the world in your year and the world a few years after that. Given that you only care about the holistic view, you might be tempted to conclude that there are no substantive differences looking forward or backward. That’s fine—maybe there aren’t any.

But if there are differences within (1) or within (2), you should then be able to compare those differences themselves. Think of it this way. I grew up in Madison, Wis., which is about two hours from Chicago, Ill. Thus one difference between Madison and Chicago is location, and we can measure that difference in distance quantified by how long the drive is. Madison is about five hours from Minneapolis, Minn. Put this way, it’s natural to compare the differences between Madison-Chicago (two hours) and Madison-Minneapolis (five hours). That difference—that difference between differences—is three hours.

We can do roughly the same thing for your year looking back and your year looking forward. So you might say the difference between your year and the one a few years before is “small.” And you might say the difference between your year and the one a few years after is “small.” If so, the difference between the differences is zero. If the differences are “small” and “medium,” though, difference between the differences might be, e.g., “modest” or also “small.” And so on.

Whew. So the reason all this is important is that the differences between looking back a bit and looking forward bit are usually zero or small. Usually, change is constant. Our intuitions are largely built on this premise. In fact, we get terribly confused and distraught and sometimes incoherent when the difference looking forward is much greater than the difference looking back.

Consider the notion of precedent as legal philosophy. Scholars argue that precedents as reasons make the law more predictable—litigants can better predict the outcomes of law suits if they have some handle on the kind of reasoning judges will bring to bear. But why should law suits be predictable in the first place? That assumption reflects a deeper belief that similarities between previous cases and present or future cases are relevant at all. It reflects a belief that lawyers and judges can emphasize the relevant similar facts and ignore the relevant distinguishing facts. But how should lawyers and judges be able to make value judgments about which similar and distinguishing facts matter across different cases? It can only make sense to conform to precedent inasmuch as it makes sense—on average, I suppose, though even that is tricky given fat-tail risk—to take history as a competent guide to the future. Mere differences in time mean nothing.

But what if something so essential to our ability to reason did mean something? What if mere differences in time mattered legally? Or what if something seemingly arbitrary variable about the world mattered? What if, for example, we thought that events that obtained under a full moon were different from events that obtained under a new moon? We’d have at least two entirely different sets of legal precedents—one for full moons, one for new moons, and maybe one for other times. The legal world would be turned upside-down, inside-out. It would be nearly indistinguishable from what we actually have. The arguments that work under a full moon in our actual world almost certainly wouldn’t work in this crazy possible world. Not only would they almost certainly fail to persuade, but they would almost certainly seem deluded or insane—probably as insane as arguments highly sensitive to the lunar calendar would seem in the actual world.

The good news is that upside-down, inside-out changes are rare. The bad news is that their rarity doesn’t equip us well for when they inevitably crop up. We forget that there are any such changes, especially when they’re not attended by frighteningly salient facts, like nuclear weapons. It is easier, in other words, to wrap our minds around how thoroughgoing theories like mutually assured destruction change our reasoning, forcing us to question very basic assumptions, if we can at the same time point to devastating bombs and mushroom clouds.

But there is nothing so salient as for the news. Layoffs and newspaper closings amid a wider economic downturn just don’t cut the mustard. Sometimes the differences looking forward a decade dwarf the differences looking back a decade.

When that happens, when the near history no longer contains implicit clues about the near future, we are unmoored, and we look to look to the differences between differences as a partial guide—but just come to terms with our own imaginations, just to maintain some footing amid upheaval. In order to grasp some sense of how sweeping the next decade of changes in the news business will be, we’ve got keep inching back through time and technology till we arrive at the gut feeling that the differences are equal. When the differences looking forward and the differences looking backward are equal—idealized, vaguely but substantively—we can look to see what is common between the past world and the future one. And those common facts or truths are the only facts and truths we can carry forward as precedent, more or less unquestioned.

As with many businesses facing disruption from the internet, it is far from clear that there is anything common between what the news business will see a decade forward and what it saw even a century back. This is a muddled exercise in which we accomplish little more than calibrating our intuitions about what to discard and what to keep. But there is so much to discard that our intuitions are critical.

My gut tells me this, nuclear holocaust notwithstanding: It is no longer reasonable to carry facts true about the history of the news business into the future without detouring through first principles about journalism and why it’s important. Nothing true of journalism in a decade’s time will turn out also to have been true of journalism at any time in the past except those facts that will always be true.

All else is gone—that is what Shirky means when he writes, “There is no general model for newspapers to replace the one the internet just broke.” All else is gone, but first principles remain. And grasping first principles is why it’s imperative that “we shift our attention from ’save newspapers’ to ’save society.’” But don’t take “unthinkable” too literally; the future is thinkable. Shirky’s is a terribly useful figure of speech, but it is false. We cannot know or predict what the world will look like, but we can and should conduct experiments thoughtfully, not wildly. If we clear our minds of accumulated implicit assumptions about the newspaper business cloaked as timeless verities of journalism, we can arrive at a clean slate of first principles and begin to rebuild.

### Obstreperous Minnesota

Every once in a while—and maybe more often than I’d like to admit—I re-read Clay Shirky. Today, I re-read “Ontology Is Overrated.”

And today, I’m ready to disagree with it around the margins.

On fortune telling. Yes, Shirky’s correct that we will sometimes mis-predict the future, as when we infer that some text about Dresden is also about East Germany and will be forever. But, no, that doesn’t have to be a very strong reason for us not to have some lightweight ontology that then inferred something about a city and its country. We can just change the ontology when the Berlin Wall falls. It’s much easier than re-shelving books, after all; it’s just rewriting a little OWL.

On mind reading. Yes, Shirky’s correct that we will lose some signal—or increase entropy—when we mistake the degree to which users agree and mistakenly collapse categories. And, yes, it might be generally true about the world that we tend to “underestimate the loss from erasing difference of expression” and “overestimate loss from the lack of a thesaurus.” But it doesn’t have to be that way, and for two reasons.

First, why can’t we just get our estimations tuned? I’d think that the presumption would be that we could at least give a go and, otherwise, that the burden of demonstrating that we just cannot for some really deep reason falls on Shirky.

Second, we don’t actually need to collapse categories; we just need to build web services that recognize synonymy—and don’t shove them down our users’ throats. I take it to be a fact about the world that there are a non-trivial number of people in the world for whom ‘film’ and ‘movies’ and ‘cinema’ are just about perfect synonyms. At the risk of revealing some pretty embarrassing philistinism, I offer that I’m one of them, and I want my web service to let me know that I might care about this thing called ‘cinema’ when I show an interest in ‘film’ or ‘movies.’ I agree with Shirky that we can do this based solely on the fact that “tag overlap is in the system” while “the tag semantics are in the users” only. But why not also make put the semantics in the machine? Ultimately, both are amenable to probabilistic logic.

Google showed it is the very best at serving us information when we know we care about something fuzzy and obscure—like “obstreperous minnesota.” I don’t think Shirky would dispute this, but it’s important to bear in mind that we also want our web services to serve us really well when we don’t know we care about something (see especially Daniel Tunkelang on HCIR (@dtunkelang)). That something might be fuzzy or specific, obscure or popular, subject to disagreement or perfectly unambiguous.

People and organizations tend to be unambiguous. No one says this fine fellow Clay Shirky (@cshirky) is actually Jay Rosen (@jayrosen_nyu). That would be such a strange statement that many people wouldn’t even understand it in order to declare it false. No one says the National Basketball Association means the National Football League them. Or if someone were to say that J.P. Morgan is the same company as Morgan Stanley, we could correct him and explain how they’re similar but not identical.

Some facts about people and organization can be unambiguous some of the time, too. Someone could argue that President Obama’s profession is sports, but we could correct her and explain how it’s actually politics, which maybe sometimes works metaphorically like sports. That doesn’t mean that Obama doesn’t like basketball or that no one will ever talk about him in the context of basketball. There may be more than a few contexts in which many people think it makes little sense to think of him as a politician, like when he’s playing a game of pick-up ball. But I think we can infer pretty well ex ante that it makes lots of sense to think of Obama as a politician when he’s giving a big televised speech, signing legislation, or meeting with foreign leaders. After all, what’s the likelihood that Silvio Berlusconi or Hu Jintao would let himself get schooled on the court? Context isn’t always that dependent.