January 16, 2013

How Social Network Analysis Could Change the Future of Reporting

When it comes to reporting, we often understandably focus on two things: What we know and what we don't know.

What we know is easy. It's all the information we have gathered, from interviews and documents to observations and data. To use that information, we have to fully understand it, toss out what is not relevant or interesting, apply context and present it in a compelling and clear manner. We know how to do all this.

What we don't know is harder. It's all the questions we still have. Much of reporting is focused on getting this category smaller and the first category bigger. We largely know how to do this: Keep digging. More interviews, more sources, more documents.

But there is a third category: What we don't know we don't know.

Think of it as journalism's dark matter.

Of course, we often turn this up by pursuing what we don't know. One question leads to another, which leads to another. Two documents when put together point out something new. Two data sets when combined reveal something fresh, original and important.

Every veteran reporter has had those moments of revelation — usually right before they run into an editor's office with that holy–cow–this–is–big look on their face.

As we have started working on several projects in our newsroom that deal with relationships between people and groups — and trying to identify relationships that may be significant but ones we don't yet know exist — we've spent more time thinking about whether there is a way to surface more of those relationships.

Is there a tool that can make this kind of work easier for reporters? One that can, in effect, flesh out some of that dark matter in a systematic way?

I think everyone can imagine what such a thing would do: A reporter plugs a bunch of information into a computer, tosses in a couple databases and a little institutional knowledge, runs a query and a question is magically answered: Who is the governor's most influential campaign contributor?

To be sure, you can — and should — try to answer that question through old–fashioned shoe–leather reporting: You talk to the governor, to those you know are close to the governor, to those you think may be close to the governor. You scan and tally campaign contribution, review old clips on the governor and others.

And in the end, you can produce a well–informed piece that will undoubtedly be a service to readers.

But what about that name you don't recognize? Maybe it is a person who you don't know is on the board of a corporation that employs the wife of the governor's top fund–raiser. And that corporation may have just won a major state contract when it wasn't the low bidder.

Is there effective software that can make some of those connections for you? One that could make the connections on deadline, as soon as you knew the contract had been awarded. Or even before, as soon as the bids were filed.

Now, I don't know how to build such a thing. But as a longtime reporter and editor, I think I can identify some of the hurdles and issues faced in doing so.

  1. What information do you gather? Every decision of what to include and not include becomes its own limit to the usefulness of the tool. Missing a key connection can lead to a skewed result, or no useful result. How do you identify what to include or not include?

  2. How do you keep it up to date? If you are not updating the underlying information regularly, say names of gubernatorial appointees to boards and commissions, the usefulness of the tool is progressively degraded.

  3. How do you account for institutional knowledge? It's relatively easy to put a group of existing databases together. But there is no database you can request of the governor's oldest friends. Often this information comes in bits and pieces — it resides in a reporter's head, an old notebook or even a past clip in the library.

  4. How do you tap that newsroom knowledge in a comprehensive way? Maybe the religion reporter did an item about the governor's pastor. Maybe a business reporter knows where the head of that corporation went to college. And someone else knows that the governor's top fund−raiser is on an advisory board there, because he and his wife met while in school. Any tool needs a quick and simple mechanism to pull in as much of this information as possible. Too cumbersome, and reporters won't add it. The information will just stay in notebooks and memories.

Of course, there are many other questions and hurdles. These do not even address what a public−facing site might look like or how it would function, only what may be useful for the reporter on deadline who wants to plug in the name of the corporation and figure out why it got that major contract.

And even if a tool like this existed — something that could quickly and accurately identify various connections — it is only a starting point.

But it does move information out of the category of what we don't know we don't know and into the category of what we don't know but can find out. And from there, it's a whole lot easier to get it into what we do know.