In Defense of Content Silos...

For the longest time, many people believed that one of the biggest information management problems is simply getting access to data. Previously, data was hidden away in a "silo," making it difficult to obtain, because you had to deal with an "information broker." You know the stereotype: somebody who rations their hoard of information for job security purposes, and refuses to share unless forced to... Not always, but many times this "broker" frequently acted more like a "bottleneck."

20 years ago, this led people to believe that if we could only bypass these brokers, and access the information directly, then all our problems would be solved! If only we could get the raw, unfiltered data, we would be much more efficient!

It turns out, not so much...

If you want a successful ECM/Knowledge Management/ Enterprise 2.0 initiative, one of the biggest mistakes you could make is to focus too much on sharing information.

Don't get me wrong: there are many situations where sharing information is vital... however, those situations are pretty obvious. You won't need to look hard to find them, and solving information access problems is fairly straightforward. Sometimes its technical, in which case basic content management tools can help out. Some times the problem is political, in which case the optimal information management strategy requires you to first understand the cultural reasons why your employees refuse to share information... after which you can either chose a software solution, or just force everybody to go to anger management training.

But... assuming that you break down the cultural bottlenecks to innovation, now you have another problem: "infoglut." In other words, information overload. Emails you don't read, presentations that don't make sense, reports that don't flow, the proliferation of websites, blogs, wikis, and social software across the enterprise... people talking a lot, but communicating very little.

In many ways, the best solution to infoglut is to bring back the information broker. Now that information is completely free, you need some kind of filter to let you know what information is relevant. This does not mean that you should bring back the information bottlenecks... instead, you need tools that makes the information broker more effective. This may include:

  • Smarter search engines: the current ones try to determine relevance just based on keywords, which doesn't work so well. Google's search engine does a great job on the heavily hyperlinked web, but its a terrible tool for the whole Enterprise. Smarter search engines need to use identity management and analytics to know what content is currently popular with people similar to the current user. They also need a human to maintain a controlled thesaurus, in order to get a vague idea about what content is similar.
  • Polite relevance filters for email: how many of you would love to have a customer support queue for your email inbox? "Thank you for your email! I currently have 1000 unread items in my inbox, all flagged 'important.' The average wait time for a response is 97 hours." Of course, you might also need one for your phone...
  • Recommendation sites: these allow your greater audience to "vote" on what content is relevant. Examples include Digg and Reddit, which are both good, but only for highly broad topics. I know what's hot in "Technology," but not what's hot in "Embedded Linux."
  • Better tools for human information brokers: I firmly believe the broker is the solution, not the problem. Instead of replacing them with technology, work with them to design systems that are more effective at brokering information. If they see their influence increase the more they share, they will do everything they can to get the right information to the right people at the right time. There isn't much commercially software along these lines at the moment... but that doesn't stop sites like AllTop from doing it anyway.

In short, don't blindly bash content silos and information brokers. Sure, they kept data hidden from you... but it was data you probably didn't want to see in the first place. Replacing a human broker with a Digg-clone may work for a while... but that's easy. And since its easy, soon everybody will be doing it, and it will cease to be a competitive advantage. As I've said many times, no technology can ever replace a human being who actually gives a damn.

Technology should empower your brokers, not replace them. Otherwise, you'll soon be swimming in useless information... just like everybody else.

UPDATE: There's a bit of confusion about what I mean by "broker." I do not mean to imply that information should go back to being "locked away." That is both illogical and impossible. Rather, I'm stating that the role of the "broker" has changed into being more of a "filter," and is arguably more important than ever. As Clay Shirky says, "It’s Not Information Overload. It’s Filter Failure." Sometimes a broker is one person, sometimes its several people, sometimes its a software algorithm. None is superior to the other: they all have merits. Therefore, its important for the filter to be a combination: better search algorithms based on relevance, Digg clones for the enterprise, and better tools to help individuals who choose to become brokers. This means multiple silos will be built upon the same raw information, based solely on which filter you pick. Like it or not, its already happening... and people will use whichever one makes them more productive.

Comments

Information Brokers and Hoarders are different

Filtering out noise should be a major concern of any information management or social software solution. There's simply too much information available for any one human being to scan, much less analyze. So the role of information brokers, trusted advisers, or content aggregators is becoming more important, not less.

But while brokers, advisers, and agreggators are providing a service, most people think of an information hoarder as being an impediment to the process. They keep information bottled up in content silos for a variety of reasons: they might fear a loss of control over the information, or a loss of influence within the organization, or that others might misuse or abuse the data.

The problem with these gatekeepers is that there's no way to evaluate whether they're doing a good job or not. You can't know what information they're withholding, so you can't know if they're doing something valuable -- reducing noise -- or preventing access to information you need -- reducing signal.

true... but...

There are several studies that show that even highly popular, highly "democratic" sites like Digg are pretty much ruled by an oligarchy of moderators.

http://www.seomoz.org/blog/entry-4-are-the-digg-100-the-new-rupert-murdoch

There isn't really that much difference between the gatekeeper who hoards information... and the gatekeeper that "tags" information as relevant. In the end, you mostly just get the information they choose to let you have.

Information hoarding is not a good long-term strategy, because data wants to be free. Also, brokers have higher influence than hoarders, so the hoarders will eventually become less numerous. That's just the rules of evolution ;-)

However, those hoarders have a lot of useful organizational knowledge... if they could be "won over" instead of bypassed, then we'd have the optimal solution.

Die Information Broker Die

response post here: http://blogs.oracle.com/fusionecm/2008/12/content_silos_and_information.html

don't confuse "broker" with "bottleneck"

I believe you're confusing "broker" with "bottleneck" or "hoarder," similar to the first poster.

I'm perfectly capable of shopping around for my own insurance, but I prefer checking with a broker as well. Why? They are better connected, and have better tools to stay up-to-date. They know things I do not, even if all the information is freely available. It also frees up my time so I can do more important things.

I'm perfectly capable of shopping around for my own mortgage, but I prefer checking with a broker as well. Why? They are better connected, and have better tools to stay up-to-date. They know things I do not, even if all the information is freely available. It also frees up my time so I can do more important things.

I'm perfectly capable of... well, you get the picture.

Companies are already using brokers, whether they chose it or not. The broker is the well-connected person who navigates multiple information systems, talks to many people about what they are working on, and is the "go-to" guy for lots of random questions. They might not know the answer, but they probably know who knows the answer. Some do this out of natural curiosity, others do it to expand their influence, still others do it simply because they need broad access to recent data to do their jobs.

Like it or not, Billy... you -- like most folks in product management -- are an information broker. You blog, you wiki, you're all over social networks, you check-in reports and presentations, and you answer questions on email lists... and yet people still email you directly because they can't find stuff on their own. People are either lazy, or its just an extension of Calvin Mooers law:

An information retrieval system will tend not to be used whenever it is more painful and troublesome for a customer to have information than for him not to have it.

'tis your fate... I'm sure you'd love to be replaced by a computer so you can retire, but that's not in the cards ;-)

re: die information broker die

it's not that the old dinosaurs (aka brokers) don't have good information, just that they're not in your context and therefore are at least 2 steps removed from getting you the information you *really* want/need. To date they seem to have done a fairly good job (or at least we're accustomed to the job they do). But that did not stop the proliferation of sites, blogs, and available information. People want(ed) more. There's another kind of broker - the travel agent - that is all but dead due to the de-siloing of information. I predict the others will go that way with insurance headed there (next?). Movie ticket sales agents are another kind of broker being downstaffed due to movie times and tix being searched and bought online and picked up at will-call or automated kiosks.

The semantic web technology has the promise (though not yet the reality) to be the eventual personal proxy - your cyberspace personal assistant your avatar.

Yeah I'm a broker right now but I don't just hand people my information, I tell them how to get it AND how to participate with it. After all, isn't that what we're doing right now?

the broker NEVER goes away

I disagree... the travel agent "broker" has not disappeared... simply changed.

The most mundane of travel agent tasks has been taken over by Expedia, Travelocity, and Orbitz... who do a good job getting you from point A to point B, and getting a hotel. But now the bar is raised. What hotels are any good? What shows should you see? What tours should you take? Can you rely on top listings in Expedia? Can you trust comments? Should you use TripAdvisor for reviews, and somebody else for purchasing tickets?

Nearly every hotel I've been to in the last 5 years had an on-site travel agent and/or concierge... so the industry is far from dead... its just more niche. Many companies still use them for business travelers, and many tourists still use them to find the "out of the way" adventures that aren't yet on the web. If Orbitz goes the way of Amazon -- allowing small partners to sell niche services through their interface -- I can see them coming back strong in that there "long tail" people keep talking about.

This is no different than innovation in any industry. Initially new technology is disruptive, and replaces people... such as how the sewing machine replaced tailors, and the printing press replaced scribes. However, the next leap in innovation occurs when humans learn to work with technology, to go after newer markets, and be more productive than technology alone. Tailors initially got fired... but the drop in the price of garments created a huge demand for clothes. Tailors who knew how to use sewing machines had more business than they could shake a stick at. Printing presses initially made scribes less useful... but cheap books created a huge demand for the printed word, and the printing industry exploded.

Probably the best recent example of this is the industrial base in China. Instead of single-purpose machines to spit out one piece of cheap plastic crap... they have a combination of multi-purpose machines and human labor. That way, they can spit out a great variety of cheap plastic crap, to keep up with demand. Its a lot cheaper to re-train a human, than re-program a computer. Ironically, in some well-managed and fast-changing industries, its not cost effective to train computers to keep up with humans...

When it comes to innovations in information technology, this rule still holds. I highly doubt the answer is better technology alone... nor will it be the "wisdom of crowds" alone. The truly influential will be those who use new technology, monitor crowd behavior for extra data, then add their own judgment.

RE: Update broker = filter

I still disagree but the blog fodder is good. Response at http://blogs.oracle.com/fusionecm

Kurt R. Karst of Hyman,

Kurt R. Karst of Hyman, Phelps & McNamara, P.C. (and co-chief blogger of FDALawBlog.net) will be speaking at two upcoming conferences on various Hatch-Waxman Act product lifecycle management issues.  The first conference is the Center for Business Intelligence's Premier Bio/Pharmaceutical Summit on Legal & Regulatory Product Lifecycle Strategies, January 20-21, 2009, at the Hilton Baltimore in Baltimore, Maryland.  Information on the conference is available here.  Conference sessions include:  For more information or to register for the conference, please contact the Center for Business Intelligence toll-free by phone at 1-800-817-8601 or via e-mail at cbireg@cbinet.com. The second conference is a Thompson Publishing Group audio conference, titled "New Drug Exclusivity Provisions: How They'll Impact Your Product Lifecycle Management," and will take place on January 13, 2009 from 1:00 PM – 2:30 PM.  Information on the audio conference is available here.  This conference will explore the "old" antibiotic and enantiomer exclusivity provisions recently added to the FDC Act, as well as recent issues concerning 180-day generic drug exclusivity and patent term extensions.

relevance?

good to know... but I doubt my audience cares...

Post new comment

CAPTCHA
This form prevents comments spam... all letters in lowercase
Copy the characters (respecting upper/lower case) from the image.