Monthly Archives: January 2012

Twitterstorm redux

David Weinberger on Too Big to Know

David Weinberger – another of those authors whose books I read as soon as they come out – recently published Too Big to Know: Rethinking Knowledge Now That the Facts Aren’t the Facts, Experts Are Everywhere, and the Smartest Person in the Room Is the Room, a very long title on the topic of how to separate the wheat from the chaff in a world where knowledge is seemingly inexhaustible. As anyone who has edited Wikipedia knows, knowledge is now dynamic, networked, and crowdsourced, both in academia and outside. Knowledge – good and bad – spreads blisteringly fast and can flatfoot many an authority.

I attended a seminar with David Weinberger today, at the Berkman Center – the turnout was quite good, about 150 people in my estimate. Here are my notes:

  • Physical instantiations of knowledge coming apart (encyclopedias, newspapers, libraries) because of one little innovation: The hyperlink.
  • Everyone is entitled to his own opinion, not his own facts (senator Patrick Moynihan.)
  • Knowledge seen as building on bricks on bricks, nailed down, and then a product of filtering
  • Too much to know, the world is too big to know – the strategy is to break off a brain-sized (“skulls don’t scale”) part of the world and allow an expert to know it really well. We can ask the expert, then get an answer and then we can stop asking. A system of stopping points – you don’t have to rerun the experiment, you can trust experts based on credentials.
  • Books are not linked – linear, winnowed (through good writing), permanent
  • Following footnotes used not to be done, now it is trivial.
  • Knowledge is picking up the properties of the new medium, just as it did pick up properties of the old medium.
  • Clay Shirky: No such thing as information overload, just filter failure.
  • Information overload (Toffler) came from sensory overload idea, 60s. People worried about information overload, would not keep you sane.
  • What constitutes information overload has changed – we tolerate much more stuff now.
  • Previously, stuff that was filtered out (by publishers and newspapers) was not available, but now it is, in blog posts. We filter forward on the Internet, we do not filter out.
  • New strategy: Include everything, the cost of getting rid of something is higher than getting rid of it. So you include almost everything. And you filter on the way out. (you’d never keep notes from library committee meetings in Wozilla, Alaska, because they would not be interesting, until Sarah Palin becomes vice-presidential candidate)
  • We are good at making order out of things. Knowing categories is to know the world – categorization is a serious pursuit for thousands of years. But physical entities need to have one and only one categorization – you cannot sort your CD collection alphabetically and by genre. But on the web, you can have thousands of playlists – a mess but a very rich mess.
  • Messiness is how you scale meaning.
  • For every fact on the Internet there is an equal and opposite fact. The Internet is a stew of disagreement. We don’t agree on anything and we never will. And that is fine.
  • We don’t even know if Moynihan really said that thing about facts and opinions
  • Shows picture of platypus, lots of arguing about its categorization – now we don’t care any more. We can have different namespaces that allow us to choose categorizations based on what we prefer.
  • We like to hang out with people like ourselves, and that is a problem – because we can create echo-chambers, which fragment and amplify disagreements.
  • Idea from the enlightenment – deep, down to the level of facts, anything else is not a real conversation. But this is a fallacy, for in order to do that you need to have large degree of similarity. Not going to solve that here, but Ethan Zuckerman and Yochai Benkler (both present) are working on it.
  • Long-form arguments are loosing their pre-eminence as highest form of human discourse. (Yes, I know I wrote a book, get the irony.) Not going away, but losing its preeminence. Darwin would, if he published now, be tweeting from the Beagle, had a conversation about finches’ beaks. And this web of knowledge would have more value than Darwin’s original work.
  • Michael Nielsen: Redesigning discovery., scientists posting papers and discussing them.
  • Destructuring of knowledge is happening at all levels, also at the level of the data themselves. Darwin studied barnacles for 7 years after Beagle trip. Data is not like that any more. Data commons happening in field after field: Genetics, astronomy, government, libraries. posting raw data because cleaned and curated data doesn’t scale.
  • Tremendous value in getting data out – and fast. Peer review doesn’t scale. Cannot scale science research, unless it is peer-to-peer review – open access journals that are peer reviews.
  • The third way data is changing is that it is linked – computers can make sense and link three different knowledge nuggets about platypuses (characterizing them as platypus, watermole, and ornithorhyncus anatinus) can be linked by linking to references.
  • This process is fractal and recurring.
  • Data are getting linked, fractal and destructured.
  • Networked knowledge may or may not be truer about the world, but it is truer about knowing.
  • What we have in common is not knowledge about which we agree, but a shared world about which we disagree.

Professor Ann M. Blair (author of Too Much to Know: Managing Scholarly Information before the Modern Age) with question: On the pyramid from data to information to knowledge to wisdom, Plato said that the purpose is wisdom. Aristotle wanted a disciplinary (certain) form of knowledge, middle ages brought us information concept (Bacon). Information explosion already in the 1800s, that’s when data enters the language, takes off during 1950s. Now it is just raw data.

Good things about the book: Nuanced, neither technology deterministic or not, but you can find data and authoritative knowledge behind every position. Like that it is not about substitution but net as an addition. The net cannot stand in for current institutions. The book is optimistic – we need to understand and use the net. I do hope that we are imparting mental maps and the wherewithal to make judgments – though I am of the wrong generation. Not “don’t use Wikipedia” but learning how to notice deficiencies.

Comment from librarian Harvard (missed the name). What are going to do about building a knowledge infrastructure? Knowledge is lumpy, intertwiningly linked and so on, but there are still tensions between truth and untruth. We do not have one foundation on which we can rest for very long. Today we are caught in a time warp between the long form book and the Net revolution, and we don’t have a handle on this new form of knowledge yet? Refer to Thinking, Fast and Slow: Is the net making us change from thinking slow to thinking fast – i.e., making decisions based on reactiveness rather than analysis?

Ethan Zuckerman: This is the book that shifts us from the early Weinberger to the middle Weinberger. I don’t think this a happy book – we just had a very smart man stand up and tell us that facts is not what we thought they were and consensus is probably not achievable. If this doesn’t unsettle you, what will? David’s central point is that this is not economics: What killed the Boston Globe was that some fundamental processes change the world, in how we know things and how we find it. He is making the argument that we are going to put something in a book and make it authoritative is challenged. We are now three nanoseconds after the Big Bang, and it will take us a very long time to navigate through this. The deep challenge he is putting forward is to understand the world is to understand and accept the complexity of the world. Those of us who figure out to navigate this space are given the possibility of succeeding in a new and very different way. Houseman: The advent of economic complexity: Think of it in terms of person-bytes: Houseman argues that you can figure out what economies can or cannot do, by understanding how many person-bytes you have in it. You can line up experts and that is good, but it works really well if the knowledge lies between the experts – understanding knowledge as a process. So what I am hoping for is some understanding of how we are going to navigate this web of knowledge. This is the most exciting question you can wrestle with. The world David is describing is much messier, but by helping us wrestle with it he has helped us.


How does the definition of truth change – have we gotten truth wrong?

We all have categories, fight about them, are you saying they are losing relevance? No, not at all, but the notion of a single, right categorization is losing its primacy. We discuss whether bloggers are journalists or not, many things hang on it, but we understand that there is not one right answer.

Importance of personal relationship rising parallel to web? Social dimension to knowledge? Will take the easy way out – I learn from mailing lists, and they eventually become social bonds. We want to turn information into communication.

Norwegian Data Inspectorate outlaws Google App use

In a letter (reported at to the Narvik Municipality (which has started to use Google Mail and other cloud-based applications, effectively putting much of its infrastructure in the Cloud) the Norwegian Data Inspectorate (, a government watchdog for privacy issues, effectively prohibits use of Google Apps, at least for communication of personal information. A key point in this decision seems to be that Google will not tell where in the world the data is stored, and, under the Patriot Act, the US government can access the data without a court order.

Companies and government organizations in Norway are required to follow the Norwegian privacy laws, which, amongst other things, requires that “personal information” (of which much can be communicated between a citizen and municipal tax, health and social service authorities) should be secured, and that personal information collected for one purpose may not be used for other purposes without the owner’s expressed permission.

This has interesting implications for cloud computing – many European countries have similar watchdogs as Norway, and many public and private organizations are interested in using Google’s services for their communication needs. My guess is that Google will need to offer some sort of reassurance that the data is outside of US jurisdiction, or effectively forgo this market to other competitors, such as Microsoft of some of the local consulting companies, which are busy building their own private clouds. Should be an interesting discussion at Google – the Data Inspectorate is a quite popular watchdog, Norway has some of the strongest privacy protection laws in the world (though, for some reason, it publishes people’s income and tax details), and Google’s motto of “Don’t be evil” might be put to the test here – national laws limiting global infrastructures.

Computer security is about finding front doors

This excellent little piece in Wired tells about a security researchers who could spy on corporate meetings by simply scanning for conference phones with “automatic accept” configured:

Using a program that Moore wrote, the researchers found the conference rooms by scanning the Internet for videoconference systems that were set up outside firewalls and configured to automatically answer calls.

In less than two hours, they found systems installed in 5,000 conference rooms around the country, including an attorney-inmate meeting room at a prison, an operating room at a university medical center, and a venture capital company where prospects were pitching their companies while laying out their financial details on a screen in the room.

As I always say – introduce too complex technology and too onerous password rules, and you end up with people using the same password for everything, ditching passwords altogether – or writing the password on a Post-It note and taping it to the back of their keyboards.

Manufacturing is changing, and so is productivity

Two excellent articles on increasing productivity, and why this will not result in many new jobs:

Davison describes the new kind of manufacturing, where everything is done by multi-step, highly complex machines, producing small series, requiring very high-skilled workers with rather sophisticated education. But they also need unskilled workers doing simple things, like moving parts between machines. The problem is, the pay scale for the second type is very low, and the difference in training to get to the skilled level so high, that no company will provide it:

For Maddie to achieve her dreams—to own her own home, to take her family on vacation to the coast, to have enough saved up so her children can go to college—she’d need to become one of the advanced Level 2s. A decade ago, a smart, hard-working Level 1 might have persuaded management to provide on-the-job training in Level-2 skills. But these days, the gap between a Level 1 and a 2 is so wide that it doesn’t make financial sense for Standard to spend years training someone who might not be able to pick up the skills or might take that training to a competing factory.

It feels cruel to point out all the Level-2 concepts Maddie doesn’t know, although Maddie is quite open about these shortcomings. She doesn’t know the computer-programming language that runs the machines she operates; in fact, she was surprised to learn they are run by a specialized computer language. She doesn’t know trigonometry or calculus, and she’s never studied the properties of cutting tools or metals. She doesn’t know how to maintain a tolerance of 0.25 microns, or what tolerance means in this context, or what a micron is.

The reason Maddie – hardworking and dedicated – has a job is simply one of distance: Shipping fragile parts to China for the unskilled operations is too risky and expensive. So Maddie has a job, but not career prospects. And the company’s management is facing very hard competition – their customers see them as a distributor – and is constantly scanning for things that can be outsourced or bought from another vendor.

Mandel describes the differences in productivity increases from improving productivity in domestic production – doing things smarter – and lowering cost by bargaining and optimizing the supply chain before it reaches the domestic organization. Both show up as productivity improvements, but have vastly different effects on domestic jobs:

But here’s the rub: both of these corporate strategies— domestic productivity improvements and global supply chain management—show up as productivity gains in U.S. economic records. When federal statisticians calculate the nation’s economic output, what they are actually measuring is domestic “value added”—the dollar value of all sales minus the dollar value of all imports. “Productivity” is then calculated by dividing the quantity of value added by the number of American workers. American workers, however, often have little to do with the gains in productivity attributed to them. For instance, if Company A saves $250,000 simply by switching from a Japanese sprocket supplier to a much cheaper Chinese sprocket supplier, that change shows up as an increase in American productivity—just as if the company had saved $250,000 by making its warehouse operation in Chicago more efficient.

This is known as import bias, and may be a problem, as it overestimates domestic productivity increases. Mandel goes on to show that this bias affect both left and right, and the difference in views is largely one about how to effectuate a change: Stimulus or tax relief.

Both authors advocate better data and better education as a way out, but quick fixes they aren’t. This is a real puzzler.

What you can learn from your LinkedIn network

LinkedIn Maps is a fascinating service that lets you map out your contact network. Here is my first-level network, with 848 nodes (click for larger image):


The colors are added automatically by LinkedIn, presumably by profile similarity and link to other networks. You have to add the labels yourself – they are reasonably precise, at least for the top five groups (listed according to size and, I presume, interconnectedness).

As can be seen, I am a gatekeeper between a network of consultants and researchers in the States (the orange group) and reasonably plugged into the IT industry, primarily Norwegian (the dark blue). The others are fairly obvious, with the exception of the last category, which happens to be an eclectic group that I interact with quite a lot, but which are hard to categorize, at least from their backgrounds.

Incidentally, the “shared” map, which takes away names, provides more information for analysis. Note the yellow nodes in my green network on the right: These are the people hired by BI to manage or teach in China. They are, not in nationality but in orientation, foreigners in their own organization.

My LinkedIn policy is to accept anyone I know (i.e. have had dealings with and would like in my network), which, naturally, includes a number of students (I will friend any student of my courses as long as I can remember them, though I must admit I am a bit sloppy there.)

What is missing? Two things stand out: I have many contacts in Norwegian media and in the international blogosphere, which isn’t here because, well, Norwegian media use Twitter or their own outlets, and bloggers use, well, their blogs. Hence, the commentariat is largely invisible in the LinkedIn world (except for Jill Walker Rettberg, who sicced me onto LinkedIn Maps). Also, a number of personal friends are not here, simply because LinkedIn is a professional network – and as such captures formal relationships, not your daily communications.

Now, what really would make me curious is what this map would look like for my Facebook, Twitter and Gmail accounts – and how they overlap. But the network in itself is interesting – and tells me that increasing the interaction between my USA network and the Norwegian IT industry wouldn’t hurt.

Twitter redux