2/23/2011 Nancy Komlanc, Multimodal Information Access and Synthesis Center
Written by Nancy Komlanc, Multimodal Information Access and Synthesis Center
The Information Age has created an increasing abundance of data and has, thanks to the rise of the Internet, made huge amount of information instantly available to humans and computers alike. However, when different sources make contradictory claims, or provide partial pieces of information and each may have their own, sometimes hidden, motivations in doing so, we don’t know what to believe—in other words, what information is “trustworthy”?
The area of “information trustworthiness” is being researched by University of Illinois computer science professor Dan Roth, with support of a Faculty Research Award from Google. Roth’s work in Information Trust was detailed in a recent paper entitled “Comprehensive Trust Metrics for Information Networks”, which won a Best Paper award at the recent 27th Army Science Conference. This paper studies the premise of truthfulness, completeness, and bias of information reported.
Roth says, “given that it’s so easy to publish today, and even once reliable sources need to compete and publish very quickly, there are many voices clamoring for attention on the on-line media, blogs, etc. It is difficult to sift through it all, to figure out who's telling the truth and who's not, to know what to believe.“
Ideally we would like to identify those sources, documents and facts we would trust if we had the time and ability to consider all the information that is available and incorporate it with the background knowledge and beliefs we have. Then we can select which news article, report or, more importantly, which on-line medical advice to believe.”
Roth’s work in this direction will focus on the development of a mechanism for determining trust that can be reliably substituted for the user's own informed judgment, even in domains where being fully informed is human-infeasible (i.e. one cannot read every document on the web, but we can still use Google to search them). We must also determine exactly what we mean by trustworthiness of information and formalize the problem so that meaningful evaluation is possible.
As a simple example of the type of problem that we have when discerning the “trustworthiness of information” consider that one author claims Mumbai is the largest city in the world, and another claims it is Seoul; who do we believe? One or both authors could be intentionally lying, honestly mistaken, or just have different viewpoints of what constitutes a “city”. Is it the city ‘proper’ or does it include the metropolitan area? Truth, in this case, is not objective: there may be many valid definitions of “city”, but we should believe the claim that accords with our user's viewpoint. The real problem is, of course, much broader and far reaching – it has to do with forming political views based on truthful, or not so ``facts” presented about past and present event, about the truthfulness of the sources our kids learn from, about product reviews and about medical advice so commonly available today on the web, with very little ability of the user of verifying its trustworthiness.
Trustworthiness of information is a problem our society needs to address. As computer scientists, we have to begin developing a principled approach to this difficult problem. There are multiple challenges here, from the problem of retrieving relevant evidence, to that of dealing with conflicting claims, to developing a level of trust that agrees with one’s subjective beliefs and common sense knowledge.
Prof. Roth is professor of Computer Science and a University of Illinois Scholar. He is the director of a Department of Homeland Security funded center that is investigating solutions to problems and benefits of the huge amounts of data available today. The Multimodal Information Access & Synthesis Center brings together some of the world’s leading experts in data sciences to develop new technologies for dealing with how to locate, organize, access, analyze and synthesize unstructured data.