Researchers Win Best Paper for Work on Trust Metrics for Information Networks

University of Illinois computer science professor Dan Roth and computer science PhD student Jeff Pasternack took home Best Paper honors at the recent 27th Army Science Conference for their paper, entitled “Comprehensive Trust Metrics for Information Networks."

This research in “information trustworthiness” studies the premise of truthfulness, completeness, and bias of information reported in various sources (ex. Newspaper, web, magazine articles, etc.). Truthfulness of information relates to its accuracy; completeness is thorough reporting and not purposefully leaving pertinent information out; bias is guiding toward a positive or negative opinion. Roth and Pasternack also find that the user himself is essential in calculating the trustworthiness of a source, in addition to the prior knowledge he/she may bring to the article read. In their research they use the following example on which to build their research in “information trustworthiness”.

Consider a short document authored by Sarah: “John is running against me. Last year, John spent $100,000 of taxpayer money on travel. John recently voted to confiscate, without judicial process, the private wealth of citizens." If all of these statements are true, Sarah and her document would thus be considered highly trustworthy.

However, if we know more about the background, we might find that Sarah is misleading us and is in fact, quite untrustworthy despite her factuality (i.e. we should not consider her to be a reliable source of information). If “John is running against Sarah" is a well-known, “easy" fact, Sarah's correct assertion thereof is unimportant to produce a seemingly-trustworthy document, regardless of its other content. Further, if $100,000 in travel expenses is par for John's office because it necessitates a great deal of travel, Sarah has conveniently neglected to mention this, instead inviting the reader to compare his costs to their own prior expectation of what “typical" travel expenses should be and conclude, incorrectly, that John has enjoyed gratuitously luxurious accommodations. Similarly, Sarah's “wealth confiscation" typically goes by the slightly more innocuous term “taxation", but her biased language suggests to the reader that John has approved of something unusually heinous.

In their research Roth and Pasternack have introduced three new metrics for measuring the trustworthiness of information sources: truthfulness, completeness, and bias, and shown that these are able to convey a more useful and more robust idea of how much (and in what way) an information source should be trusted than the current practice. By computing trustworthiness consistently across algorithms, they also enable direct cross system performance comparisons, and the evaluation of computed trustworthiness against human judgment.