Highlight
The Value of Internet Search: Time Saved vs. Quality
Achievement/Results
With the advent of the Web and search engines, online searching has become a common method for obtaining information. While Web searching is easy and convenient compared to going to a library, one might be concerned about the quality of the information obtained by those searches. Of the millions of search results, some may be inaccurate, or come from untrustworthy sources. While a searcher may save time by searching online, this time-saving may come at the expense of information quality.
IGERT PI Yan Chen and two doctoral students at the University of Michigan School of Information, Grace Jeon and Yong-Mi Kim, examine the time saved vs. quality tradeoff by comparing the processes and outcomes of Web searches in comparison with more traditional information searches using an academic library. Our search queries and questions come from two different sources. First, we obtain a random sample of queries from a major search engine, which reflects the spectrum of information users seek online. As these queries might be biased towards searches more suitable for the Web, we also use a set of real reference questions users sent to librarians in the Internet Public Library (IPL). The mission of the IPL was to bridge the digital divide, i.e., to serve populations not proficient in information technology and lacking good local libraries. Thus, we expect that the IPL questions came from users who could not find answers from online searches. Compared to queries from the search engine, we expect the IPL questions to be less biased in favor of online searches. In sum, a combination of the two sources enables us to obtain a more balanced picture comparing Web and non-Web search outcomes. As reported in our paper, for questions generated from search-engine queries, a Web search takes on average 7 minutes, whereas the corresponding offline search takes 22 minutes. For questions from the IPL, however, a Web search takes on average 9 minutes, whereas the offline search takes 19 minutes. Thus, search-engine questions lead to more time saved using the Web (15 minutes), whereas IPL questions generate a smaller margin in terms of time saved (10 minutes).
Next, we use trained raters to evaluate the quality of sources from the Web and non-Web searches. Perhaps the most surprising finding is that the overall Web source quality is not significantly different from that of non-Web sources when questions come from search-engine queries. Incidentally, the most frequently accessed URL by our searchers is the English Wikipedia. For IPL questions, however, non-Web sources are judged to be of significantly higher quality. Therefore, while consumers save time without sacrificing quality by using Web search for most of their information needs, sometimes they obtain higher quality information from offline sources. Is the quality differential worth the extra time for offline searching? One hopes that the gap will eventually disappear once library contents are fully digitized and searchable online. This research combines techniques from experimental economics and interactive information retrieval to address an important questions on the impact of information technology on productivity. We have benefited from the interdisciplinary research community at the University of Michigan enabled by our IGERT grant.
Address Goals
Current empirical literature in the field of industrial organization relies on census, observational and survey data to assess the impact of IT on productivity. Such studies have examined the impact of IT on either worker productivity or firm-level productivity in specific domain areas or industries. Beyond productivity, Goolsbee and Klenow (2006) estimate consumer welfare gains from the Internet by using not only how much money consumers spent on access but also the amount of leisure time they spent online, using a survey data set. In comparison, our study investigates the impact of search technology on productivity in everyday life, using individual level data obtained from a real-effort experiment. Therefore, our estimates are more accurate compared to previous studies. It pushes the frontier of productivity estimations. While designing and conducting the study, two doctoral students and several REU students were actively involved, thus, learning the basics of conducting real-effort experiments.
