"Shaping the Web: Why the Politics of Search Engines Matters" by Lucas D. Introna and Helen Nissenbaum, in The Information Society (July–Sept. 2000), Taylor & Francis, 325 Chestnut St., Ste. 800, Philadelphia, Pa. 19106.

Commercialization has already dampened hopes that the World Wide Web will serve as an egalitarian force. Now, Introna, a lecturer in information systems at the London School of Economics, and Nissenbaum, a lecturer at Princeton University’s Center for Human Values, worry that "biased" search engines are making some Web sites more "equal" than others.

The World Wide Web contains, by one estimate, some 800 million "pages." Search engines steer users to particular Web pages. A 1999 study of leading search engines found that none indexed more than 16 percent of the total, and that all combined covered only 42 percent. An unindexed Web page is almost impossible for users to find if they do not know its Uniform Resource Locator (URL), or "address."

Who decides whether to index a particular Web page? At "directory-based" search engines such as Yahoo!, editors do most of the work. The criteria for inclusion are vague, and apparently not applied with any consistency, Introna and Nissenbaum assert. At Yahoo!, by one estimate, a submitted Web page has roughly a 25 percent chance of being accepted. Inclusion becomes more likely, the authors say, as the number of links a page has to and from other sites increases. Also, "when editors feel they need more references within a category, they lower the entry barriers."

Other search engines, such as Alta Vista, Lycos, and Hotbot, dispense with the human editors and use software "spiders" to identify candidates. Precise details about how the spiders operate are closely guarded trade secrets, which stirs the suspicion of Introna and Nissenbaum. Pages with many links from other valued sites, especially sites that themselves have many "backlinks," are likely candidates.

Getting noticed by a search engine is only the first hurdle for creators of Web pages, the authors note. "Because most search engines display the 10 most relevant hits on the first page of the search results, Web designers jealously covet those . . . top slots." Search engine owners are reluctant to detail their ranking rules, but a site’s chances of doing well apparently improve if it has many keywords and they are high up in the document, and if many other sites are linked to it.

In the end, Introna and Nissenbaum argue, "popular, wealthy, and powerful sites" threaten to overwhelm the Web’s other voices. They urge full disclosure of search engines’ underlying rules, and the development of "more egalitarian and inclusive search mechanisms."