Article 1 of 1
Net Interest

Finding What You Want
On the Web May Get Easier
New `Filters'
Could Help
Narrow Searches

By Rebecca Quick
The Wall Street Journal
(Copyright (c) 1997, Dow Jones & Company, Inc.)


With over 100 million World Wide Web pages on-line, the Internet is a virtual Library of Congress. Now it's about to get a few more librarians.

As any surfer knows, sifting through pages and pages of irrelevant information to find worthwhile nuggets can be a chore. Meanwhile, "filters" -- sites that perform searches for you -- offer only limited help. Search for the words "mountain climbing" on the AltaVista search engine and it calls up 13,187 references. Deja News, a search engine for the discussion areas called newsgroups, comes back with 3,134 listings when the word "diapers" is searched.

But, inundated users, take heart. A variety of new tools that will make surfing a lot easier are in the planning stage or actually available on-line. Whether culling newsgroup postings or tracking down an expert over the Web, this next generation of filters attempts to address the Internet's most nagging problem: It's almost impossible to get exactly what you want.

There are basically two types of Internet filters. First, "profile" systems. These include search engines, which look for pages containing words or phrases that you type in, and personalized "channel" services, which feed you updated news stories, sports scores and the like based on a profile you create.

These systems are very good at dredging up reams of information, but that also is their prime fault -- they tend to go overboard, bringing back much more than is necessary or desired.

That problem gave rise to the second type of filter: the "recommender," where somebody else does the sifting for you. Take the Mining Co. Web site. Here, workers search the Web for the best sites on myriad topics. Type in your key words, and you see the results of the scouring. Or, as the site says, "over 500 passionate and intelligent Guides . . . have done the searching and found what interests you."

But, because hundreds of new Web sites come on-line every day, it can be difficult for the guides to keep on top of what's new on the Internet -- particularly considering that being a guide isn't a full-time job.

Most of the new filters being designed are refined recommender systems. Group-Lens, a project initiated by the University of Minnesota in Minneapolis, tackles newsgroups, the electronic bulletin boards centered around particular interests such as quilting or kayaking. These postings can pile up rapidly, and finding a specific -- or useful -- reference can be difficult.

"It's commonly felt that 80% of the information on newsgroups is just useless," says Jon Herlocker, the lead graduate student on the project. "But our filter lets us pull out high-quality articles you'd be interested in."

With GroupLens, users look through newsgroups using a specially designed browser. After reading a message, the browser asks you to rate it on a scale of one to five. The system tracks your ratings and, after it sees about a dozen or so of them, creates a profile of you. It then compares your profile to other users', and suggests postings for you to read based on those other users' preferences.

For example, if you gave a high rating to a certain bunch of postings on kayaking, the system would track down other users who also liked those posts. Then the system would suggest that you read other postings that those users liked.

The researchers are still fine-tuning the technology so it can be used in broader applications. They chose newsgroups because they thought it would be an easy way to find correlations between people's likes and dislikes. But they have found that while it may be easy to match up people's tastes on a straightforward topic such as computer science, it is tougher to find matches for more subjective interests, such as humor.

Or vacations. At one point, researchers tested the technology on a travel Web site, comparing a user's responses to other surfers' to find destinations the user might like. But the software kept suggesting one of the researchers visit Cleveland.

"I've been to Cleveland, and I know there's no way I'd want to go there," says John Riedl, the researcher.

How did the system make such an off-base recommendation? Mr. Riedl's choice destinations didn't match with other users'. So the system assumed Mr. Riedl might like to go somewhere the others didn't. And they had shunned Cleveland.

The GroupLens project, which ran for two trial periods during which hundreds of people volunteered, is on hold while the team analyzes the data it collected. But it is likely to be commercialized eventually. Net Perceptions Inc. of Minneapolis has marketed some of the group's research to corporate Web sites such as and E-Online!'s Movie Finder site. Those sites use the technology to follow consumers' moves on the site and suggest books or movies most suited to their tastes.

Another team of researchers is looking at a way to handle another common Web chore: tracking someone down on-line. ReferralWeb works on the popular conception that there are just six degrees of separation between all people in the world; this program attempts to find the links between you and anyone you want to meet.

For example, if a user wants to find someone with expertise in a certain field, such as a First Amendment expert, he or she types in a query that brings back the names of other people who might know such an expert. Eventually, the theory goes, the user will discover that he knows one of the people who knows the expert and can then ask for an introduction.

"The idea is to mimic the way people find people," says Bart Selman , a Cornell University professor working on the project, which is being developed at the AT&T Research Laboratory in conjunction with the Massachusetts Institute of Technology and Cornell.

Here's how it works. The system employs a program that scans through Web pages, looking for names. It does this by comparing text from the pages to a database of phone books to identify which words on the sites are proper names.

"Sometimes it has problems with obscure names, but the phone book of New Jersey covers 99% of all possible names," Mr. Selman says.

Meanwhile, as the program collects names, it looks for connections among the names by seeing which sites link to each other. Granted, just because someone links to another person's home page doesn't mean the two know each other -- legions of fan pages link to Madonna's Web site, for instance. But the researchers are working on a solution.

"These celebrity pages only have links in one direction -- you point at a star, but they never point back at you," says Mr. Selman. So the system only assumes there's a relationship if sites link to each other -- not just one-way.

Many problems spring to mind. What if people don't mention their own name, or their full name, on their sites? Or what if two people whose sites link to each other mention the same celebrity's name?

For reasons such as these, ReferralWeb's creators say the system works best not as a universal people-finder, but as a way for professionals in specialized communities to find each other. For example, the test so far has focused on computer scientists, about 15,000 to 20,000 of whom are on the Web. The researchers found that most of these scientists are separated only by two or three links. Mr. Selman sees the program working in similar communities of professionals, such as other types of scientists or lawyers.

Some researchers are working at keeping humans entirely out of the process by making smarter technology for old-fashioned profile filters. This software not only would recognize key words but also natural language patterns, so they could look for word sequences to identify key words in the proper context. That technology can recognize when a word is used as a proper noun instead of a common noun, so that someone looking for articles about "Jaws" would get only references to the book or movie, instead of Web pages about dentists, tigers or steel traps.

Other researchers are studying technology that watches a user's behavior on line to determine his or her interests, instead of requiring the user to enter key words into a profile. But even those types of advanced profile filters won't necessarily find a high-quality article about the president, or one that agrees with a user's political bias.

"The problem with key-word searches is that they still may bring up a lousy or outdated article," says Pattie Maes, an associate professor at the MIT Media Laboratory. For that, she says, input from humans most likely will always be needed.

Or, at least, for a while.


                Filtering Out the Noise

                       NEWS FLASH
-- My Yahoo!

Allows user to set up a profile for items such as news topics of interest, a stock portfolio, sports scores, local weather and horoscopes.

-- My Excite Channel

Formerly known as Excite Live, it includes much the same fare as My Yahoo!

-- The Wall Street Journal Interactive Edition's Personal Journal

Enables reader to build a personal profile that tracks news interests, regular features and columns and stock portfolios.

-- CNN Custom News


Lets user set up a profile for certain news topics, available from CNN and hundreds of other news sources.

-- TotalNEWS Personal Edition

Gives user the ability to create a profile of various news sources to compare articles.


-- Firefly Passport

Compiles a personal profile of a user's demographic information and taste in things like music, movies and books.

Firefly-enabled Web sites read your profile as you enter and suggest goods that are likely to be of interest.

-- The Mining Co.

Picks out high-quality Web sites for users. Is hosted by guides who are "experts" in their areas.

                    IN THE PIPELINE

-- GroupLens

Weeds out irrelevant postings on electronic bulletin boards. Uses ratings from other people to determine which postings will interest you.

-- ReferralWeb

Analyzes personal relationships by tracking links between people's home pages. Attempts to make it easier to meet people by finding a person both people know to set up an introduction.

Copyright © 1997 Dow Jones & Company, Inc. All Rights Reserved.