Article 1 of 1
Finding What You Want
On the Web May Get Easier
By Rebecca Quick
The Wall Street Journal
(Copyright (c) 1997, Dow Jones & Company, Inc.)
With over 100 million World Wide Web pages on-line, the Internet is a
virtual Library of Congress. Now it's about to get a few more
As any surfer knows, sifting through pages and pages of irrelevant
information to find worthwhile nuggets can be a chore. Meanwhile,
"filters" -- sites that perform searches for you -- offer only limited
help. Search for the words "mountain climbing" on the AltaVista search
engine and it calls up 13,187 references. Deja News, a search engine for
the discussion areas called newsgroups, comes back with 3,134 listings
when the word "diapers" is searched.
But, inundated users, take heart. A variety of new tools that will
make surfing a lot easier are in the planning stage or actually
available on-line. Whether culling newsgroup postings or tracking down
an expert over the Web, this next generation of filters attempts to
address the Internet's most nagging problem: It's almost impossible to
get exactly what you want.
There are basically two types of Internet filters. First, "profile"
systems. These include search engines, which look for pages containing
words or phrases that you type in, and personalized "channel" services,
which feed you updated news stories, sports scores and the like based on
a profile you create.
These systems are very good at dredging up reams of information, but
that also is their prime fault -- they tend to go overboard, bringing
back much more than is necessary or desired.
That problem gave rise to the second type of filter: the
"recommender," where somebody else does the sifting for you. Take the
Mining Co. Web site. Here, workers search the Web for the best sites on
myriad topics. Type in your key words, and you see the results of the
scouring. Or, as the site says, "over 500 passionate and intelligent
Guides . . . have done the searching and found what interests you."
But, because hundreds of new Web sites come on-line every day, it can
be difficult for the guides to keep on top of what's new on the Internet
-- particularly considering that being a guide isn't a full-time job.
Most of the new filters being designed are refined recommender
systems. Group-Lens, a project initiated by the University of Minnesota
in Minneapolis, tackles newsgroups, the electronic bulletin boards
centered around particular interests such as quilting or kayaking. These
postings can pile up rapidly, and finding a specific -- or useful --
reference can be difficult.
"It's commonly felt that 80% of the information on newsgroups is just
useless," says Jon Herlocker, the lead graduate student on the project.
"But our filter lets us pull out high-quality articles you'd be
With GroupLens, users look through newsgroups using a specially
designed browser. After reading a message, the browser asks you to rate
it on a scale of one to five. The system tracks your ratings and, after
it sees about a dozen or so of them, creates a profile of you. It then
compares your profile to other users', and suggests postings for you to
read based on those other users' preferences.
For example, if you gave a high rating to a certain bunch of postings
on kayaking, the system would track down other users who also liked
those posts. Then the system would suggest that you read other postings
that those users liked.
The researchers are still fine-tuning the technology so it can be
used in broader applications. They chose newsgroups because they thought
it would be an easy way to find correlations between people's likes and
dislikes. But they have found that while it may be easy to match up
people's tastes on a straightforward topic such as computer science, it
is tougher to find matches for more subjective interests, such as humor.
Or vacations. At one point, researchers tested the technology on a
travel Web site, comparing a user's responses to other surfers' to find
destinations the user might like. But the software kept suggesting one
of the researchers visit Cleveland.
"I've been to Cleveland, and I know there's no way I'd want to go
there," says John Riedl, the researcher.
How did the system make such an off-base recommendation? Mr. Riedl's
choice destinations didn't match with other users'. So the system
assumed Mr. Riedl might like to go somewhere the others didn't. And they
had shunned Cleveland.
The GroupLens project, which ran for two trial periods during which
hundreds of people volunteered, is on hold while the team analyzes the
data it collected. But it is likely to be commercialized eventually. Net
Perceptions Inc. of Minneapolis has marketed some of the group's
research to corporate Web sites such as Amazon.com and E-Online!'s Movie
Finder site. Those sites use the technology to follow consumers' moves
on the site and suggest books or movies most suited to their tastes.
Another team of researchers is looking at a way to handle another
common Web chore: tracking someone down on-line. ReferralWeb works on
the popular conception that there are just six degrees of separation
between all people in the world; this program attempts to find the links
between you and anyone you want to meet.
For example, if a user wants to find someone with expertise in a
certain field, such as a First Amendment expert, he or she types in a
query that brings back the names of other people who might know such an
expert. Eventually, the theory goes, the user will discover that he
knows one of the people who knows the expert and can then ask for an
"The idea is to mimic the way people find people," says Bart Selman ,
a Cornell University professor working on the project, which is being
developed at the AT&T Research Laboratory in conjunction with the
Massachusetts Institute of Technology and Cornell.
Here's how it works. The system employs a program that scans through
Web pages, looking for names. It does this by comparing text from the
pages to a database of phone books to identify which words on the sites
are proper names.
"Sometimes it has problems with obscure names, but the phone book of
New Jersey covers 99% of all possible names," Mr. Selman says.
Meanwhile, as the program collects names, it looks for connections
among the names by seeing which sites link to each other. Granted, just
because someone links to another person's home page doesn't mean the two
know each other -- legions of fan pages link to Madonna's Web site, for
instance. But the researchers are working on a solution.
"These celebrity pages only have links in one direction -- you point
at a star, but they never point back at you," says Mr. Selman. So the
system only assumes there's a relationship if sites link to each other
-- not just one-way.
Many problems spring to mind. What if people don't mention their own
name, or their full name, on their sites? Or what if two people whose
sites link to each other mention the same celebrity's name?
For reasons such as these, ReferralWeb's creators say the system
works best not as a universal people-finder, but as a way for
professionals in specialized communities to find each other. For
example, the test so far has focused on computer scientists, about
15,000 to 20,000 of whom are on the Web. The researchers found that most
of these scientists are separated only by two or three links. Mr. Selman
sees the program working in similar communities of professionals, such
as other types of scientists or lawyers.
Some researchers are working at keeping humans entirely out of the
process by making smarter technology for old-fashioned profile filters.
This software not only would recognize key words but also natural
language patterns, so they could look for word sequences to identify key
words in the proper context. That technology can recognize when a word
is used as a proper noun instead of a common noun, so that someone
looking for articles about "Jaws" would get only references to the book
or movie, instead of Web pages about dentists, tigers or steel traps.
Other researchers are studying technology that watches a user's
behavior on line to determine his or her interests, instead of requiring
the user to enter key words into a profile. But even those types of
advanced profile filters won't necessarily find a high-quality article
about the president, or one that agrees with a user's political bias.
"The problem with key-word searches is that they still may bring up a
lousy or outdated article," says Pattie Maes, an associate professor at
the MIT Media Laboratory. For that, she says, input from humans most
likely will always be needed.
Or, at least, for a while.
Filtering Out the Noise
-- My Yahoo!
Allows user to set up a profile for items such as news topics of
interest, a stock portfolio, sports scores, local weather and
-- My Excite Channel
Formerly known as Excite Live, it includes much the same fare as My
-- The Wall Street Journal Interactive Edition's Personal Journal
Enables reader to build a personal profile that tracks news
interests, regular features and columns and stock portfolios.
-- CNN Custom News
Lets user set up a profile for certain news topics, available from
CNN and hundreds of other news sources.
-- TotalNEWS Personal Edition
Gives user the ability to create a profile of various news sources to
GAUGING YOUR INTERESTS
-- Firefly Passport
Compiles a personal profile of a user's demographic information and
taste in things like music, movies and books.
Firefly-enabled Web sites read your profile as you enter and suggest
goods that are likely to be of interest.
-- The Mining Co.
Picks out high-quality Web sites for users. Is hosted by guides who
are "experts" in their areas.
IN THE PIPELINE
Weeds out irrelevant postings on electronic bulletin boards. Uses
ratings from other people to determine which postings will interest you.
Analyzes personal relationships by tracking links between people's
home pages. Attempts to make it easier to meet people by finding a
person both people know to set up an introduction.