SUBJECT SEARCH

DIRECTORY AND

INVISIBLE WEB

Subject Search Directories

v.

Search Engines v. Subject Search Directories

Search Engines

• Websites selected by “spiders” through automated searching

• Goal is to have as big an index as possible

• Not much information about site except URL; some have small descriptions

• Order of websites unknown to user; metadata can be tampered with for higher rating, pay for higher level

• Sponsored links and advertising on sites

Subject Search Directories

• Websites selected by humans

• Small number of sites selected,

indicating “the best”

• Many have annotations, written

by humans

• Generally well-organized in

hierarchies of subjects

• Little or no advertising on most

sites

Subject Search Directory Definition

Directories are a collection of web sites that are organized

by broad subject categories. When you search a directory,

you are only searching the web sites identified and

included in that directory. Directories may be broad

(covering all subject areas), or subject specific (focusing on

a particular subject).

When to Use?

Search Engines

• When you want to see

everything

• When you are looking for

something specific

Subject Search Directories

• When you want

something that has been

reviewed

• When you are browsing

Examples

Search Engines

• Patron A:

I would like to know the

address of the Proctor &

Gamble company.

Subject Search Directories

• Patron B:

I would like to know good

resources on the web that I

could use with my high

school business class that

will allow them to search for

businesses. I want to be

able to give them

something I can trust.

Other Examples

• If you know very little about the subject that you are being

asked about, start with a subject search directory such as

ipl2.

• If you are needing annotated sites.

• If you are trying to find a database on some kind of

information. Before everyone knew about IMDB (Internet

Movie DataBase) it would have shown up here (which it

still does, and with even further breakdowns.)

• If you are working with a student whose teacher doesn’t

want them to use the “internet.” These are more reliable

sites than what a student could find on their own.

How to Use

• Entering a search term.

This is sometimes less successful because you have to hit the right

term. (I tried to come up with a term I couldn’t hit, but I think too

much like a librarian now. Your customers could definitely stumble

with the wrong words.)

• Drilling down.

Drilling down involves clicking on the subject that you want from the

main page, and selecting where you next want to go. For instance, I

selected Resources by Subject on the front page of ipl2. Then I

selected Business & Economics, then Business & Economic News,

then on Advancing Women, a site that is for advancing women in the

workplace as well as other diversity issues. This is a site that I could

get to, but someone has indicated that it is unusual enough to make

it on this list.

Keyword v. Drilling;

a Final WordSometimes you will get different sources from using keyword versus the drilling down methods. Best suggestion is to do both until you see how your search goes and you become more familiar with the tool.

In the past, half the students walked away feeling this was almost the best thing they learned during the semester; others felt like it was a total waste of time that they would never use.

My final word: It depends on the search and the searcher. It takes patience to drill down and you have to set your frustration meter to 0. That’s because you don’t have the directory of terms to use that the person who created the search directory.

Read the About section of each subject search engine and see if it gives you criteria for inclusion.

Perfection? No…

Some of the sites included require you create a user name

and password.

However, the individuals who created the subject search

directory work through criteria and decided that this was

good information or unique enough information to make it

important to include it.

You will also see websites that you have never seen

before. These may be websites that would appear on the

fourth and fifth page of a search, beyond the normal depth

a searcher would go.

Subject Search Directories of Choice

• Ipl2 (Internet Public Library)

• Infomine

• Open Directory

Specialty Search Engines

• Healthline.com (health)

• Scirus.com (science)

• Knowledgestorm.com (computer industry)

• Trulia.com (real estate)

• The Find (shopping)

• And many, many more

Specialty search engines are vertical search engines and

instead of searching the breadth of the world wide web,

they search one subject area in great depth.

Deep Web, Non-Public Information

As you learned in the last lesson, there is a lot of information that is not “showing” in regular search engines. You learned about some of the ways to find information in the Deep Web for government information.

There is one other source that will help find information in the Deep Web.

Complete Planet

There are others that are more specific, like the specialty search engines, but this is the lone survivor of the Deep Web tools.

Readings

• Digital Resource Images on the Deep Web

• 100 Useful Tools and Tips to Research the Deep Web