In the DEEP Web

Let’s go fishing…

As summarized from a White Paper by Michael K Bergman entitled, “The Deep Web: Surfacing Hidden Value”

A link to the 50+ page document is provided

at the end of this lecture (1).

“Searching on the Internet today can be compared to dragging a net across the surface of the ocean.”

Here’s another way to look at it.

Some key points to remember

“Most of the Web’s information is buried down on dynamically generated sites, and standard search engines never find it.”

“The deep Web is qualitatively different from the surface Web.”

Bright Planet has quantified the size and relevancy of the deep Web in a study based on data collected in 2000.

Key findings from the study include:

Public information on the deep Web in currently 400-550 times larger than the commonly defined WWW.

Deep Web contains 7,500 terabytes of information compared to 19 terabytes on the surface Web.

Deep Web contains nearly 550 billion individual documents compared to 1 billion on the surface Web.

More than 200,000 deep Web sites presently exist.

Key findings from the study include:

The typical deep Web site is not well known to the Internet-searching public.

The deep Web is the largest growing category of new information on the Internet.

More than half of the deep Web content resides in topic-specific databases.

A full 95% of the deep Web is publicly accessible information – not subject to fees or subscriptions.

I Love this quote!

“Eighty-five percent of Web users use search engines to find needed information, but nearly as high a percentage

cite the inability to find desired information as one of their biggest frustrations.”

Bright Planet did an analysis of the largest deep Web sites And compiled a list of 60 known, largest deep Web sites

containing data of about 750 terabytes of roughly 40 times the size of the known surface Web.

Roughly two-thirds of these sites are public ones, representing about 90% of the content available within this group of sixty.

We will be exploring two of these sites in our exploration of government resources, US Census and GPO Access.

The full list of 60 sites is presented as Table 2 in the linked White Paper at the end of this lecture .

Bear with me as we review some basic civics we learned in the 4th grade. It makes for a nice framework moving forward.

Who remembers this?

Three Ring Government

Video is attached in the Unit 3 Support section and is provided courtesy of YouTube.

Yes, that is right.

Executive Branch

Legislative Branch

Judicial Branch

All three branches are represented in the deep Web.

Executive Branch

President

Vice-President

Departments (Presidential Cabinet) – Some examples include

Department of Transportation

Department of Labor

Department of Justice

Executive Branch

President

Vice-President

Departments (Presidential Cabinet) – Some examples include Department of Transportation

Department of Labor

Department of JusticeNow, let’s just drill a little deeper on this one department.

Department of Justice The primary rulemaking components of the

Department of Justice include:

Drug Enforcement Administration (DEA)

Bureau of Alcohol, Tobacco, Firearms, and Explosives (ATF)

Federal Bureau of Investigation (FBI)

Bureau of Prisons (BOP)

Civil Rights Division

Office of Justice Programs

Executive Office for Immigration Review

Department of Justice The primary rulemaking components of the

Department of Justice include:

Drug Enforcement Administration (DEA) Bureau of Alcohol, Tobacco, Firearms, and Explosives

(ATF)

Federal Bureau of Investigation (FBI)

Bureau of Prisons (BOP)

Civil Rights Division

Office of Justice Programs

Executive Office for Immigration ReviewOnce again let’s drive down on one facet – the DEA

Drug Enforcement Administration

(DEA) enforces federal controlled substances laws and regulations. Through its diversion control program (DCP), the DEA regulates drug manufacturers, distributors, importers, exporters, hospitals, doctors, pharmacists, and others involved with controlled substances, and tracks transactions involving designated chemicals that have legitimate uses but are subject to diversion for illicit purposes.

Just one example… Controlled Substances Act (2).

Just one of the laws they enforce. It is codified as part of the US Code. The DEA is charged with enforcing its provisions. US Code is one of the sources we will work with in this unit.

Okay let’s back up.

Executive Branch

Legislative BranchJudicial Branch

Legislative Branch

Congress

House of Representatives

Senate

What do they do?

No jokes here. They make the laws.

How?

Nope, I couldn’t resist one more.

It’s Bill!

Video is attached in the Unit 3 Support section and is provided courtesy of YouTube.

Judicial BranchCourts decide arguments about the

meaning of laws, how they are applied, and whether they break the rules of the Constitution.The Supreme Court and the other courts

in the Federal system issue opinions (or rulings) and access to those opinions are considered part of the deep Web.

As we have discussed, there are 3 branches of government. We have focused thus far on the Federal Level. In addition to the Federal level, we must also learn about the other two levels of government:

State and LocalThere are deep Web resources for those as well.

Please note as we move through this portion of the lecture, I am using examples based on the State of MISSOURI and the Local of SPRINGFIELD. You can adapt these examples to fit your state and locality.

Let us proceed.

Executive BranchState Local

Missouri Governor, Jay NixonWithin the Executive Branch are six statewide elected officials and sixteen executive departments responsible for executing the laws of the state.

Springfield Mayor, Bob Stephens works with City staff in the various departments to see that the laws (city ordinances) are enforced.

Legislative BranchState Local

House of Representatives and Senate make up the Missouri Congress in Jefferson City.

The bill to law process matches the Federal system.

Springfield City Council

Citizens, City staff or Council members suggest an ordinance that may resolve a problem or address a concern.

Those proposed ordinances go through a series of readings, debate, committee work, and a vote before some are enacted into law. Some die just like at Fed or State level.

Judicial BranchState Local

Missouri Court System Springfield Municipal Court

Deep Web at the State and Local Levels

Missouri Revised Statutes (3) Missouri Code of State Regulations (4) Casenet – Missouri Court records case records including docket

entries, parties, judgments and charges in public court. Restrictions apply (5).

Springfield City Code and Zoning Ordinances (6) Springfield MO Outstanding Municipal Warrants (7)

I point these out not because we will necessarily cover them, but to illustrate that these are all government resources that can be considered part of the deep Web. Including them may also encourage you to seek out such resources in your own jurisdictions.

This is a three-week unit on Government Resources.

Week One – Reading and Discussion Board ParticipationWeek Two – Exploration of ResourcesWeek Three – Practice via Assignment

Please visit the Discussion Board, Unit Support, and Assignments sections of Blackboard to proceed.

BIBLIOGRAPHY OF LECTURE RESOURCES

1. http://quod.lib.umich.edu/cgi/t/text/text-idx?c=jep;view=text;rgn=main;idno=3336451.0007.104

2. http://www.deadiversion.usdoj.gov/21cfr/21usc/index.html

3. http://www.moga.mo.gov/statutes/statutes.htm

4. www.sos.mo.gov/adrules/csr/csr.asp

5. https://www.courts.mo.gov/casenet/base/welcome.do

6. http://library.municode.com/index.aspx?clientId=11598

7. http://www.springfieldmo.gov/spd/wanted/municipalwarrants/default.aspx