Events Unimportant: A Short History of the VSO
Transcript of Events Unimportant: A Short History of the VSO
Events Unimportant:A Short History of the VSO
Joe GurmanNASA Goddard Space Flight Center
Heliophysics DivisionLaboratory for Solar Physics
andThe VSO Implementers
History, n. An account mostly false, of events unimportant, which are brought about by rulers
mostly knaves, and soldiers mostly fools.- Ambrose Bierce, The Devil’s Dictonary
VOiG Session I, 2007•06•12 J.B. Gurman
There was a definite process by which one made people into friends, and it involved talking to them and listening to them
for hours at a time.- Rebecca West
• Rick Bogart, Stanford
• Alisdair Davey, SwRI
• Frank Hill, NSO
• Joe Hourclé, NASA GSFC
• Piet Martens, MSU
• Igor Suàrez-Sola, NSO
• Karen Tian, Stanford
• Keiji Yoshimura, MSU
• Kevin Reardon, Arcetri
2
VOiG Session I, 2007•06•12 J.B. Gurman
People that are really very weird can get into sensitive positions and have a tremendous impact on history.
- Dan Quayle
3
VOiG Session I, 2007•06•12 J.B. Gurman
People that are really very weird can get into sensitive positions and have a tremendous impact on history.
- Dan Quayle
• First attempts at a VSO were organized by Bernhard Fleck (1995 - 1997), as the “Whole Sun Catalog”
3
VOiG Session I, 2007•06•12 J.B. Gurman
People that are really very weird can get into sensitive positions and have a tremendous impact on history.
- Dan Quayle
• First attempts at a VSO were organized by Bernhard Fleck (1995 - 1997), as the “Whole Sun Catalog”
• Unfunded by European sources, 1999
3
VOiG Session I, 2007•06•12 J.B. Gurman
People that are really very weird can get into sensitive positions and have a tremendous impact on history.
- Dan Quayle
• First attempts at a VSO were organized by Bernhard Fleck (1995 - 1997), as the “Whole Sun Catalog”
• Unfunded by European sources, 1999
• BoF session at 2000 Solar Physics Division meeting in Lake Tahoe, proposal effort led by Frank Hill
3
VOiG Session I, 2007•06•12 J.B. Gurman
People that are really very weird can get into sensitive positions and have a tremendous impact on history.
- Dan Quayle
• First attempts at a VSO were organized by Bernhard Fleck (1995 - 1997), as the “Whole Sun Catalog”
• Unfunded by European sources, 1999
• BoF session at 2000 Solar Physics Division meeting in Lake Tahoe, proposal effort led by Frank Hill
• Unfunded by NSF, 2001
3
VOiG Session I, 2007•06•12 J.B. Gurman
People that are really very weird can get into sensitive positions and have a tremendous impact on history.
- Dan Quayle
• First attempts at a VSO were organized by Bernhard Fleck (1995 - 1997), as the “Whole Sun Catalog”
• Unfunded by European sources, 1999
• BoF session at 2000 Solar Physics Division meeting in Lake Tahoe, proposal effort led by Frank Hill
• Unfunded by NSF, 2001
• Parallel effort led by Rick Bogart
3
VOiG Session I, 2007•06•12 J.B. Gurman
People that are really very weird can get into sensitive positions and have a tremendous impact on history.
- Dan Quayle
• First attempts at a VSO were organized by Bernhard Fleck (1995 - 1997), as the “Whole Sun Catalog”
• Unfunded by European sources, 1999
• BoF session at 2000 Solar Physics Division meeting in Lake Tahoe, proposal effort led by Frank Hill
• Unfunded by NSF, 2001
• Parallel effort led by Rick Bogart
• Unfunded by NASA LWS DATM (now TR&T), 20013
VOiG Session I, 2007•06•12 J.B. Gurman
History teaches us that men and nations behave wisely once they have exhausted all other alternatives.
- Abba Eban
4
VOiG Session I, 2007•06•12 J.B. Gurman
History teaches us that men and nations behave wisely once they have exhausted all other alternatives.
- Abba Eban
• Proposed as leading to the “withering away” of the Solar Data Analysis Center
4
VOiG Session I, 2007•06•12 J.B. Gurman
History teaches us that men and nations behave wisely once they have exhausted all other alternatives.
- Abba Eban
• Proposed as leading to the “withering away” of the Solar Data Analysis Center
• Funded by NASA Sun-Earth Connections senior review, 2001
4
VOiG Session I, 2007•06•12 J.B. Gurman
History teaches us that men and nations behave wisely once they have exhausted all other alternatives.
- Abba Eban
• Proposed as leading to the “withering away” of the Solar Data Analysis Center
• Funded by NASA Sun-Earth Connections senior review, 2001
• Effort was to include:
4
VOiG Session I, 2007•06•12 J.B. Gurman
History teaches us that men and nations behave wisely once they have exhausted all other alternatives.
- Abba Eban
• Proposed as leading to the “withering away” of the Solar Data Analysis Center
• Funded by NASA Sun-Earth Connections senior review, 2001
• Effort was to include:
• one-year of design work
4
VOiG Session I, 2007•06•12 J.B. Gurman
History teaches us that men and nations behave wisely once they have exhausted all other alternatives.
- Abba Eban
• Proposed as leading to the “withering away” of the Solar Data Analysis Center
• Funded by NASA Sun-Earth Connections senior review, 2001
• Effort was to include:
• one-year of design work
• one year of development, two of maintenance
4
VOiG Session I, 2007•06•12 J.B. Gurman
Much of the social history of the Western world over the last three decades has involved replacing what worked with what
sounded good. - Thomas Sowell, Is Reality Optional? (1993)
• Two incentives for moving to a distributed data service environment
• Data centers no longer provided either:
• unique expertise or hardware
• services too expensive to distribute
• Prospective mission data sets appeared too large to replicate over the network
5
VOiG Session I, 2007•06•12 J.B. Gurman
History repeats itself. That’s one of the things wrong with history.
- Clarence Darrow
• Not even log-linear.... ulp
6
VOiG Session I, 2007•06•12 J.B. Gurman
Indeed, what is there that does not appear marvelous when it comes to our knowledge for the first time? How many things, too, are looked upon as quite impossible until they have been actually effected?
- Pliny the Elder, Natural History
• Design, 2002
• Would have like to believe our design was unique
• But similar designs were converged on by NVO, PDS
• So probably just one quasi-optimal way to do this
7
VOiG Session I, 2007•06•12 J.B. Gurman
I must study politics and war that my sons may have liberty to study mathematics and philosophy. My sons ought to study mathematics and philosophy, geography, natural
history, naval architecture, navigation, commerce and agriculture in order to give their children a right to study painting, poetry, music, architecture, statuary, tapestry, and
porcelain.- John Adams, in a letter to Abigail Adams
8
VOiG Session I, 2007•06•12 J.B. Gurman
I must study politics and war that my sons may have liberty to study mathematics and philosophy. My sons ought to study mathematics and philosophy, geography, natural
history, naval architecture, navigation, commerce and agriculture in order to give their children a right to study painting, poetry, music, architecture, statuary, tapestry, and
porcelain.- John Adams, in a letter to Abigail Adams
• Design requirements:
8
VOiG Session I, 2007•06•12 J.B. Gurman
I must study politics and war that my sons may have liberty to study mathematics and philosophy. My sons ought to study mathematics and philosophy, geography, natural
history, naval architecture, navigation, commerce and agriculture in order to give their children a right to study painting, poetry, music, architecture, statuary, tapestry, and
porcelain.- John Adams, in a letter to Abigail Adams
• Design requirements:• Distributed
8
VOiG Session I, 2007•06•12 J.B. Gurman
I must study politics and war that my sons may have liberty to study mathematics and philosophy. My sons ought to study mathematics and philosophy, geography, natural
history, naval architecture, navigation, commerce and agriculture in order to give their children a right to study painting, poetry, music, architecture, statuary, tapestry, and
porcelain.- John Adams, in a letter to Abigail Adams
• Design requirements:• Distributed
• Search for and access multiple missions/observatories/wavelengths without intimate knowledge of the data organization (e.g. by physical observable and/or mission/instrument)
8
VOiG Session I, 2007•06•12 J.B. Gurman
I must study politics and war that my sons may have liberty to study mathematics and philosophy. My sons ought to study mathematics and philosophy, geography, natural
history, naval architecture, navigation, commerce and agriculture in order to give their children a right to study painting, poetry, music, architecture, statuary, tapestry, and
porcelain.- John Adams, in a letter to Abigail Adams
• Design requirements:• Distributed
• Search for and access multiple missions/observatories/wavelengths without intimate knowledge of the data organization (e.g. by physical observable and/or mission/instrument)
• Provide access to analysis software, instrument descriptions, &c. that enable use in scientific research
8
VOiG Session I, 2007•06•12 J.B. Gurman
I must study politics and war that my sons may have liberty to study mathematics and philosophy. My sons ought to study mathematics and philosophy, geography, natural
history, naval architecture, navigation, commerce and agriculture in order to give their children a right to study painting, poetry, music, architecture, statuary, tapestry, and
porcelain.- John Adams, in a letter to Abigail Adams
• Design requirements:• Distributed
• Search for and access multiple missions/observatories/wavelengths without intimate knowledge of the data organization (e.g. by physical observable and/or mission/instrument)
• Provide access to analysis software, instrument descriptions, &c. that enable use in scientific research
• Given the funding level, had to draw a “small box” around a small set of attributes useful for doing science
8
VOiG Session I, 2007•06•12 J.B. Gurman
Happy are the people whose annals are blank in history books.
- Thomas Carlyle, Cromwell’s Letter and Speeches
9
VOiG Session I, 2007•06•12 J.B. Gurman
Happy are the people whose annals are blank in history books.
- Thomas Carlyle, Cromwell’s Letter and Speeches
• What we decided we couldn’t offer, at least at first:
9
VOiG Session I, 2007•06•12 J.B. Gurman
Happy are the people whose annals are blank in history books.
- Thomas Carlyle, Cromwell’s Letter and Speeches
• What we decided we couldn’t offer, at least at first:
• Data mining (knowledge discovery)
9
VOiG Session I, 2007•06•12 J.B. Gurman
Happy are the people whose annals are blank in history books.
- Thomas Carlyle, Cromwell’s Letter and Speeches
• What we decided we couldn’t offer, at least at first:
• Data mining (knowledge discovery)
• Remote processing: left for EGSO, CoSEC
9
VOiG Session I, 2007•06•12 J.B. Gurman
Happy are the people whose annals are blank in history books.
- Thomas Carlyle, Cromwell’s Letter and Speeches
• What we decided we couldn’t offer, at least at first:
• Data mining (knowledge discovery)
• Remote processing: left for EGSO, CoSEC
• Grid computing (as opposed to data grids)
9
VOiG Session I, 2007•06•12 J.B. Gurman
Happy are the people whose annals are blank in history books.
- Thomas Carlyle, Cromwell’s Letter and Speeches
• What we decided we couldn’t offer, at least at first:
• Data mining (knowledge discovery)
• Remote processing: left for EGSO, CoSEC
• Grid computing (as opposed to data grids)
• Access control (ever)
9
VOiG Session I, 2007•06•12 J.B. Gurman
Happy are the people whose annals are blank in history books.
- Thomas Carlyle, Cromwell’s Letter and Speeches
• What we decided we couldn’t offer, at least at first:
• Data mining (knowledge discovery)
• Remote processing: left for EGSO, CoSEC
• Grid computing (as opposed to data grids)
• Access control (ever)
• We wanted only to be in the business of helping data providers to give data away, not preventing its access
9
VOiG Session I, 2007•06•12 J.B. Gurman
Web standards keep you out of the dustbin of history.- Josh Greenburg, Digital Preservation and Blogs (SXSW 2006)
10
VOiG Session I, 2007•06•12 J.B. Gurman
Web standards keep you out of the dustbin of history.- Josh Greenburg, Digital Preservation and Blogs (SXSW 2006)
• Best decisions (in hindsight):
10
VOiG Session I, 2007•06•12 J.B. Gurman
Web standards keep you out of the dustbin of history.- Josh Greenburg, Digital Preservation and Blogs (SXSW 2006)
• Best decisions (in hindsight):
• Chose SOAP over various competing methodologies
10
VOiG Session I, 2007•06•12 J.B. Gurman
Web standards keep you out of the dustbin of history.- Josh Greenburg, Digital Preservation and Blogs (SXSW 2006)
• Best decisions (in hindsight):
• Chose SOAP over various competing methodologies
• Were forced to KISS (“small box”) because of very constrained budget
10
VOiG Session I, 2007•06•12 J.B. Gurman
Web standards keep you out of the dustbin of history.- Josh Greenburg, Digital Preservation and Blogs (SXSW 2006)
• Best decisions (in hindsight):
• Chose SOAP over various competing methodologies
• Were forced to KISS (“small box”) because of very constrained budget
• Few non-data services, no public outreach features
10
VOiG Session I, 2007•06•12 J.B. Gurman
Web standards keep you out of the dustbin of history.- Josh Greenburg, Digital Preservation and Blogs (SXSW 2006)
• Best decisions (in hindsight):
• Chose SOAP over various competing methodologies
• Were forced to KISS (“small box”) because of very constrained budget
• Few non-data services, no public outreach features
• Supported by SolarSoft for a good fraction of community’s needs
10
VOiG Session I, 2007•06•12 J.B. Gurman
Web standards keep you out of the dustbin of history.- Josh Greenburg, Digital Preservation and Blogs (SXSW 2006)
• Best decisions (in hindsight):
• Chose SOAP over various competing methodologies
• Were forced to KISS (“small box”) because of very constrained budget
• Few non-data services, no public outreach features
• Supported by SolarSoft for a good fraction of community’s needs
• Continually went to community for input
10
VOiG Session I, 2007•06•12 J.B. Gurman
Web standards keep you out of the dustbin of history.- Josh Greenburg, Digital Preservation and Blogs (SXSW 2006)
• Best decisions (in hindsight):
• Chose SOAP over various competing methodologies
• Were forced to KISS (“small box”) because of very constrained budget
• Few non-data services, no public outreach features
• Supported by SolarSoft for a good fraction of community’s needs
• Continually went to community for input
• AAS, SPD, AGU meetings/BoF’s
10
VOiG Session I, 2007•06•12 J.B. Gurman
Web standards keep you out of the dustbin of history.- Josh Greenburg, Digital Preservation and Blogs (SXSW 2006)
• Best decisions (in hindsight):
• Chose SOAP over various competing methodologies
• Were forced to KISS (“small box”) because of very constrained budget
• Few non-data services, no public outreach features
• Supported by SolarSoft for a good fraction of community’s needs
• Continually went to community for input
• AAS, SPD, AGU meetings/BoF’s
• Beta testing and feedback
10
VOiG Session I, 2007•06•12 J.B. Gurman
Perhaps the history of the errors of mankind, all things considered, is more valuable and interesting than that of
their discoveries.-Benjamin Franklin, Report to the King of France on Animal Magnetism
11
VOiG Session I, 2007•06•12 J.B. Gurman
Perhaps the history of the errors of mankind, all things considered, is more valuable and interesting than that of
their discoveries.-Benjamin Franklin, Report to the King of France on Animal Magnetism
• What we had to change or add
11
VOiG Session I, 2007•06•12 J.B. Gurman
Perhaps the history of the errors of mankind, all things considered, is more valuable and interesting than that of
their discoveries.-Benjamin Franklin, Report to the King of France on Animal Magnetism
• What we had to change or add
• Registry
11
VOiG Session I, 2007•06•12 J.B. Gurman
Perhaps the history of the errors of mankind, all things considered, is more valuable and interesting than that of
their discoveries.-Benjamin Franklin, Report to the King of France on Animal Magnetism
• What we had to change or add
• Registry
• Replication of registry
11
VOiG Session I, 2007•06•12 J.B. Gurman
Perhaps the history of the errors of mankind, all things considered, is more valuable and interesting than that of
their discoveries.-Benjamin Franklin, Report to the King of France on Animal Magnetism
• What we had to change or add
• Registry
• Replication of registry
• Data model (still being expanded)
11
VOiG Session I, 2007•06•12 J.B. Gurman
Perhaps the history of the errors of mankind, all things considered, is more valuable and interesting than that of
their discoveries.-Benjamin Franklin, Report to the King of France on Animal Magnetism
• What we had to change or add
• Registry
• Replication of registry
• Data model (still being expanded)
• Writing SOAP servers for some data providers
11
VOiG Session I, 2007•06•12 J.B. Gurman
Perhaps the history of the errors of mankind, all things considered, is more valuable and interesting than that of
their discoveries.-Benjamin Franklin, Report to the King of France on Animal Magnetism
• What we had to change or add
• Registry
• Replication of registry
• Data model (still being expanded)
• Writing SOAP servers for some data providers
• Providing proxies for database-less providers
11
VOiG Session I, 2007•06•12 J.B. Gurman
Perhaps the history of the errors of mankind, all things considered, is more valuable and interesting than that of
their discoveries.-Benjamin Franklin, Report to the King of France on Animal Magnetism
• What we had to change or add
• Registry
• Replication of registry
• Data model (still being expanded)
• Writing SOAP servers for some data providers
• Providing proxies for database-less providers
• Even some hardware for one data provider
11
VOiG Session I, 2007•06•12 J.B. Gurman
Perhaps the history of the errors of mankind, all things considered, is more valuable and interesting than that of
their discoveries.-Benjamin Franklin, Report to the King of France on Animal Magnetism
• What we had to change or add
• Registry
• Replication of registry
• Data model (still being expanded)
• Writing SOAP servers for some data providers
• Providing proxies for database-less providers
• Even some hardware for one data provider
• Help in getting IDL-callable API working
11
VOiG Session I, 2007•06•12 J.B. Gurman
Women with pasts interest men... they hope history will repeat itself.
- Mae West
• What do we have to offer to newer VxO’s?• One possible model for how a VxO can be architected
• Humility: we have never approached the issues of building a VO with integrated software and/or visualization services
• Solar eye for the plasma guy: a possibly different approach, possibly optimized for remote sensing rather than in situ measurements, and generally accepted file standards
• A test case for tolerance: we view SPASE as a communication protocol for your queries, not a solution to all problems
• A willingness to work with you to build a Heliophysics meta-VO
12