Insight into Process - Ohio State...
1
Insight into Process Alex Morison, David Tinapple, Dan Zelik, Luke Woods The goal of this project is to find new and interesting directions to explore in exposing information analysis process. In order to accomplish this we setup a workstation, similar to the workstation you see setup here., to record the process of a single analyst. The workstation consists of a computer with an internet browswer, an editor, and a means to record short audio tracks. Several students studying Secu- rity and Intelligence at Ohio State University performed a one hour analysis on a single question related to their area of expertise. Their product was a one page analysis, which was completed by the end of the one hour period. Below we provide a sample screen capture of the workstation computer and include a summary of the different channels of information collected during the analysis. We collected all website URLs visited during the one hour analysis. The logged websites are also time-stamped. We collected the active application and keystrokes entered. Also, we associated with this information a single time-stamp. The computer screen was also captured throughout the entire analysis time- stamped at full resolution with a frequency of 1Hz (1 screen capture per second) . In addition, during the entire analysis we record the mouse position and time-stamp at a rate of 20Hz (20 positions per second) During the analysis we provide a sketch- pad to the analyst, which we recorded once every 20 seconds. The sketchpad provided a free-form environment for ana- lysts to work outside the computer. This data is extremely valuable as not all useful inforrmation is conducive to a virtual medium. During the analysis we provide the analyst with an audio recorder to note information he/she finds especially interesting. The audio recordings are time-stamped and were not accessible to the analyst during the analysis. The audio recordings lasted from ~10secs up to ~1min. How can we make an information analysis process observable? Sketchpad Keystrokes Websites Audio Recordings Computer Screen Mouse In order to obtain a single view of the screen activity for a single analysis, we took advantage of existing computer vision al- gorithms to measure the amount of motion between sequential screen cap- tures. The larger the value the more screen motion and vice versa. As you can see we capture very high activity peiods as well as long periods of little to no motion Screen Analysis Figure: A plot of screen activity versus time. This specific analysis was one hour and focused on the future of the Russian economy, specifically related to the political situation and upcoming ‘08 elections. Applying Computer Vision Algorithms to Computer Screen Analysis
Transcript of Insight into Process - Ohio State...
Insight into ProcessAlex Morison, David Tinapple, Dan Zelik, Luke Woods
The goal of this project is to �nd new and interesting directions to explore in exposing information analysis process. In order to accomplish this we setup a workstation, similar to the workstation you see setup here., to record the process of a single analyst. The workstation consists of a computer with an internet browswer, an editor, and a means to record short audio tracks. Several students studying Secu-rity and Intelligence at Ohio State University performed a one hour analysis on a single question related to their area of expertise. Their product was a one page analysis, which was completed by the end of the one hour period. Below we provide a sample screen capture of the workstation computer and include a summary of the di�erent channels of information collected during the analysis.
We collected all website URLs visited during the one hour analysis. The logged websites are also time-stamped.
We collected the active application and keystrokes entered. Also, we associated with this information a single time-stamp.
The computer screen was also captured throughout the entire analysis time-stamped at full resolution with a frequency of 1Hz (1 screen capture per second) .
In addition, during the entire analysis we record the mouse position and time-stamp at a rate of 20Hz (20 positions per second)
During the analysis we provide a sketch-pad to the analyst, which we recorded once every 20 seconds. The sketchpad provided a free-form environment for ana-lysts to work outside the computer. This data is extremely valuable as not all useful inforrmation is conducive to a virtual medium.
During the analysis we provide the analyst with an audio recorder to note information he/she �nds especially interesting. The audio recordings are time-stamped and were not accessible to the analyst during the analysis. The audio recordings lasted from ~10secs up to ~1min.
How can we make an information analysis process observable?
Sketchpad
Keystrokes
Websites
Audio Recordings
Computer Screen
Mouse
In order to obtain a single view of the screen activity for a single analysis, we took advantage of existing computer vision al-gorithms to measure the amount of motion between sequential screen cap-tures. The larger the value the more screen motion and vice versa. As you can see we capture very high activity peiods as well as long periods of little to no motion
Screen Analysis
Figure: A plot of screen activity versus time. This speci�c analysis was one hour and focused on the future of the Russian economy, speci�cally related to the political situation and upcoming ‘08 elections.
Applying Computer Vision Algorithms to Computer Screen Analysis
