What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… ·...
Transcript of What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… ·...
![Page 1: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/1.jpg)
What We’re Talking About When We’re Talking About Sound
Tanya Clement, Assistant Professor iSchool University of Texas at Austin
![Page 2: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/2.jpg)
HiPSTAS
1. Introducing High Performance Sound Technologies in Access and Scholarship
2. A brief look at current sound/text technologies
3. A brief history of sound technologies and information science
4. Four use cases
![Page 3: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/3.jpg)
http://blogs.ischool.utexas.edu/hipstas/
![Page 4: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/4.jpg)
Recording equipment in the back of John Lomax’s car, late 1930s. (American Roots Music 60)
![Page 5: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/5.jpg)
The 1930s PRESTO Model “K” Recorder
![Page 6: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/6.jpg)
Acetate and aluminum PRESTO disc used by Alan Lomax
![Page 7: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/7.jpg)
Reel-to-reel tape recorder/reproducer.
![Page 8: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/8.jpg)
http://www.clir.org/pubs/abstract/reports/pub148
![Page 9: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/9.jpg)
![Page 10: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/10.jpg)
Stampede at the Texas Advanced Computing Center
![Page 11: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/11.jpg)
HiPSTAS collections • poetry from PennSound at the University of
Pennsylvania
• folklore at the Dolph the Briscoe Center for American History at UT Austin
• speeches at the Lyndon B. Johnson Library and Presidential Museum (LBJ Library) in Austin
• storytelling traditions at the Native American Projects (NAP) at the American Philosophical Society in Philadelphia
![Page 12: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/12.jpg)
HiPSTAS is NOT about
searching or analyzing sound by searching or indexing text
![Page 13: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/13.jpg)
![Page 14: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/14.jpg)
http://whitehousetapes.net/
![Page 15: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/15.jpg)
http://whitehousetapes.net/
![Page 16: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/16.jpg)
http://solstice.ischool.utexas.edu/projects/index.php/LBJ_Gulf_of_Tonkin_Speech
![Page 17: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/17.jpg)
Based on a history of sound machines, psychoacoustics and information
theory
1. The phonoscope: reproducing the speaker’s voice
2. The ear phonoautograph: the middle ear and tympanic instruments
3. The audiometer: the inner ear and psychoacoustics
Sterne, Jonathan. MP3: The Meaning of a Format. Duke University Press Books, 2012. Print.
![Page 18: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/18.jpg)
Jean-Pierre Rousselot’s “phonoscope” (1880s). From Richard Sieburth’s “The Sound of Pound: A Listener’s Guide” on PennSound
![Page 19: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/19.jpg)
Scripture, Edward Wheeler. The Elements of Experimental Phonetics. C. Scribner’s Sons, 1902: 354.
![Page 20: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/20.jpg)
Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901: 94
![Page 21: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/21.jpg)
Scripture, Edward Wheeler. The Elements of Experimental Phonetics. C. Scribner’s Sons, 1902: 42.
![Page 22: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/22.jpg)
Alexander Bell and Clarence Blake, ear phonautograph, 1874
![Page 23: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/23.jpg)
Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901. Print.
![Page 24: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/24.jpg)
Detail of a phonautogram made in 1859 made from Scott’s early phonaugraph
![Page 25: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/25.jpg)
Writing tempo and volume
![Page 26: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/26.jpg)
Carl Seashore, audiometer, 1899
![Page 27: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/27.jpg)
Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901. Print.
![Page 28: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/28.jpg)
In the 1940s and ‘50s, the spectrographic display at Bell Telephone Laboratories http://www.spectrogramsforspeech.com/
![Page 29: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/29.jpg)
HZ, a unit of frequency
Time
a heat based color scheme. White – hottest, most intense Yellow Red Green Blue Black – coolest, least intense
Energy represented by
![Page 30: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/30.jpg)
OpenMary Laynor Stein
Spectogram created with ARLO of the line “. . . some such thing.” (by OpenMary, Gertrude Stein, and Gregory Laynor) from Gertrude Stein's The Making of Americans
Use case #1: same content, different voices
![Page 31: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/31.jpg)
Use case #1: same content, different voices
Spectogram created with ARLO of the line “. . . some such thing.” (by OpenMary, Gertrude Stein, and Gregory Laynor) from Gertrude Stein's The Making of Americans generated by ARLO (Adaptive Recognition with Layered Optimization)
![Page 32: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/32.jpg)
OpenMary Laynor Stein
Spectogram created with ARLO of the line “. . . some such thing.” (by OpenMary, Gertrude Stein, and Gregory Laynor) from Gertrude Stein's The Making of Americans
![Page 33: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/33.jpg)
Pound’s Canto XLV recorded in Harvard, 1939 and DC, 1958
Use case #2: same content, same voices
![Page 34: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/34.jpg)
Pound’s Canto XLV recorded in DC, 1958 and Harvard, 1939
Use case #2: same content, same voices
![Page 35: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/35.jpg)
“design” vs. “delight” in Pound’s Canto XLV recorded in DC, 1958 and Harvard, 1939
DC, 1958
Harvard, 1939
Use case #2: same content? Same voices
![Page 36: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/36.jpg)
“Forgive me” in William Carlos Williams, “This is Just to Say”, Dec., 1951, Harvard
Use case #3: finding similar patterns
![Page 37: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/37.jpg)
Finding “Forgive Me” in ARLO
Use case #3: finding similar patterns
![Page 38: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/38.jpg)
Frost reading poems on Side B. of William A. Owens’ folklore recordings
Use case #4: finding different patterns
![Page 39: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/39.jpg)
Frost reading poems on Side B. of William A. Owens’ folklore recordings
Use case #4: finding different patterns
![Page 40: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/40.jpg)
Use case #4: finding different patterns
![Page 41: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/41.jpg)
Use case #4: finding different patterns
Excerpt from a 2007 interview with Larry Aitken, the tribal historian from the Leech Lake Band of Ojibwe conducted by Dr. Tim Powell when he was director of the Center for Native American studies at the University of Pennsylvania. The second image has been tagged by a user to show the different ways that English, spoken Ojibwe, drum beats, and chanting are visualized.
![Page 42: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/42.jpg)
Use case #4: finding different patterns
Excerpt from a 2007 interview with Larry Aitken, the tribal historian from the Leech Lake Band of Ojibwe conducted by Dr. Tim Powell when he was director of the Center for Native American studies at the University of Pennsylvania. The second image has been tagged by a user to show the different ways that English, spoken Ojibwe, drum beats, and chanting are visualized.
![Page 43: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/43.jpg)
What are we talking about when we talk about sound?
• Tempo, pitch, tone/timbre, dynamics
• Damping ratios, gain, frequencies, spectra, energy, and pitch energy
• What do humanists and archivists tag? Laughter, silence, emotions, applause, pauses, feedback, changing speakers, changing tracks, changing genres
![Page 44: What We’re Talking About When We’re Talking About Sound/67531/metadc228310/m2/1/high_re… · Rousselot, Pierre Jean. Principes de phonétique expérimentale. H. Welter, 1901.](https://reader036.fdocuments.net/reader036/viewer/2022071421/611a412ad311de236b4a5c4e/html5/thumbnails/44.jpg)
Thank you!
Questions?
Special thanks to Tim Powell, my co-PI’s David Tcheng and Loretta Auvil, and the National Endowment for the Humanities.