From 'Netern' to Intern: Student Developed Applications Through Internet2
description
Transcript of From 'Netern' to Intern: Student Developed Applications Through Internet2
![Page 1: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/1.jpg)
From 'Netern' to Intern: Student Developed Applications Through Internet2
Andrew Lake, Albion College
Internet2 Fall Member Meeting 2005
Philadelphia, PA
![Page 2: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/2.jpg)
2
Outline
• Neternship• Internet2 Detective• Tests• Interesting issues encountered
• BlackSquirrel• What it is• How it works
• BRUW• Overview of technology• Future development
![Page 3: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/3.jpg)
3
Neternship
• An internship over the internet• Distributed learning environment• Gives students chance to learn
about and use Internet2 services• Great chance for smaller colleges to
get involved
![Page 4: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/4.jpg)
4
Internet2 Detective
• Program that detects Abilene connection and other capabilities
• Originally stand-alone application
• Java Applet developed through Neternship
![Page 5: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/5.jpg)
5
Java Applets
• Java program embedded in a web page• Runs on user's computer• Must give “permission” for certain tasks• Signing identifies creator and informs
user of what to expect
![Page 6: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/6.jpg)
6
Abilene Test
• Abilene test server sits on backbone
• Detective says “hello” and waits for server to say “hello” back
• Simple and quick
![Page 7: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/7.jpg)
7
Bandwidth Test
• Installs iperf in temporary directory
• Runs iperf in UDP mode
• Parses results from test
• Compares to selected bandwidth
![Page 8: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/8.jpg)
8
Multicast Test
• Uses Java's built-in classes to join multicast group
• If can join, multicast enabled; if not, multicast not enabled
![Page 9: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/9.jpg)
9
IPv6 Test
• Runs iperf from temporary directory (see Bandwidth Test)
• Contacts iperf server using IPv6 address and -V option
• If iperf fails, attempts to run ping6
![Page 10: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/10.jpg)
10
Problem: Firewalls
• Detective uses many non-standard ports• Have to worry about source and
destination ports in each direction• Solution: Directly specify all ports and
document numbers
![Page 11: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/11.jpg)
11
Problem: Network Interfaces
• If multiple network connections, which one should be tested?
• Default connection hard to predict• Solution: Allow user to choose from
list of available interfaces
![Page 12: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/12.jpg)
12
Problem: IPv6 Fragmentation
• Presented by Bill Cerveny at Summer 2005 Joint Techs
• IPv6 Test unable to reach iperf server
• Packets being fragmented causing an extension header to be used and packet to be misidentified by filter
• Solution: Set datagram size to 1450 using iperf's -l option
![Page 13: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/13.jpg)
13
Detective Widget
• Small program that runs in Mac OS X's dashboard
• Not a full detective, just runs Abilene Test
• Fun toy for Mac users, but Konfabulator version coming soon
![Page 14: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/14.jpg)
14
BlackSquirrel
![Page 15: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/15.jpg)
15
BlackSquirrel Background
• BlackSquirrel started as a DV Guide project
• Many digital video stream distributed across Abilene
• Content accessible but how do you find it?
• Eventually, the same questions arose but for archived digital video
• ...and again for video conferencing
• ...and again for non-video data (telescope data, microscope data, images, audio, etc)
• ...and again for bringing all these types of information together
• Realize need for a standard way to describe and join multiple types of distributed content
![Page 16: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/16.jpg)
16
BlackSquirrel Guidelines
1. Primarily targeted toward developers on multiple platforms
2. Creates “back-end” for developer allowing more focus on UI
3. Uses open technologies
4. Minimal install and effort to run
5. Flexible support for many types of descriptions
6. Scalable
7. Simple technical model
![Page 17: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/17.jpg)
17
Technical Model
• Describing the data• Use XML to descibe data
• Make available in feed form (RSS)
• Joining the data• Use hierarchal structure of XML to develop table model
for relational database
• Sharing the data• “Confederacy of parsers” - many parsers retrieve certain
feeds then store in an aggregate feed to be shared
• Using the data• Once in database, developer creates interface to use the
data
![Page 18: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/18.jpg)
18
BlackSquirrel Model
• In simpler terms, a black squirrel retrieves acorns from a squirrel feed then takes the acorns back to the drey where a baby squirrel consumes them. A black squirrel can also take the acorns to another feed to share with other black squirrels.
• Translating back to the more technical version:Acorns = XML tags that describe dataFeed = The XML file that contains all the tagsAnother Feed = Aggregate XML feedDrey = the database that stores the data retrievedBlack Squirrel = the program that retrieves and stores the dataBaby Squirrel = The UI that presents the data
![Page 19: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/19.jpg)
19
Acorns & Feeds
• An acorn refers to individual pieces of description data
• A feed is a text file full of acorns, i.e. description data
• An aggregate feed contains all the acorns from one feed and combines them with acorns from other feeds
![Page 20: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/20.jpg)
20
BlackSquirrel & The Drey
• A BlackSquirrel takes acorns from a feed and puts them in a drey
• The drey is a database where the information is kept for later use
• The drey is a database called Derby developed by the Apache Foundation
![Page 21: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/21.jpg)
21
Baby Squirrels
• A baby squirrel is the consumer of acorns brought back to the drey. This means it is the program that actually uses the data.
• A baby squirrel may be a web page or another program
• Any language that can support Apache Derby will work. This list includes PHP, Perl, ASP, JSP, and Java
• Allows developer to focus more on the UI and less on developing a system to manage data
![Page 22: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/22.jpg)
22
Video Diagram
![Page 23: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/23.jpg)
23
Telescope Diagram
![Page 24: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/24.jpg)
24
BlackSquirrel Diagram
![Page 25: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/25.jpg)
25
Podcasting
• “Podcasting” is the automatic download of an audio file attached to an RSS item
• BlackSquirrel podcasting works with any type of file.
• BlackSquirrel supports this by allowing a user to specify a special BLOB field in the database associated with a URL property or element
• When specified, automatically downloads resource and stores in database
• Useful because content can sit directly next to description
• Database allows for quick access
• Easy to extract
![Page 26: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/26.jpg)
26
BlackSquirrel Future
• Expand Interface• Add user preferences• Allow selection of database location• Export BLOB data• Auto-Start Derby
• Expand support for downloading protocols (FTP, HTTPS, etc)
• Add BitTorrent functionality
• Create a SOAP server to update database in real time
• Add security and authentication features
![Page 27: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/27.jpg)
27
BRUW – Got Bandwidth?
![Page 28: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/28.jpg)
28
BRUW Overview
• BRUW stands for Bandwidth Reservation for User Work
• Started by Soo-yeon Hwang in 2004• Users reserve bandwidth through a web
interface (front-end)• MPLS Tunnels periodically built to
“reserve” bandwidth requested for that time (back-end)
• Front-end and back-end share database
![Page 29: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/29.jpg)
29
BRUW
![Page 30: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/30.jpg)
30
Authentication
• Shibboleth• Create new BRUW account• Administrator authorizes the account• User sent activation key to activate
account
![Page 31: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/31.jpg)
31
Making a Reservation
![Page 32: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/32.jpg)
32
Back-End
• Runs in cron• Builds tunnels just before every hour• Removes tunnels just after every hour
• Discovers edge routers• Makes reservation between edge routers
in each direction
![Page 33: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/33.jpg)
33
Edge Discovery
• In order to make reservation, BRUW must locate the edge nodes
• Edge nodes are the first and last router that a transmission will cross on Abilene
• Ingress Node is router used to enter Abilene
• Egress Node is the router used to exit Abilene
![Page 34: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/34.jpg)
34
Reservation Building
• Once edge routers discovered, bandwidth can be reserved between nodes
• Communicate with routers using JUNOScript and XML to change configurations and build/remove necessary MPLS tunnels on Juniper Routers
• Tunnels built in both directions
![Page 35: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/35.jpg)
35
Back-End Diagram
![Page 36: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/36.jpg)
36
Test Results
• BRUW has successfully built tunnels between Chicago and Los Angeles
• Work still needs to be done to determine the exact configurations necessary to run BRUW
![Page 37: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/37.jpg)
37
Future of BRUW
• Merge codebase with OSCARS• Interdomain reservations• Further Shibboleth integration• Get closer to solving last mile problem• Look into cooperation with other similar
projects
![Page 38: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/38.jpg)
38
What I learned...
• Worked with IPv6 and Multicast• Use of network measurement tools• SAX• Apache Derby• Java Server Pages• Routing Protocols• MPLS• Lots more!!!
![Page 39: From 'Netern' to Intern: Student Developed Applications Through Internet2](https://reader036.fdocuments.net/reader036/viewer/2022062803/568147ab550346895db4e6f3/html5/thumbnails/39.jpg)
39
http://vbvod.internet2.edu/internship