Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac
description
Transcript of Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac
![Page 1: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/1.jpg)
Low Cost, Scalable Proteomics Data Analysis Using Amazon's Cloud Computing Services and Open
Source Search Algorithms
Brian D. Halligan, Ph.D.Medical College of Wisconsin
http://proteomics.mcw.edu/[email protected]
ViPDAC
![Page 2: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/2.jpg)
What is ViPDAC?
• ViPDAC => Virtual Proteomics Data Analysis Cluster
• One of the slowest parts of proteomics is data analysis.
• Single CPU machines analyze data much slower than instruments can generate it.
• Computer Clusters offer increased speed, but have high costs to implement and maintain.
![Page 3: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/3.jpg)
Cloud Computing
• Distributed or Cloud computing allows for the use of virtual computers to perform computer intensive tasks without having to own the computer.
• Amazon has built large scale computing facilities that they offer for use on an hourly basis.
• The cost of analysis using this system is very low and the size of the cluster can expand, contract or even disappear based on need.
![Page 4: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/4.jpg)
Amazon Web Services (AWS)
• EC2 – Amazon Elastic Compute Cloud“a web service that provides resizable compute
capacity in the cloud. It is designed to make web-scale computing easier for developers. “
• S3 - Amazon Simple Storage Service“provides a simple web services interface that can
be used to store and retrieve any amount of data, at any time, from anywhere on the web.”
![Page 5: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/5.jpg)
Overview
![Page 6: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/6.jpg)
Workflow
![Page 7: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/7.jpg)
Time vs. Nodes
![Page 8: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/8.jpg)
ViPDAC Costs per Run
Charge Amount Used Unit Size Units Cost / Unit Cost EC2 EC2 - Data Transfer In 156 MB 1 GB 1 $0.10 $0.10 EC2 - Data Transfer Out 3.3 MB 1 GB 1 $0.17 $0.17 High CPU Instance (Medium) 2 instance-hr 1 instance-hr 2 $0.20 $0.40 S3 Request - Tier 1 227 1,000 1 $0.01 $0.01 Request - Tier 2 394 10,000 1 $0.01 $0.01 C3 Data Transfer In 191 MB No charge C3 Data Transfer Out 798 MB No charge Storage 36.6 MB 1 GB 1 $0.15 $0.15 Total $0.84 Amount for requests is the number of message and as indicated for data and computation. Unit is the metric that Amazon uses to assess charges. Charges are assessed for any partial unit usage.
![Page 9: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/9.jpg)
Advantages of ViPDAC
• Low cost– No startup costs– Low hourly usage costs– No cost when not in use
• Scalable– Everyone is first in line– Launch as few or as many worker nodes as needed– Fast costs the same as slow – 1 instance for 20 hrs = 20 instances for 1 hr
![Page 10: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/10.jpg)
Advantages of ViPDAC
• Secure– Data is stored and transferred in a secure system– Your data/database does not leave your control
and is not seen or shared with others• Stable
– AMI can be cloned and saved– Consistent data analysis for long term projects– SOP across laboratories
![Page 11: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/11.jpg)
Advantages of ViPDAC
• Cost Accounting– Very easy to determine cost of a single run with
ViPDAC compared to physical cluster
• Freedom to experiment– Can perform complex analysis on a dataset
without blocking routine analysis– Custom interface or analysis
![Page 12: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/12.jpg)
![Page 13: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/13.jpg)
![Page 14: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/14.jpg)
![Page 15: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/15.jpg)
![Page 16: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/16.jpg)
![Page 17: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/17.jpg)
![Page 18: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/18.jpg)
![Page 19: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/19.jpg)
![Page 20: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/20.jpg)
![Page 21: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/21.jpg)
![Page 22: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/22.jpg)
![Page 23: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/23.jpg)
![Page 24: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/24.jpg)
![Page 25: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/25.jpg)
![Page 26: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/26.jpg)
![Page 27: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/27.jpg)
![Page 28: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/28.jpg)
![Page 29: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/29.jpg)
![Page 30: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/30.jpg)
![Page 31: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/31.jpg)
![Page 32: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/32.jpg)
![Page 33: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/33.jpg)
![Page 34: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/34.jpg)
![Page 35: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/35.jpg)
![Page 36: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/36.jpg)
![Page 37: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/37.jpg)
![Page 38: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/38.jpg)
![Page 39: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/39.jpg)
![Page 40: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/40.jpg)
![Page 41: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/41.jpg)
![Page 42: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/42.jpg)
Acknowledgments
• Joey F. Geiger• Andrew K. Vallejos• Simon N. Twigger• Andrew S. Greene
– MCW NHLBI Proteomics Center
http://proteomics.mcw.edu/vipdac
![Page 43: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/43.jpg)
![Page 44: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/44.jpg)
![Page 45: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/45.jpg)
![Page 46: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/46.jpg)
![Page 47: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/47.jpg)
![Page 48: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/48.jpg)
![Page 49: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/49.jpg)
![Page 50: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/50.jpg)
![Page 51: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/51.jpg)
![Page 52: Brian D. Halligan, Ph.D. Medical College of Wisconsin proteomics.mcw/vipdac](https://reader033.fdocuments.net/reader033/viewer/2022061616/56814901550346895db62ee4/html5/thumbnails/52.jpg)
Acknowledgments
• Joey F. Geiger• Andrew K. Vallejos• Simon N. Twigger• Andrew S. Greene
– MCW NHLBI Proteomics Center
http://proteomics.mcw.edu/vipdac