building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor...
Transcript of building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor...
![Page 2: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/2.jpg)
Scholarly Article Recommendation
• Information Overload– 50m – 150m articles in existence
![Page 3: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/3.jpg)
Google Scholar
• Recommendation vs Search– Serendipity
• Homonymity• Synonymity
![Page 4: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/4.jpg)
Netflix/Spotify/Amazon
• User ratings (explicit, implicit)• Density– # user-item interactions >> # items
• Netflix Competition (2006)1
– 100m ratings– 480k users– 17k movies
1: http://www.netflixprize.com/community/viewtopic.php?id=68
![Page 5: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/5.jpg)
Barriers to Research
• Hard to get datasets• Difficult to measure effectiveness– Judges– Citation prediction
![Page 6: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/6.jpg)
Enter Babel
• Provide access to private data sets• Provide scholarly article recommendations,
freely to anyone– Feedback data in return
• Evaluate recommenders using usage data– With enough traffic could be very fast
![Page 7: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/7.jpg)
Audience
• Publishers– Offload expensive research into recommender systems
to academia– Better recommendations drive more traffic/purchases
• Tool Developers• Researchers
![Page 8: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/8.jpg)
Requirements
• Fast• Reliable• Scalable (lots of data!)• Easy to use• Cheap
![Page 9: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/9.jpg)
REST APIcurl http://babel-us-east-1.eigenfactor.org/recommendation/aminer/12345{"transaction_id": "46bb84190e9ddfd17700bfafb500ab3c","results": [
{"paper_id": "672","publisher": "aminer"
},{"paper_id": "11274","publisher": "aminer"
} ]
}
![Page 10: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/10.jpg)
http://babel.eigenfactor.org
![Page 11: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/11.jpg)
Browser Plugins
![Page 12: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/12.jpg)
http://labs.jstor.org/sustainability/
![Page 13: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/13.jpg)
Babel Architecture
Recommenders
EigenFactor Recommends
Co-Citation
Bibliographic Coupling
Metadata Database
update.eigenfactor.org
Object Store
Archive
Metadata Extraction
Recommender Frontend
Publisher
DemoWebsite
Chrome Plugin Analytics
Normalization
Researcher
Recommendation Cache
DesktopApp
![Page 14: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/14.jpg)
Frontend
Recommenders
EigenFactor Recommends
Co-Citation
Bibliographic Coupling
Metadata Database
update.eigenfactor.org
Object Store
Archive
Metadata Extraction
Recommender Frontend
Publisher
DemoWebsite
Chrome Plugin Analytics
Normalization
Researcher
Recommendation Cache
DesktopApp
![Page 15: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/15.jpg)
Frontend
AWS Elastic Bean Stalk
Application
Package
Deploy
![Page 16: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/16.jpg)
Frontend
AWS Elastic Bean Stalk
Application
Package
Deploy
![Page 17: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/17.jpg)
![Page 18: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/18.jpg)
Swagger UI
![Page 19: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/19.jpg)
Swagger UI
![Page 20: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/20.jpg)
Frontend
AWS Elastic Bean Stalk
Application
Package
Deploy
![Page 21: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/21.jpg)
![Page 22: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/22.jpg)
Frontend
AWS Elastic Bean Stalk
Application
Package
Deploy
![Page 23: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/23.jpg)
AWS Elastic Bean Stalk
Image:Part1:Develop,Deploy,andManageforScalewithElasticBeanstalkandCloudFormation Series byEvanBrown, AWS
![Page 24: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/24.jpg)
DynamoDB
• AWS NoSQL– Key-value store
• Very fast (<10ms)• Very scalable – Specify throughput
• Not too expensive
Recommendation Cache
![Page 25: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/25.jpg)
Issues
• Not all AWS services are created equal– Data Pipeline– Cloud Search
• Documentation• SDK/Tooling• Python & GIL• Access Keys
![Page 26: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/26.jpg)
Future Directions
• Finish backend• Expand clients (publishers, tool developers)• Actually get more recommenders• Babel 3.0 – simple middleware– Automatically logs & add transaction info to outgoing
requests
![Page 27: building babel - s3-us-west-2.amazonaws.com · Babel Architecture Recommenders EigenFactor Recommends Co-Citation Bibliographic Coupling Metadata Database update.eigenfactor.org Object](https://reader035.fdocuments.net/reader035/viewer/2022070800/5f0259417e708231d403d471/html5/thumbnails/27.jpg)
http://[email protected]