(GAM407) Quiplash: Multiscreen, Multiplayer Game for 10,000
-
Upload
amazon-web-services -
Category
Technology
-
view
1.081 -
download
10
Transcript of (GAM407) Quiplash: Multiscreen, Multiplayer Game for 10,000
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Mike Bilder & Evan Jacover, Jackbox Games
Peter Heinrich, Amazon
October 2015
GAM407
Quiplash: The Multiscreen, Multiplayer
Game for 3 to 10,000 Players
The New Fan Base
Foosball
Everyone loves
Popular worldwide, BUT
- Who’s your favorite player?
- Do you have a jersey? A hat?
- Do you watch on TV or radio?
It’s a pastime for players,
not fans
A fan base
Compare to a sport with
More Fun for the Players, PLUS
Diverse Revenue Opportunities
• Merchandise
• Broadcasting
• Events
Lower Costs
• User acquisition
• Content creation
It's just the start
Minimum viable product
What takes your game
to the next level?
How do you reach the
“new fan base”?
User
acquisition
Monetization
loop
Game
loop
Early adopters
The new fan base requires
Influential
Show the fun
Start the conversation
Influencers
Monetization
loop
User
acquisitionGame
loop
Influencer types
Content creators
The new fan base includes
Greater variety
Longer lasting
Content
creation
Influencers
Monetization
loop
User
acquisitionGame
loop
Content types
• Entertainment
• Education
• eSports
• Levels and add-ons
• Fan art, fiction
Alternative revenue
The new fan base enables
Physical goods
Experiences outside the
game
Alternative
revenue
Content
creation
Influencers
Monetization
loop
User
acquisitionGame
LOOP
Additional revenue and
builds your fan base
External engagement
A major benefit of generating
revenue outside of your app is the
opportunity to expand your fan
base
Alternative
revenue
Content
creation
Influencers
Monetization
loop
Support the influencers
Encourage content
creation
Create revenue
opportunities outside
the game
Game
loop
User
acquisition
A Brief History of Jackbox Games
Walter Schirra snuck it on the first
Gemini space mission in 1965
STREAMERS!
Our first game with an
audience feature!
Technology
Technology
• Controllers
• Server architecture
• Tuning and optimization
• Audience
• Scaling
Technology: Controllers
JackPad
JackPad problems
JackPad solutions
Technology: Servers
Architecture
Game host
Console application
instance
Architecture
Game Host
Console Application
Game Controller
Mobile-optimized
Website
Architecture
Game host
Console application
Game controller
Mobile-optimized
website
SetRoomBlob({json})SetCustomerBlob(userid, {json})SendMessageToRoomOwner({json})
Architecture
twizzler hoho churro
ABCDWDJOEVAN
AAAAQWOP
MMQEZOZB
RESTful
PORT 80
PORT 443
Socket.io
PORT 38202
HTTPS/WSS PORT 38203
Each Blobcast instance runs two servers
twizzler hoho churro
ABCDWDJOEVAN
AAAAQWOP
MMQEZOZB
blobcast.jackboxgames.com
Sort Value Score
1 hoho-blobcast-production.jackboxgames.com 308
2 twizzler-blobcast-production.jackboxgames.com 344
3 churro-blobcast-production.jackboxgames.com 391
Technology: Tuning and Optimizing
Choosing instances
• How many games and players can run on a single
instance?
• What is the most efficient instance type to use for our
application? (CPU bound? I/O bound?)
• Are four large servers more cost-efficient than one XXL
server?
Linux tuning
# /etc/sysctl.confnet.core.rmem_max = 67108864 net.core.wmem_max = 67108864 net.ipv4.tcp_rmem = 4096 87380 33554432net.ipv4.tcp_wmem = 4096 65536 33554432net.core.netdev_max_backlog = 30000
...
https://fasterdata.es.net/host-tuning/linux/
Real world simulation
1. Write a script that does exactly what a player does
a. Look for specific game states
b. Respond with appropriate messages
2. Run that script thousands of times
3. Monitor the servers
Bees with machine guns
“...[The Bees] are, more-or-less a distributed denial-of-
service attack in a fancy package…”
https://github.com/newsapps/beeswithmachineguns
https://github.com/jugglinmike/beeswithmachineguns (fork for shell commands)
./bees up -s 10 -g sec-group -k keypair --instance ami-xxxxxxx
./bees exec - "git pull && npm install“
./bees exec – "node server.js play quiplash -p 1000 -r QWOP"
Tuning: Instance types
0
20
40
60
80
100
120
1000 2000 3000 4000 5000 6000 7000
Worst case CPU
c3.large c4.large m4.large r3.large c4.xlarge
Tuning: Instance types
0
20
40
60
80
100
120
1000 2000 3000 4000 5000 6000 7000
Worst case CPU
c3.large c4.large c4.xlarge
Technology: Audience
Big audiences
• Big streamers get upwards of 30,000 live viewers
• We want to be able to have more people participating in
a game than a single server can handle
twizzler hoho churro
ABCD(Host)
ABCD ABCD
PUBLISHchan:room:ABCD
SUBSCRIBEchan:room:ABCD
Multinode rooms: Redis PUB/SUB
twizzler hoho churro
ABCD(Host)
ABCD ABCD
RPUSHchan:hoho:ABCDchan:churro:ABCD
100ms Loop
LPOPchan:hoho:ABCD
100ms Loop
LPOPchan:churro:ABCD
Multinode rooms: Redis queue polling
twizzler hoho churro
ABCD(Host)
ABCD ABCD
100 ms Loop
LPOPchan:ABCD
RPUSHchan:ABCD
RPUSHchan:ABCD
Multinode rooms: Redis queue polling
Big audiences
• The vast majority of our games are still eight or fewer
players
• Don’t add latency for regular players• Wait for over 100 players to join a room before they can join on other
servers
• No single bottleneck
twizzler hoho churro
ABCD(Host)
ABCD ABCD
100 ms Loop
LPOPchan:ABCD
RPUSHchan:ABCD
RPUSHchan:ABCD
Multinode rooms: No single bottleneck
Efficient voting
Create a new vote “session”• Redis set with a list of user IDs that have voted
• Redis hash with a list of voting options
key count
Cropdust my heart 0
Smells like love 0
vote:QWOP:useridsSET
vote:QWOP:votesHASH
Efficient voting
Create a new vote “session”• Redis set with a list of user IDs that have voted
• Redis hash with a list of voting options
Validate new voters• Check for user ID SISMEMBER O(1)
key count
Cropdust my heart 0
Smells like love 0
vote:QWOP:useridsSET
vote:QWOP:votesHASH
Efficient voting
Create a new vote “session”• Redis set with a list of user IDs that have voted
• Redis hash with a list of voting options
Validate new voters• Check for user ID SISMEMBER O(1)
Accept vote• Add user ID to voters list SADD O(1)
• Increment vote HINCRBY O(1)
56cdb7d5-9c7f-4723-a562-b07659e04e40
key count
Cropdust my heart 1
Smells like love 0
vote:QWOP:useridsSET
vote:QWOP:votesHASH
Efficient voting
Create a new vote “session”• Redis set with a list of user IDs that have voted
• Redis hash with a list of voting options
Validate new voters• Check for user ID SISMEMBER O(1)
Accept vote• Add user ID to voters list SADD O(1)
• Increment vote HINCRBY O(1)
Query vote • Get all elements of vote hash HGETALL O(N)
56cdb7d5-9c7f-4723-a562-b07659e04e40
key count
Cropdust my heart 1
Smells like love 0
vote:QWOP:useridsSET
vote:QWOP:votesHASH
Technology: Scaling
One-Click Scaling
AWS OpsWorks
Why not use OpsWorks autoscaling?
Blobcast autoscaling Cron job running every minute
Scale audience and games separately
• Keep 10,000 connection
capacity available
• Scaling can proceed slower
as needed
Audience Games
What’s next
• Leveraging regions for geographic speed
• Multicore support (Cluster Node.js)
• Use AWS Lambda for autoscaling
• Better analytics
We love Fire TV
Why Fire TV is our development platform
• They’re cheap
• They’re portable
• Tooling is easy
• It has a USB port
• We’re developing for it anyway
It’s a “dream product”
It’s been successful
All 4 of our products are in the Top
Grossing Games list
Twitch TV
Making a game for streamers
• Our marketing budget: $0
• Every stream is an ad for our game
• Building online multiplayer is hard, leveraging Twitch is
easy
• Chat is a secondary way to play
• It’s a new way of interacting with games
Add additional ways for the audience to
interact beyond voting
• Audience scoring and scoreboards
• Promote audience members to players
• Track and manipulate statistical information
• Audience member get their name on screen
Provide more tools for the host to
manage the game
• The ability to boot/ban agitators
• An option to filter terrible, racist text
• Private codes to join the game
Technical improvements
• More than 10,000 people
• Reduce latency
Crazy stuff
• What if you could instantly
get a T-shirt of a drawing
from Drawful?
Burger King’s Skeleton drawing by @chilliards
T-Shirt mock-up via Custom Ink
Next Steps
Thank you!For more information, connect with us:
http://jackboxgames.com
http://bit.ly/AmznAppDev
@PeterDotGames@JackboxGames
Drawing winners announced:
http://bit.ly/appstoregiveaway
Remember to complete
your evaluations!