[CCP Games] Versioning Everything with Perforce
-
Upload
perforce -
Category
Technology
-
view
139 -
download
3
description
Transcript of [CCP Games] Versioning Everything with Perforce
1
Versioning Everything with Perforce
Halldor Fannar CTO CCP
Logo area
2
Facts and figures
Released in 2003 for Windows PC Grown year-on-year to 500,000+ subscribers Record PCU of 63,170 players
Company founded in 1997 Privately held Over 500 employees worldwide
Released 2012 on Sony PS3 (in beta) Free-to-play with microtransactions
3
Connecting Dust and Eve
Play short movie showing DUST & EVE
4
Perforce server is our Definitive Software Library
• Art assets (audio, textures, geometry, shaders, etc) • Built binaries (executables, dynamic libs, static data) • External libraries and tools • Configuration files (settings, ini files, etc) • Text (UI messages, translations, etc)
Branching is used where it makes sense!
5
Versioning art assets
//depot/content/eve //depot/games/branches
model.mb image1.psd image2.psd image3.psd
EVE-‐TRILAMBDA
[file sizes ~100 MB]
model.gr2 image1.dds image2_and_3.dds
[file sizes ~1 MB] settings.red
6
• Ar*sts want technology that is user friendly and provides seamless workflow
• Visual tools p4win & p4v are good but … • Crea*ng/managing client specs is a burden • Selec*ng the right folder/branch to sync is a burden
• Without any support this is how they make it “seamless”: • Create a “god spec” • Sync the root folder
Artists and versioning
7
• The problem isn’t restricted to ar*sts – we have 3rd party tools and libraries that we don’t branch (e.g. //depot/games/sdk)
• Temp*ng to go for a ‘god spec’ in such an environment
• Because syncing from two loca*ons is annoying • Because crea*ng new client specs is annoying
→ We remove those annoying bits.
Shared and branched files
8
Streamlining Perforce user experience
9
Stream-like branching model
MAIN
EVE-‐DX11
EVE-‐TRILAMBDA
EVE-‐JAPAN
EVE-‐TRANQUILITY EVE-‐2012-‐R2
EVE-‐2012-‐R2-‐ALBERTO
EVE-‐2012-‐R2-‐BERYL
sandbox staging release
10
Perforce streams are popular on smaller projects
//eveLauncher
11
Perforce streams are popular on smaller projects
//virtualGoodsService
12
No one size fits all
13
• Treat all your configura*on values as code and version them in source control
• Otherwise you don’t really have a Defini*ve So[ware Library
Versioning your configuration
14
Nobody is born a versioning hero – you become one after a freak software accident
15
• Myth: “Built binaries have no place in source control” • Usual reasoning:
• Because it takes up so much space • Because your build process should be perfectly repeatable and you
are then storing redundant informa*on
• Perforce is our Defini*ve So[ware Library (DSL) for development – don’t spread your development so[ware across Perforce, file share folders, [p sites, and a bunch of DVDs
Versioning built binaries
16
Example: Python packages
• Checked into //pythonPackages • Automagically appear on http://pypi • Bi-winning:
• Makes the desired behavior convenient • Offers two deployment models
17
Example: External libraries
• Make Perforce your one stop shop for SDKs • Applying that policy to the reference PNG library, libpng:
Transform with a branch spec: sdk/libpng<-‐sdk_installs/libpng
18
Example: External library requiring more tricks
• Applying our policy to Microso[’s DirectX is more challenging • Crea*ng the “easy to consume” compile *me environment is easy
19
Example: External library requiring more tricks
• Crea*ng the right run*me environment is the tricky part (system lib) • O[en developers will ask you to manually do this:
20
Example: External library requiring more tricks
• Create a bootstrapping sequence for automa*c silent installa*on:
Import failure
Import library that depends on run*me
• Store the silent installer in Perforce for each version of the SDK • Developers can just sync and run! (same flow for our customers)
no
Execute silent installer for run*me
yes
21
Yes, it’s extra work but every developer will thank you Actually, they won’t because they won’t notice a thing
22
• “Sta*c data” covers all game related data that doesn’t change during play and isn’t captured in industry standard formats
• Typically this data describes the construc*on of an object or a game level by referring to external binary files and providing specific sekngs for how they are combined
Versioning static data
23
• 2002: Sta*c data is stored unversioned in an MS SQL DB • 2003-‐2011: Explosive growth and explosive issues with this approach • 2012: Versioning of sta*c data moves over to Perforce
Abridged history of static data at CCP
24
• Syntax for code well understood and helps us with resolving conflicts and merging
• Compiler can most o[en tell us when we botched the resolve or merge • File granularity for code projects is usually reasonable and known
upfront, thus reducing chances of conflic*ng changes • Organiza*on within a code file follows rules and conven*ons that also
reduce chances of conflicts
How does this map to sta9c data?
Code versus Static data
25
• It doesn’t map! • Using a typical text merging tool on XML-‐like data will most o[en end in
tears
Code versus Static data
26
products: - sku : BL394D quantity : 4 description : Basketball price : 450.00 - sku : BL4438H quantity : 1 description : Super Hoop price : 2392.00
• We use YAML (less cru[)
Stable ordering of items (sets rather than lists)
Uniquely iden*fiable objects
Making static data merge-able
27
Custom merge and diff tool
28
• Mul*ple smaller files reduce chances of resolving – but inefficient for the game to load and tough on your file system
• Use automated build steps to transform the data from the authoring/merge-‐able format to an op*mal run*me format (also versioned in Perforce)
• Build process also verifies the referen*al integrity of the data across files and validates that each file is not viola*ng its schema
Solving other static data issues
29
What about dynamic data?
• Of course stored in a DB • No branching support so changes must be backwards
compatible • Schema and stored procedures are versioned in
Perforce • CI servers monitor all changes and can verify
backwards compatibility
30
Catch the problem at the source - not downstream
31
Versioning everything requires an investment It’s worth it.