Symfony Live San Francisco 2017 - Symfony @ OpenSky
-
Upload
pablo-godel -
Category
Technology
-
view
388 -
download
9
Transcript of Symfony Live San Francisco 2017 - Symfony @ OpenSky
Symfony @ OpenSkyPablo Godel
Symfony @ OpenSky
What is OpenSky?
Symfony @ OpenSky
What is OpenSky?• Multiple businesses powered by Symfony:
1. opensky.com 2. dotandbo.com 3. 55mulberry.com
Symfony @ OpenSky
What is OpenSky?• Multiple businesses powered by Symfony:
1. opensky.com 2. dotandbo.com 3. 55mulberry.com
• +100 employees
Symfony @ OpenSky
What is OpenSky?• Multiple businesses powered by Symfony:
1. opensky.com 2. dotandbo.com 3. 55mulberry.com
• +100 employees • Offices all around the world in New York,
Portland, Nashville, Miami, Chico, Bucharest and Los Angeles
Symfony @ OpenSky
HISTORY
Symfony @ OpenSky
HISTORY
Symfony @ OpenSky
• Early adopter of Symfony2 in 2010 before 2.0 stable was released
History
Symfony @ OpenSky
• Early adopter of Symfony2 in 2010 before 2.0 stable was released
• Migrated from Magento -> symfony1 -> Symfony2
History
Symfony @ OpenSky
• Early adopter of Symfony2 in 2010 before 2.0 stable was released
• Migrated from Magento -> symfony1 -> Symfony2
• Aspects of Magento cart utilized throughout migration process until completely phased out with proprietary cart/checkout technology
History
Symfony @ OpenSky
THE CURRENT STACK
Symfony @ OpenSky
• Symfony 2.8 • RequireJS + Backbone • PHP 7.1.10 • MySQL 5.6 + MongoDB 3.2 • Redis • Varnish + Nginx • VMWare • Vagrant + VirtualBox for Dev VMs
The Current Stack
Symfony @ OpenSky
• ~400 Controllers • +800 Routes • ~1,900 Services • 285 Commands • 374 Entities • 467 Documents • 808,532 LoC • 3,738 Classes • 156 Interfaces • 46 Traits • +10,000 Unit Tests
The Current Stack
Symfony @ OpenSky
The Current Stack
Symfony @ OpenSky
The Current Stack
Symfony @ OpenSky
CODING WORKFLOW
Symfony @ OpenSky
Symfony at OpenSkyGitHub Enterprise
GitHub Enterprise
Coding workflow
Symfony @ OpenSky
• production branch (master)
Coding workflow
Symfony @ OpenSky
• production branch (master) • feature / fix branches
Coding workflow
Symfony @ OpenSky
• production branch (master) • feature / fix branches • Pull Requests linked to JIRA
Coding workflow
Symfony @ OpenSky
• production branch (master) • feature / fix branches • Pull Requests linked to JIRA • Integration with Jenkins for Unit/
Functional Tests
Coding workflow
Symfony @ OpenSky
• production branch (master) • feature / fix branches • Pull Requests linked to JIRA • Integration with Jenkins for Unit/
Functional Tests • Pre-commit Hooks for php-cs-fixer,
gulp-jshint, gulp-sass-lint
Coding workflow
Symfony @ OpenSky
• Symfony Coding Standards
Coding workflow
Symfony @ OpenSky
• Symfony Coding Standards • Short Array Syntax
Coding workflow
Symfony @ OpenSky
• Symfony Coding Standards • Short Array Syntax • Ordered Use Statements
Coding workflow
Symfony @ OpenSky
• Symfony Coding Standards • Short Array Syntax • Ordered Use Statements • Ensuring an EOL at end of file
Coding workflow
Symfony @ OpenSky
• PHP7 Type-Hints when possible
Coding workflow
Symfony @ OpenSky
• PHP7 Type-Hints when possible • DocBlocks only when adds clarity
Coding workflow
Symfony @ OpenSky
• PHP7 Type-Hints when possible • DocBlocks only when adds clarity • ClassName::class
Coding workflow
Symfony @ OpenSky
• PHP7 Type-Hints when possible • DocBlocks only when adds clarity • ClassName::class
• ‘\Namespace\ClassName’
Coding workflow
Symfony @ OpenSky
• PHP7 Type-Hints when possible • DocBlocks only when adds clarity • ClassName::class
• ‘\Namespace\ClassName’ • In forms use class constant
Coding workflow
Symfony @ OpenSky
• PHP7 Type-Hints when possible • DocBlocks only when adds clarity • ClassName::class
• ‘\Namespace\ClassName’ • In forms use class constant
• $builder->add('name', TextType::class);
Coding workflow
Symfony @ OpenSky
• PHP7 Type-Hints when possible • DocBlocks only when adds clarity • ClassName::class
• ‘\Namespace\ClassName’ • In forms use class constant
• $builder->add('name', TextType::class); • Always use ‘use’ statement
• $time = new \DateTime()
Coding workflow
Symfony @ OpenSky
Before production deployment:
Coding workflow
Symfony @ OpenSky
Before production deployment: • Peer Reviews using GitHub PRs
Coding workflow
Symfony @ OpenSky
Before production deployment: • Peer Reviews using GitHub PRs • 100% test pass (unit & functional)
Coding workflow
Symfony @ OpenSky
Before production deployment: • Peer Reviews using GitHub PRs • 100% test pass (unit & functional) • Integration deploy/testing
Coding workflow
Symfony @ OpenSky
Before production deployment: • Peer Reviews using GitHub PRs • 100% test pass (unit & functional) • Integration deploy/testing • UA / QA pass (tracked in JIRA)
Coding workflow
Symfony @ OpenSky
SYMFONY BUNDLES
Symfony @ OpenSky
• Split things into broad bundles. Don’t over bundle.
Symfony Bundles
Symfony @ OpenSky
• Split things into broad bundles. Don’t over bundle. • MainBundle
Symfony Bundles
Symfony @ OpenSky
• Split things into broad bundles. Don’t over bundle. • MainBundle • AdminBundle (500+ routes, 300 extra
services in the container)
Symfony Bundles
Symfony @ OpenSky
• Split things into broad bundles. Don’t over bundle. • MainBundle • AdminBundle (500+ routes, 300 extra
services in the container) • APIBundle
Symfony Bundles
Symfony @ OpenSky
• Split things into broad bundles. Don’t over bundle. • MainBundle • AdminBundle (500+ routes, 300 extra
services in the container) • APIBundle
• 3rd Party Bundles
Symfony Bundles
Symfony @ OpenSky
• Split things into broad bundles. Don’t over bundle. • MainBundle • AdminBundle (500+ routes, 300 extra
services in the container) • APIBundle
• 3rd Party Bundles • AvalancheImagineBundle
Symfony Bundles
Symfony @ OpenSky
• Split things into broad bundles. Don’t over bundle. • MainBundle • AdminBundle (500+ routes, 300 extra
services in the container) • APIBundle
• 3rd Party Bundles • AvalancheImagineBundle • FOSUserBundle
Symfony Bundles
Symfony @ OpenSky
• Split things into broad bundles. Don’t over bundle. • MainBundle • AdminBundle (500+ routes, 300 extra
services in the container) • APIBundle
• 3rd Party Bundles • AvalancheImagineBundle • FOSUserBundle • Using 3rd party bundles may impact
maintainability in the long term
Symfony Bundles
Symfony @ OpenSky
DOCTRINE
Symfony @ OpenSky
• MongoDB and MySQL together
Doctrine
Symfony @ OpenSky
• MongoDB and MySQL together • References between MongoDB and MySQL
Doctrine
Symfony @ OpenSky
• MongoDB and MySQL together • References between MongoDB and MySQL • Using the Doctrine ORM & MongoDB ODM to
make the models for the two databases transparent to the application.
Doctrine
Symfony @ OpenSky
• MongoDB and MySQL together • References between MongoDB and MySQL • Using the Doctrine ORM & MongoDB ODM to
make the models for the two databases transparent to the application.
• Never delete data due to referential integrity problems. Use soft delete (deletedAt)
Doctrine
Symfony @ OpenSky
• MongoDB and MySQL together • References between MongoDB and MySQL • Using the Doctrine ORM & MongoDB ODM to
make the models for the two databases transparent to the application.
• Never delete data due to referential integrity problems. Use soft delete (deletedAt)
• Traits for common functionality like createdAt/updatedAt/deletedAt
Doctrine
Symfony @ OpenSky
Symfony @ OpenSky
• How long did it take to run?
Doctrine Migrations
Symfony @ OpenSky
• How long did it take to run? • Is it backwards compatible?
Doctrine Migrations
Symfony @ OpenSky
• How long did it take to run? • Is it backwards compatible? • Up & Down
Doctrine Migrations
Symfony @ OpenSky
• How long did it take to run? • Is it backwards compatible? • Up & Down • schema.sql / data.sql up-to-date
Doctrine Migrations
Symfony @ OpenSky
• How long did it take to run? • Is it backwards compatible? • Up & Down • schema.sql / data.sql up-to-date • Add indexes
Doctrine Migrations
Symfony @ OpenSky
• How long did it take to run? • Is it backwards compatible? • Up & Down • schema.sql / data.sql up-to-date • Add indexes • Remove data later in separate
migration
Doctrine Migrations
Symfony @ OpenSky
SYMFONY MISC
Symfony @ OpenSky
• Lean
Controllers
Symfony @ OpenSky
• Lean • No Business Logic
Controllers
Symfony @ OpenSky
• Lean • No Business Logic • Only Flow control
Controllers
Symfony @ OpenSky
• Lean • No Business Logic • Only Flow control • Extend Custom BaseController
Controllers
Symfony @ OpenSky
• Two tiers of authentication implemented using role voters.
Security
Symfony @ OpenSky
• Two tiers of authentication implemented using role voters. • PartiallyAuthenticatedVoter
Security
Symfony @ OpenSky
• Two tiers of authentication implemented using role voters. • PartiallyAuthenticatedVoter • AuthenticatedRecentlyVoter
Security
Symfony @ OpenSky
• Two tiers of authentication implemented using role voters. • PartiallyAuthenticatedVoter • AuthenticatedRecentlyVoter
• When a user is recently authenticated, they are logged in to 2nd tier of authentication and have access to account, checkout, etc. After 15 minutes of inactivity, user is logged out of 2nd tier.
Security
Symfony @ OpenSky
• Two tiers of authentication implemented using role voters. • PartiallyAuthenticatedVoter • AuthenticatedRecentlyVoter
• When a user is recently authenticated, they are logged in to 2nd tier of authentication and have access to account, checkout, etc. After 15 minutes of inactivity, user is logged out of 2nd tier.
• Admin integrates with LDAP (using zendframework/zend-ldap)
Security
Symfony @ OpenSky
• Two tiers of authentication implemented using role voters. • PartiallyAuthenticatedVoter • AuthenticatedRecentlyVoter
• When a user is recently authenticated, they are logged in to 2nd tier of authentication and have access to account, checkout, etc. After 15 minutes of inactivity, user is logged out of 2nd tier.
• Admin integrates with LDAP (using zendframework/zend-ldap)
• Multiple firewalls/authentication providers for consumer facing, admin and api.
Security
Symfony @ OpenSky
Runtime Settings/Parameters• Parameters defined in config/parameters.yml
Symfony @ OpenSky
• Parameters defined in config/parameters.yml • Override stored in MongoDB
Runtime Settings/Parameters
Symfony @ OpenSky
• Parameters defined in config/parameters.yml • Override stored in MongoDB • Parameters can have start & end datetime
Runtime Settings/Parameters
Symfony @ OpenSky
• Parameters defined in config/parameters.yml • Override stored in MongoDB • Parameters can have start & end datetime • All parameters stored in cache
Runtime Settings/Parameters
Symfony @ OpenSky
• Parameters defined in config/parameters.yml • Override stored in MongoDB • Parameters can have start & end datetime • All parameters stored in cache • When parameter value is changed, cache is
updated and queue workers restarted
Runtime Settings/Parameters
Symfony @ OpenSky
• Parameters defined in config/parameters.yml • Override stored in MongoDB • Parameters can have start & end datetime • All parameters stored in cache • When parameter value is changed, cache is
updated and queue workers restarted • Allows for configuration changes without
deployment
Runtime Settings/Parameters
Symfony @ OpenSky
• Parameters defined in config/parameters.yml • Override stored in MongoDB • Parameters can have start & end datetime • All parameters stored in cache • When parameter value is changed, cache is
updated and queue workers restarted • Allows for configuration changes without
deployment • Heavily used for feature flags and enable/
disable of new features
Runtime Settings/Parameters
Symfony @ OpenSky
Runtime Settings/Parameters
Symfony @ OpenSky
Runtime Settings/Parameters
Symfony @ OpenSky
Runtime Settings/Parameters
Symfony @ OpenSky
EVENTS
Symfony @ OpenSky
• Perform tasks after the response has been sent to client
Event kernel.terminate
Symfony @ OpenSky
• Perform tasks after the response has been sent to client • Store Request Log in MongoDB Collection
Event kernel.terminate
Symfony @ OpenSky
• Perform tasks after the response has been sent to client • Store Request Log in MongoDB Collection • Analyze request for further actions
Event kernel.terminate
Symfony @ OpenSky
• Perform tasks after the response has been sent to client • Store Request Log in MongoDB Collection • Analyze request for further actions
• Only effective with FastCGI
Event kernel.terminate
Symfony @ OpenSky
• Queueing events for async functionality
Events system
Symfony @ OpenSky
• Queueing events for async functionality • Event serializer/unserializer
Events system
Symfony @ OpenSky
• Queueing events for async functionality • Event serializer/unserializer • Scheduling events
Events system
Symfony @ OpenSky
• Queueing events for async functionality • Event serializer/unserializer • Scheduling events • Event retries, retry max amount
Events system
Symfony @ OpenSky
• Queueing events for async functionality • Event serializer/unserializer • Scheduling events • Event retries, retry max amount • One queue per mongodb collection
Events system
Symfony @ OpenSky
• Queueing events for async functionality • Event serializer/unserializer • Scheduling events • Event retries, retry max amount • One queue per mongodb collection • Queue workers managed with Supervisord
Events system
Symfony @ OpenSky
Events system
Symfony @ OpenSky
Events system
Symfony @ OpenSky
Events system
Symfony @ OpenSky
Events system
Symfony @ OpenSky
Events system
Symfony @ OpenSky
SLACK
Symfony @ OpenSky
• php-kip using yabot
Slack integration
Symfony @ OpenSky
• php-kip using yabot • Production deploy queue/line
Slack integration
Symfony @ OpenSky
• php-kip using yabot • Production deploy queue/line • Dev environment reservations & deploys
Slack integration
Symfony @ OpenSky
Symfony at OpenSkyPhp-kip using yubot
- Production deploy queue/line - Dev environment deploys
Slack integration
Symfony @ OpenSky
Symfony at OpenSkyPhp-kip using yubot
- Production deploy queue/line - Dev environment deploys
Slack integration
Symfony @ OpenSky
Symfony at OpenSkyPhp-kip using yubot
- Production deploy queue/line - Dev environment deploys
Slack integration
Symfony @ OpenSky
Slack integration
Symfony @ OpenSky
Slack integration
Symfony @ OpenSky
Slack integration
Symfony @ OpenSky
SEARCH
Symfony @ OpenSky
Search
• Integration with Algolia
Symfony @ OpenSky
• Integration with Algolia • algolia/algoliasearch-client-php
Search
Symfony @ OpenSky
• Integration with Algolia • algolia/algoliasearch-client-php • Update index through async Event
Listeners
Search
Symfony @ OpenSky
Payments• Integration with Braintree Payments • braintree/braintree_php
Symfony @ OpenSky
• NPM • jQuery • RequireJS • Backbone • TwigJS • Gulp / Less / Sass
The Frontend
Symfony @ OpenSky
• A/B Split Testing using Optimizely for testing UI/UX changes
• Send small % of traffic to new features • GoogleAnalytics tracking conversion rate
The Frontend
Symfony @ OpenSky
TESTING
Symfony @ OpenSky
• pr-nightmareTesting
Symfony @ OpenSky
• pr-nightmare • Runs on all commits and pull requests
Testing
Symfony @ OpenSky
• pr-nightmare • Runs on all commits and pull requests • Uses PHPChunkit to run 30 minutes of tests in 3
minutes by running groups of tests in parallel across multiple servers. https://github.com/jwage/phpchunkit
Testing
Symfony @ OpenSky
• pr-nightmare • Runs on all commits and pull requests • Uses PHPChunkit to run 30 minutes of tests in 3
minutes by running groups of tests in parallel across multiple servers. https://github.com/jwage/phpchunkit
• Selenium for browser functional testing
Testing
Symfony @ OpenSky
Testing
Symfony @ OpenSky
• ‘@app admin’ annotation to indicate web test cases which app/kernel to boot up
Testing with PHPUnit
Symfony @ OpenSky
• ‘@app admin’ annotation to indicate web test cases which app/kernel to boot up
• AutoDataCleanupListener keeps tracks of what collections and tables were inserted to in a test and cleans up afterwards. This used to be explicitly required by the developer but happens automatically now
Testing with PHPUnit
Symfony @ OpenSky
• ‘@app admin’ annotation to indicate web test cases which app/kernel to boot up
• AutoDataCleanupListener keeps tracks of what collections and tables were inserted to in a test and cleans up afterwards. This used to be explicitly required by the developer but happens automatically now
• AbstractTwigLintTest: Twig linting. Loads up all twig files to make sure there are no parse errors
Testing with PHPUnit
Symfony @ OpenSky
• ‘@app admin’ annotation to indicate web test cases which app/kernel to boot up
• AutoDataCleanupListener keeps tracks of what collections and tables were inserted to in a test and cleans up afterwards. This used to be explicitly required by the developer but happens automatically now
• AbstractTwigLintTest: Twig linting. Loads up all twig files to make sure there are no parse errors
• Generating unit tests from class definitions
Testing with PHPUnit
Symfony @ OpenSky
• ‘@app admin’ annotation to indicate web test cases which app/kernel to boot up
• AutoDataCleanupListener keeps tracks of what collections and tables were inserted to in a test and cleans up afterwards. This used to be explicitly required by the developer but happens automatically now
• AbstractTwigLintTest: Twig linting. Loads up all twig files to make sure there are no parse errors
• Generating unit tests from class definitions • PHPUnit watch command
Testing with PHPUnit
Symfony @ OpenSky
Testing with PHPUnit
Symfony @ OpenSky
Testing with PHPUnit
Symfony @ OpenSky
PHPUnit watch
Symfony @ OpenSky
PHPUnit watch
Symfony @ OpenSky
PHPUnit watch
Symfony @ OpenSky
PHPUnit watch
Symfony @ OpenSky
Twig Lint with PHPUnit
Symfony @ OpenSky
Twig Lint with PHPUnit
Symfony @ OpenSky
Twig Lint with PHPUnit
Symfony @ OpenSky
DEPLOYMENT
Symfony @ OpenSky
Deployment• Use Customized Fabric • Manually started from Jenkins • First build task
• Merge feature/fix branch into master
• Create Tag
Symfony @ OpenSky
• Final build task • git checkout • composer install --no-dev —no-scripts • Build front controller • Run gulp, requires, less, sass • Warmup cache for each app • Assetic dump • Build artifacts file & distribute it to
servers • Run MySQL & MongoDB Migrations • Rotate logs • Restart php-fpm • Reload Varnish and nginx • Restart queue workers
Deployment• Use Customized Fabric • Manually started from Jenkins • First build task
• Merge feature/fix branch into master
• Create Tag
Symfony @ OpenSky
Questions?
Symfony @ OpenSky
Thank you!Pablo Godel
@pgodel
http://slideshare.net/pgodel https://joind.in/talk/bfd1e