33degree Krzysztof Debski - Let's build a solid base for a scale
-
Upload
krzysztof-debski -
Category
Technology
-
view
564 -
download
1
Transcript of 33degree Krzysztof Debski - Let's build a solid base for a scale
@DebskiChris
#33rdDegree
Krzysztof Dębski Allegro Group
Let’s build a solid base for a scale
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Agenda
The Allegro Situation How to build a new World? The way to improve What’s in it for you?
Allegro The New World Improvement WIIFY #33rdDegree @DebskiChris
The Allegro Situation
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Allegro
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Allegro
6 million LOC
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
400 people in IT
Allegro
Allegro The New World Improvement WIIFY #33rdDegree @DebskiChris
A New Hope
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
A New Hope
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
A New Hope
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
A New Hope?
Service
Oriented
Ambiguity
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Domain Driven Design
Allegro The New World Improvement WIIFY #33rdDegree @DebskiChris
(Micro)Services
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
(Micro)Services
Business needs
User
Transaction
Offer
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
(Micro)Services
Independent
User
Transaction
Offer
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
(Micro)Services
API
User
Transaction
Offer
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
(Micro)Services
Polyglot
Cassandra
MongoDB
Oracle
User
Transaction
Offer
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
(Micro)Services
Smart Endpoints
Cassandra
MongoDB
Oracle
Offer
Transaction
User
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
The first approach
Service
Auto deployable Auto monitored Auto scalable Auto healable Auto discoverable … and many more features
Allegro The New World Improvement WIIFY #33rdDegree @DebskiChris
How to build a new World?
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
The first project
Service
Jetty Server
War file
Spring
Gradle
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Let’s REST
JAX-RS / JSR Compliant @Path("/users") @Consumes(CONTENT_TYPE_JSON) @Produces(CONTENT_TYPE_JSON) public class UsersEndpoint {
// [...] @GET public UserCollectionResponse findAllUsers() { //[...] }
}
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Let’s expose our methods
Swagger @Path("/users") @Api(value = "/users") @Consumes(CONTENT_TYPE_JSON) @Produces(CONTENT_TYPE_JSON) public class UsersEndpoint {
// [...] @ApiOperation(value=“Get all users”, response=UserCollectionResponse.class) @GET public UserCollectionResponse findAllUsers() { //[...] }
}
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Let’s expose our methods
Swagger @Path("/users") @Api(value = "/users") @Consumes(CONTENT_TYPE_JSON) @Produces(CONTENT_TYPE_JSON) public class UsersEndpoint {
// [...] @ApiOperation(value=“Get all users”, response=UserCollectionResponse.class) @GET public UserCollectionResponse findAllUsers() { //[...] }
}
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Let’s expose our methods
Swagger
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
And make them discoverable
Offer
User
Discovery ZooKeeper
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
And make them discoverable
Offer
User
Discovery ZooKeeper
Register
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
And make them discoverable
Offer
User
Discovery ZooKeeper
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
And make them discoverable
Offer
User
Discovery ZooKeeper
Get User
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
And make them discoverable
Offer
User
Discovery ZooKeeper
Get User
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
And make them discoverable
Offer
User
Discovery ZooKeeper
Get User Get offer for user
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Multiple API versions
Content negotiation @Path("/users") @Consumes(”application/json”) @Produces(”application/json”) public class UsersEndpoint {
// [...] @GET public UserCollectionResponse findAllUsers() { //[...] }
}
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Multiple API versions
Content negotiation @Path("/users") @Consumes(”application/vnd.allegro.user.v1+json”) @Produces(”application/vnd.allegro.user.v1+json”) public class UsersEndpoint {
// [...] @GET public UserCollectionResponse findAllUsers() { //[...] }
}
curl --dump-header - -H ”Accept: application/vnd.allegro.user.v1+json" -X GET http://localhost:8080/users
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Multiple API versions
Content negotiation @Path("/users") @Consumes(”application/vnd.allegro.user.v1+json”) @Produces(”application/vnd.allegro.user.v1+json”) public class UsersEndpoint {
// [...] @GET @Produces(“application/vnd.allegro.user.v2+json”) public UserCollectionResponse findAllUsers() { //[...] }
}
curl --dump-header - -H ”Accept: application/vnd.allegro.user.v2+json" -X GET http://localhost:8080/users
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
And it became slow…
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Lower startup time
External Jetty
Service
Jetty Server
War file
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Lower startup time
Embedded Jetty
Service
Jetty Server
Jar file
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Lower startup time
Embedded Jetty
Service
Jetty Server
Jar file
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Lower startup time
Embedded UnderTow
Service
Jetty Server Undertow
Jar file
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Lower startup time
Immutable Images
Service
Undertow
Zip file
Server
Immutable image
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
How to configure it?
Service
java –jar service.jar
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
How to configure it?
Service ZooKeeper
java –jar service.jar –Dzookeeper.address=zookeper.service
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
How to configure it?
Service ZooKeeper
java –jar service.jar –Dzookeeper.address=zookeper.service –Denvironment=prod
prod.properties
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
How to configure it?
Service ZooKeeper
java –jar service.jar –Dzookeeper.address=zookeper.service –Denvironment=prod -Dservice.name=testService
prod.properties
Allegro The New World Improvement WIIFY #33rdDegree @DebskiChris
Tests
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Do you test your tests?
Pitest
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Integration tests
Test without mocks Run from IDE
public class UsersIntegrationTest {
private static Map<String, String> overrideConfiguration = Maps.newHashMap();
static { overrideConfiguration.put("property.name", "This was overwritten value"); }
@ClassRule public static final RestServiceStarted DEPLOYED_SERVICE = new RestServiceStarted(overrideConfiguration);
private WebTarget getUsersResourceWebTarget() { return DEPLOYED_SERVICE.getWebTarget().path("users"); }
}
Allegro The New World Improvement WIIFY #33rdDegree @DebskiChris
Monitoring
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Let’s see what’s happening
NxLog LogStash Kibana
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Let’s see what’s happening
Graphite # Metrics metrics.reporters.graphite.enabled=true metrics.reporters.graphite.host=graphite.service metrics.reporters.graphite.port=2003 metrics.reporters.graphite.prefix=stats.Prod.service metrics.reporters.interval=30
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Monolith Alert
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Monolith Alert
Multi module project support
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Monolith Alert
Multi module project support
Allegro The New World Improvement WIIFY #33rdDegree @DebskiChris
The way to improve
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Authentication
OAuth2
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Patch Support
Standardized by IETF https://tools.ietf.org/html/rfc6902
PATCH /user/1 HTTP/1.1 Host: user.service.local Content-Length: 312
Content-Type: application/json-patch+json [
{”op”: ”test”, ”path”: ”/firstname”, ”value”: ”Jane”}, {”op”: ”add”, ”path”: ”/maidenname”, ”value”: ”Smith”}, {”op”: ”replace”, ”path”: ”/lastname”, ”value”: ”Doe”},
{”op”: ”remove", ”path”: ”/meetings”}, {”op”: ”move”, ”from”: ”/balance”, ”path”: ”sharedbalance”}, {”op”: ”copy”, ”from”: ”/a”, ”path”: ”/b”}
]
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Simplify annotations
Swagger? @Path("/users") @Api(value = "/users") @Consumes(CONTENT_TYPE_JSON) @Produces(CONTENT_TYPE_JSON) public class UsersEndpoint {
// [...] @ApiOperation(value=“Get all users”, response=UserCollectionResponse.class) @GET public UserCollectionResponse findAllUsers() { //[...] }
}
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Deployment
Docker support
Allegro The New World Improvement WIIFY #33rdDegree @DebskiChris
What’s in it for you?
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Use the latest tools
GradleW in CI ENV and parent POM issues. Don’t do DDOS yourself and your partners.
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Change is the only constant
Deployment tools Service API provider Dependency injection HTTP Server
Cargo Docker
Ecosystem
Swagger Jersey addon
Guice Spring
Jetty Undertow
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Focus on right metrics
How to have 100% code coverage? public class UserMediaType {
public static final String V1_JSON = "application/vnd.allegro.user.v1+json” public static final String V2_JSON = "application/vnd.allegro.user.v2+json”
}
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Focus on right metrics
import org.junit.Test; import static net.trajano… public class MediaTypesTest { @Test public void mediaTypesShouldBeValidUtilityClasses()
throws Throwable { assertUtilityClassWellDefined(UserMediaType.class); } }
You don’t need 100% code coverage!
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Tools also lie
Metrics tend to lie
Configuration: default PHP metrics
Tested file: 4535 CLOC Imperative programming
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Tools also lie
How many violations are there?
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Tools also lie
How many violations are there?
4 At least according to Sonar.
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Learn to REST
REST is not so obvious
/api/bi/XYZ123/1
/recommendations/1/items
/offer/100?output=ESI/JSON
#33rdDegree @DebskiChris Allegro The New World Improvement WIIFY
Community
Involve all developers in building the Bootstrap. Or they will build their own tools.
Łukasz Drumiński Mateusz Gajewski @wendigo
Allegro The New World Improvement WIIFY #33rdDegree @DebskiChris
Q & A