Post on 17-Mar-2020
RedisGearsndashRedisinmemorydataprocessingJUNE 2019 | PIETER CAILLIAU
bull ProducedinBelgium
bull (instanceof)SETomTom
bull Consultantneo4j
bull SolutionArchitectRedisLabs
bull ProductManagerRedisLabs
bull cailliaup
Aboutme
2
1 WhatisRedisandRedisEnterprise
2 StreamProcessingwithRedisGears
3 RedisGearsasaMultimodelEngine
Agenda
4
RedisisFasthellip
5
hellipExtremelyFast
DB-EnginesRanking
6
7
Andyoursquovebeenusingitalready
RedisisExtensivelyandDiverselyUsed
8
UsesRedisforTimelinefollowing
Scope10-20TB
UsesRedisforLocalsiteglobalcaching
UsesRedisforRepositoryrouter
Scope10+TB
UsesRedisforGeosearchuserprofiles
Scope10-20TB
UsesRedisforAllmessages
Scope40TB
RedisTopDifferentiators
Simplicity ExtensibilityPerformanceNoSQLBenchmark
1
RedisDataStructures
2 3
RedisModules
9
Lists
Hashes
Bitmaps
Strings
Bitfield
Streams
Hyperloglog
SortedSets
Sets
GeospatialIndexes
10
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
11
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
bull Createyourowndatatypesandcommands
bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability
bull CanbewritteninCC++GoPythonRustZig
bull Leverageexistingdatastructures
bull TurnRedisintoaMulti-Modeldatabase
ModulesExtendRedisInfinitely
12
httpsredislabscomcommunityredis-modules-hub
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
13
14
Introducing
RedisEnterprise
15
DBaaSbull Availablesincemid2013
bull 8500+enterprisecustomers
Softwarebull Availablesinceearly2015
bull 300+enterprisecustomers550K+databases
managedworldwide
bull 6oftopFortune10companies
bull 3oftop5communicationscompanies
Customersbull 3oftop4creditcardissuers
bull 3oftop5healthcarecompanies
bull CloudProvider
bull Highermarginbyndash Idleness
bull CloudLock-in
CloudProvidershavedifferentincentives
16
bull DBAASProvider
bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization
RedisEnterpriseAUniquePrimaryDatabase
HIGHESTPERFORMANCELINEARSCALING
HIGHAVAILABILITYWITHINSTANTFAILOVER
DURABILITYATMEMORYSPEEDS
ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)
BUILT-INHIGHPERFORMANCE SEARCH
MULTI-MODEL
FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)
INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)
FAST RELIABLE FLEXIBLE
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
18
Unevennumberofsymmetricnodes
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
bull ProducedinBelgium
bull (instanceof)SETomTom
bull Consultantneo4j
bull SolutionArchitectRedisLabs
bull ProductManagerRedisLabs
bull cailliaup
Aboutme
2
1 WhatisRedisandRedisEnterprise
2 StreamProcessingwithRedisGears
3 RedisGearsasaMultimodelEngine
Agenda
4
RedisisFasthellip
5
hellipExtremelyFast
DB-EnginesRanking
6
7
Andyoursquovebeenusingitalready
RedisisExtensivelyandDiverselyUsed
8
UsesRedisforTimelinefollowing
Scope10-20TB
UsesRedisforLocalsiteglobalcaching
UsesRedisforRepositoryrouter
Scope10+TB
UsesRedisforGeosearchuserprofiles
Scope10-20TB
UsesRedisforAllmessages
Scope40TB
RedisTopDifferentiators
Simplicity ExtensibilityPerformanceNoSQLBenchmark
1
RedisDataStructures
2 3
RedisModules
9
Lists
Hashes
Bitmaps
Strings
Bitfield
Streams
Hyperloglog
SortedSets
Sets
GeospatialIndexes
10
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
11
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
bull Createyourowndatatypesandcommands
bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability
bull CanbewritteninCC++GoPythonRustZig
bull Leverageexistingdatastructures
bull TurnRedisintoaMulti-Modeldatabase
ModulesExtendRedisInfinitely
12
httpsredislabscomcommunityredis-modules-hub
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
13
14
Introducing
RedisEnterprise
15
DBaaSbull Availablesincemid2013
bull 8500+enterprisecustomers
Softwarebull Availablesinceearly2015
bull 300+enterprisecustomers550K+databases
managedworldwide
bull 6oftopFortune10companies
bull 3oftop5communicationscompanies
Customersbull 3oftop4creditcardissuers
bull 3oftop5healthcarecompanies
bull CloudProvider
bull Highermarginbyndash Idleness
bull CloudLock-in
CloudProvidershavedifferentincentives
16
bull DBAASProvider
bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization
RedisEnterpriseAUniquePrimaryDatabase
HIGHESTPERFORMANCELINEARSCALING
HIGHAVAILABILITYWITHINSTANTFAILOVER
DURABILITYATMEMORYSPEEDS
ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)
BUILT-INHIGHPERFORMANCE SEARCH
MULTI-MODEL
FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)
INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)
FAST RELIABLE FLEXIBLE
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
18
Unevennumberofsymmetricnodes
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
1 WhatisRedisandRedisEnterprise
2 StreamProcessingwithRedisGears
3 RedisGearsasaMultimodelEngine
Agenda
4
RedisisFasthellip
5
hellipExtremelyFast
DB-EnginesRanking
6
7
Andyoursquovebeenusingitalready
RedisisExtensivelyandDiverselyUsed
8
UsesRedisforTimelinefollowing
Scope10-20TB
UsesRedisforLocalsiteglobalcaching
UsesRedisforRepositoryrouter
Scope10+TB
UsesRedisforGeosearchuserprofiles
Scope10-20TB
UsesRedisforAllmessages
Scope40TB
RedisTopDifferentiators
Simplicity ExtensibilityPerformanceNoSQLBenchmark
1
RedisDataStructures
2 3
RedisModules
9
Lists
Hashes
Bitmaps
Strings
Bitfield
Streams
Hyperloglog
SortedSets
Sets
GeospatialIndexes
10
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
11
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
bull Createyourowndatatypesandcommands
bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability
bull CanbewritteninCC++GoPythonRustZig
bull Leverageexistingdatastructures
bull TurnRedisintoaMulti-Modeldatabase
ModulesExtendRedisInfinitely
12
httpsredislabscomcommunityredis-modules-hub
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
13
14
Introducing
RedisEnterprise
15
DBaaSbull Availablesincemid2013
bull 8500+enterprisecustomers
Softwarebull Availablesinceearly2015
bull 300+enterprisecustomers550K+databases
managedworldwide
bull 6oftopFortune10companies
bull 3oftop5communicationscompanies
Customersbull 3oftop4creditcardissuers
bull 3oftop5healthcarecompanies
bull CloudProvider
bull Highermarginbyndash Idleness
bull CloudLock-in
CloudProvidershavedifferentincentives
16
bull DBAASProvider
bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization
RedisEnterpriseAUniquePrimaryDatabase
HIGHESTPERFORMANCELINEARSCALING
HIGHAVAILABILITYWITHINSTANTFAILOVER
DURABILITYATMEMORYSPEEDS
ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)
BUILT-INHIGHPERFORMANCE SEARCH
MULTI-MODEL
FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)
INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)
FAST RELIABLE FLEXIBLE
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
18
Unevennumberofsymmetricnodes
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
4
RedisisFasthellip
5
hellipExtremelyFast
DB-EnginesRanking
6
7
Andyoursquovebeenusingitalready
RedisisExtensivelyandDiverselyUsed
8
UsesRedisforTimelinefollowing
Scope10-20TB
UsesRedisforLocalsiteglobalcaching
UsesRedisforRepositoryrouter
Scope10+TB
UsesRedisforGeosearchuserprofiles
Scope10-20TB
UsesRedisforAllmessages
Scope40TB
RedisTopDifferentiators
Simplicity ExtensibilityPerformanceNoSQLBenchmark
1
RedisDataStructures
2 3
RedisModules
9
Lists
Hashes
Bitmaps
Strings
Bitfield
Streams
Hyperloglog
SortedSets
Sets
GeospatialIndexes
10
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
11
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
bull Createyourowndatatypesandcommands
bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability
bull CanbewritteninCC++GoPythonRustZig
bull Leverageexistingdatastructures
bull TurnRedisintoaMulti-Modeldatabase
ModulesExtendRedisInfinitely
12
httpsredislabscomcommunityredis-modules-hub
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
13
14
Introducing
RedisEnterprise
15
DBaaSbull Availablesincemid2013
bull 8500+enterprisecustomers
Softwarebull Availablesinceearly2015
bull 300+enterprisecustomers550K+databases
managedworldwide
bull 6oftopFortune10companies
bull 3oftop5communicationscompanies
Customersbull 3oftop4creditcardissuers
bull 3oftop5healthcarecompanies
bull CloudProvider
bull Highermarginbyndash Idleness
bull CloudLock-in
CloudProvidershavedifferentincentives
16
bull DBAASProvider
bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization
RedisEnterpriseAUniquePrimaryDatabase
HIGHESTPERFORMANCELINEARSCALING
HIGHAVAILABILITYWITHINSTANTFAILOVER
DURABILITYATMEMORYSPEEDS
ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)
BUILT-INHIGHPERFORMANCE SEARCH
MULTI-MODEL
FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)
INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)
FAST RELIABLE FLEXIBLE
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
18
Unevennumberofsymmetricnodes
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
5
hellipExtremelyFast
DB-EnginesRanking
6
7
Andyoursquovebeenusingitalready
RedisisExtensivelyandDiverselyUsed
8
UsesRedisforTimelinefollowing
Scope10-20TB
UsesRedisforLocalsiteglobalcaching
UsesRedisforRepositoryrouter
Scope10+TB
UsesRedisforGeosearchuserprofiles
Scope10-20TB
UsesRedisforAllmessages
Scope40TB
RedisTopDifferentiators
Simplicity ExtensibilityPerformanceNoSQLBenchmark
1
RedisDataStructures
2 3
RedisModules
9
Lists
Hashes
Bitmaps
Strings
Bitfield
Streams
Hyperloglog
SortedSets
Sets
GeospatialIndexes
10
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
11
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
bull Createyourowndatatypesandcommands
bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability
bull CanbewritteninCC++GoPythonRustZig
bull Leverageexistingdatastructures
bull TurnRedisintoaMulti-Modeldatabase
ModulesExtendRedisInfinitely
12
httpsredislabscomcommunityredis-modules-hub
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
13
14
Introducing
RedisEnterprise
15
DBaaSbull Availablesincemid2013
bull 8500+enterprisecustomers
Softwarebull Availablesinceearly2015
bull 300+enterprisecustomers550K+databases
managedworldwide
bull 6oftopFortune10companies
bull 3oftop5communicationscompanies
Customersbull 3oftop4creditcardissuers
bull 3oftop5healthcarecompanies
bull CloudProvider
bull Highermarginbyndash Idleness
bull CloudLock-in
CloudProvidershavedifferentincentives
16
bull DBAASProvider
bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization
RedisEnterpriseAUniquePrimaryDatabase
HIGHESTPERFORMANCELINEARSCALING
HIGHAVAILABILITYWITHINSTANTFAILOVER
DURABILITYATMEMORYSPEEDS
ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)
BUILT-INHIGHPERFORMANCE SEARCH
MULTI-MODEL
FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)
INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)
FAST RELIABLE FLEXIBLE
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
18
Unevennumberofsymmetricnodes
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
DB-EnginesRanking
6
7
Andyoursquovebeenusingitalready
RedisisExtensivelyandDiverselyUsed
8
UsesRedisforTimelinefollowing
Scope10-20TB
UsesRedisforLocalsiteglobalcaching
UsesRedisforRepositoryrouter
Scope10+TB
UsesRedisforGeosearchuserprofiles
Scope10-20TB
UsesRedisforAllmessages
Scope40TB
RedisTopDifferentiators
Simplicity ExtensibilityPerformanceNoSQLBenchmark
1
RedisDataStructures
2 3
RedisModules
9
Lists
Hashes
Bitmaps
Strings
Bitfield
Streams
Hyperloglog
SortedSets
Sets
GeospatialIndexes
10
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
11
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
bull Createyourowndatatypesandcommands
bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability
bull CanbewritteninCC++GoPythonRustZig
bull Leverageexistingdatastructures
bull TurnRedisintoaMulti-Modeldatabase
ModulesExtendRedisInfinitely
12
httpsredislabscomcommunityredis-modules-hub
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
13
14
Introducing
RedisEnterprise
15
DBaaSbull Availablesincemid2013
bull 8500+enterprisecustomers
Softwarebull Availablesinceearly2015
bull 300+enterprisecustomers550K+databases
managedworldwide
bull 6oftopFortune10companies
bull 3oftop5communicationscompanies
Customersbull 3oftop4creditcardissuers
bull 3oftop5healthcarecompanies
bull CloudProvider
bull Highermarginbyndash Idleness
bull CloudLock-in
CloudProvidershavedifferentincentives
16
bull DBAASProvider
bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization
RedisEnterpriseAUniquePrimaryDatabase
HIGHESTPERFORMANCELINEARSCALING
HIGHAVAILABILITYWITHINSTANTFAILOVER
DURABILITYATMEMORYSPEEDS
ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)
BUILT-INHIGHPERFORMANCE SEARCH
MULTI-MODEL
FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)
INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)
FAST RELIABLE FLEXIBLE
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
18
Unevennumberofsymmetricnodes
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
7
Andyoursquovebeenusingitalready
RedisisExtensivelyandDiverselyUsed
8
UsesRedisforTimelinefollowing
Scope10-20TB
UsesRedisforLocalsiteglobalcaching
UsesRedisforRepositoryrouter
Scope10+TB
UsesRedisforGeosearchuserprofiles
Scope10-20TB
UsesRedisforAllmessages
Scope40TB
RedisTopDifferentiators
Simplicity ExtensibilityPerformanceNoSQLBenchmark
1
RedisDataStructures
2 3
RedisModules
9
Lists
Hashes
Bitmaps
Strings
Bitfield
Streams
Hyperloglog
SortedSets
Sets
GeospatialIndexes
10
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
11
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
bull Createyourowndatatypesandcommands
bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability
bull CanbewritteninCC++GoPythonRustZig
bull Leverageexistingdatastructures
bull TurnRedisintoaMulti-Modeldatabase
ModulesExtendRedisInfinitely
12
httpsredislabscomcommunityredis-modules-hub
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
13
14
Introducing
RedisEnterprise
15
DBaaSbull Availablesincemid2013
bull 8500+enterprisecustomers
Softwarebull Availablesinceearly2015
bull 300+enterprisecustomers550K+databases
managedworldwide
bull 6oftopFortune10companies
bull 3oftop5communicationscompanies
Customersbull 3oftop4creditcardissuers
bull 3oftop5healthcarecompanies
bull CloudProvider
bull Highermarginbyndash Idleness
bull CloudLock-in
CloudProvidershavedifferentincentives
16
bull DBAASProvider
bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization
RedisEnterpriseAUniquePrimaryDatabase
HIGHESTPERFORMANCELINEARSCALING
HIGHAVAILABILITYWITHINSTANTFAILOVER
DURABILITYATMEMORYSPEEDS
ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)
BUILT-INHIGHPERFORMANCE SEARCH
MULTI-MODEL
FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)
INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)
FAST RELIABLE FLEXIBLE
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
18
Unevennumberofsymmetricnodes
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
RedisisExtensivelyandDiverselyUsed
8
UsesRedisforTimelinefollowing
Scope10-20TB
UsesRedisforLocalsiteglobalcaching
UsesRedisforRepositoryrouter
Scope10+TB
UsesRedisforGeosearchuserprofiles
Scope10-20TB
UsesRedisforAllmessages
Scope40TB
RedisTopDifferentiators
Simplicity ExtensibilityPerformanceNoSQLBenchmark
1
RedisDataStructures
2 3
RedisModules
9
Lists
Hashes
Bitmaps
Strings
Bitfield
Streams
Hyperloglog
SortedSets
Sets
GeospatialIndexes
10
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
11
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
bull Createyourowndatatypesandcommands
bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability
bull CanbewritteninCC++GoPythonRustZig
bull Leverageexistingdatastructures
bull TurnRedisintoaMulti-Modeldatabase
ModulesExtendRedisInfinitely
12
httpsredislabscomcommunityredis-modules-hub
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
13
14
Introducing
RedisEnterprise
15
DBaaSbull Availablesincemid2013
bull 8500+enterprisecustomers
Softwarebull Availablesinceearly2015
bull 300+enterprisecustomers550K+databases
managedworldwide
bull 6oftopFortune10companies
bull 3oftop5communicationscompanies
Customersbull 3oftop4creditcardissuers
bull 3oftop5healthcarecompanies
bull CloudProvider
bull Highermarginbyndash Idleness
bull CloudLock-in
CloudProvidershavedifferentincentives
16
bull DBAASProvider
bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization
RedisEnterpriseAUniquePrimaryDatabase
HIGHESTPERFORMANCELINEARSCALING
HIGHAVAILABILITYWITHINSTANTFAILOVER
DURABILITYATMEMORYSPEEDS
ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)
BUILT-INHIGHPERFORMANCE SEARCH
MULTI-MODEL
FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)
INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)
FAST RELIABLE FLEXIBLE
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
18
Unevennumberofsymmetricnodes
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
RedisTopDifferentiators
Simplicity ExtensibilityPerformanceNoSQLBenchmark
1
RedisDataStructures
2 3
RedisModules
9
Lists
Hashes
Bitmaps
Strings
Bitfield
Streams
Hyperloglog
SortedSets
Sets
GeospatialIndexes
10
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
11
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
bull Createyourowndatatypesandcommands
bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability
bull CanbewritteninCC++GoPythonRustZig
bull Leverageexistingdatastructures
bull TurnRedisintoaMulti-Modeldatabase
ModulesExtendRedisInfinitely
12
httpsredislabscomcommunityredis-modules-hub
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
13
14
Introducing
RedisEnterprise
15
DBaaSbull Availablesincemid2013
bull 8500+enterprisecustomers
Softwarebull Availablesinceearly2015
bull 300+enterprisecustomers550K+databases
managedworldwide
bull 6oftopFortune10companies
bull 3oftop5communicationscompanies
Customersbull 3oftop4creditcardissuers
bull 3oftop5healthcarecompanies
bull CloudProvider
bull Highermarginbyndash Idleness
bull CloudLock-in
CloudProvidershavedifferentincentives
16
bull DBAASProvider
bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization
RedisEnterpriseAUniquePrimaryDatabase
HIGHESTPERFORMANCELINEARSCALING
HIGHAVAILABILITYWITHINSTANTFAILOVER
DURABILITYATMEMORYSPEEDS
ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)
BUILT-INHIGHPERFORMANCE SEARCH
MULTI-MODEL
FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)
INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)
FAST RELIABLE FLEXIBLE
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
18
Unevennumberofsymmetricnodes
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
10
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
11
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
bull Createyourowndatatypesandcommands
bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability
bull CanbewritteninCC++GoPythonRustZig
bull Leverageexistingdatastructures
bull TurnRedisintoaMulti-Modeldatabase
ModulesExtendRedisInfinitely
12
httpsredislabscomcommunityredis-modules-hub
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
13
14
Introducing
RedisEnterprise
15
DBaaSbull Availablesincemid2013
bull 8500+enterprisecustomers
Softwarebull Availablesinceearly2015
bull 300+enterprisecustomers550K+databases
managedworldwide
bull 6oftopFortune10companies
bull 3oftop5communicationscompanies
Customersbull 3oftop4creditcardissuers
bull 3oftop5healthcarecompanies
bull CloudProvider
bull Highermarginbyndash Idleness
bull CloudLock-in
CloudProvidershavedifferentincentives
16
bull DBAASProvider
bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization
RedisEnterpriseAUniquePrimaryDatabase
HIGHESTPERFORMANCELINEARSCALING
HIGHAVAILABILITYWITHINSTANTFAILOVER
DURABILITYATMEMORYSPEEDS
ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)
BUILT-INHIGHPERFORMANCE SEARCH
MULTI-MODEL
FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)
INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)
FAST RELIABLE FLEXIBLE
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
18
Unevennumberofsymmetricnodes
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
11
WritteninC
Servedentirelyfrommemory
Single-threadedlockfree
MostcommandsareexecutedwithO(1)
complexity
Accesstodiscreteelements
withinobjects
Reducedbandwidthoverheadrequirements
Easytoparsenetworkingprotocol
Pipeliningforreduced
networkoverhead
Connectionpooling
OPTIMIZEDARCHITECTURE ADVANCEDPROCESSING EFFICIENTOPERATION
RedisSpeeddifferentiators
bull Createyourowndatatypesandcommands
bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability
bull CanbewritteninCC++GoPythonRustZig
bull Leverageexistingdatastructures
bull TurnRedisintoaMulti-Modeldatabase
ModulesExtendRedisInfinitely
12
httpsredislabscomcommunityredis-modules-hub
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
13
14
Introducing
RedisEnterprise
15
DBaaSbull Availablesincemid2013
bull 8500+enterprisecustomers
Softwarebull Availablesinceearly2015
bull 300+enterprisecustomers550K+databases
managedworldwide
bull 6oftopFortune10companies
bull 3oftop5communicationscompanies
Customersbull 3oftop4creditcardissuers
bull 3oftop5healthcarecompanies
bull CloudProvider
bull Highermarginbyndash Idleness
bull CloudLock-in
CloudProvidershavedifferentincentives
16
bull DBAASProvider
bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization
RedisEnterpriseAUniquePrimaryDatabase
HIGHESTPERFORMANCELINEARSCALING
HIGHAVAILABILITYWITHINSTANTFAILOVER
DURABILITYATMEMORYSPEEDS
ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)
BUILT-INHIGHPERFORMANCE SEARCH
MULTI-MODEL
FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)
INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)
FAST RELIABLE FLEXIBLE
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
18
Unevennumberofsymmetricnodes
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
bull Createyourowndatatypesandcommands
bull ReuseRedisrsquosimplicityperformancescalabilityandhighavailability
bull CanbewritteninCC++GoPythonRustZig
bull Leverageexistingdatastructures
bull TurnRedisintoaMulti-Modeldatabase
ModulesExtendRedisInfinitely
12
httpsredislabscomcommunityredis-modules-hub
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
13
14
Introducing
RedisEnterprise
15
DBaaSbull Availablesincemid2013
bull 8500+enterprisecustomers
Softwarebull Availablesinceearly2015
bull 300+enterprisecustomers550K+databases
managedworldwide
bull 6oftopFortune10companies
bull 3oftop5communicationscompanies
Customersbull 3oftop4creditcardissuers
bull 3oftop5healthcarecompanies
bull CloudProvider
bull Highermarginbyndash Idleness
bull CloudLock-in
CloudProvidershavedifferentincentives
16
bull DBAASProvider
bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization
RedisEnterpriseAUniquePrimaryDatabase
HIGHESTPERFORMANCELINEARSCALING
HIGHAVAILABILITYWITHINSTANTFAILOVER
DURABILITYATMEMORYSPEEDS
ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)
BUILT-INHIGHPERFORMANCE SEARCH
MULTI-MODEL
FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)
INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)
FAST RELIABLE FLEXIBLE
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
18
Unevennumberofsymmetricnodes
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
13
14
Introducing
RedisEnterprise
15
DBaaSbull Availablesincemid2013
bull 8500+enterprisecustomers
Softwarebull Availablesinceearly2015
bull 300+enterprisecustomers550K+databases
managedworldwide
bull 6oftopFortune10companies
bull 3oftop5communicationscompanies
Customersbull 3oftop4creditcardissuers
bull 3oftop5healthcarecompanies
bull CloudProvider
bull Highermarginbyndash Idleness
bull CloudLock-in
CloudProvidershavedifferentincentives
16
bull DBAASProvider
bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization
RedisEnterpriseAUniquePrimaryDatabase
HIGHESTPERFORMANCELINEARSCALING
HIGHAVAILABILITYWITHINSTANTFAILOVER
DURABILITYATMEMORYSPEEDS
ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)
BUILT-INHIGHPERFORMANCE SEARCH
MULTI-MODEL
FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)
INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)
FAST RELIABLE FLEXIBLE
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
18
Unevennumberofsymmetricnodes
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
14
Introducing
RedisEnterprise
15
DBaaSbull Availablesincemid2013
bull 8500+enterprisecustomers
Softwarebull Availablesinceearly2015
bull 300+enterprisecustomers550K+databases
managedworldwide
bull 6oftopFortune10companies
bull 3oftop5communicationscompanies
Customersbull 3oftop4creditcardissuers
bull 3oftop5healthcarecompanies
bull CloudProvider
bull Highermarginbyndash Idleness
bull CloudLock-in
CloudProvidershavedifferentincentives
16
bull DBAASProvider
bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization
RedisEnterpriseAUniquePrimaryDatabase
HIGHESTPERFORMANCELINEARSCALING
HIGHAVAILABILITYWITHINSTANTFAILOVER
DURABILITYATMEMORYSPEEDS
ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)
BUILT-INHIGHPERFORMANCE SEARCH
MULTI-MODEL
FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)
INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)
FAST RELIABLE FLEXIBLE
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
18
Unevennumberofsymmetricnodes
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
RedisEnterprise
15
DBaaSbull Availablesincemid2013
bull 8500+enterprisecustomers
Softwarebull Availablesinceearly2015
bull 300+enterprisecustomers550K+databases
managedworldwide
bull 6oftopFortune10companies
bull 3oftop5communicationscompanies
Customersbull 3oftop4creditcardissuers
bull 3oftop5healthcarecompanies
bull CloudProvider
bull Highermarginbyndash Idleness
bull CloudLock-in
CloudProvidershavedifferentincentives
16
bull DBAASProvider
bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization
RedisEnterpriseAUniquePrimaryDatabase
HIGHESTPERFORMANCELINEARSCALING
HIGHAVAILABILITYWITHINSTANTFAILOVER
DURABILITYATMEMORYSPEEDS
ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)
BUILT-INHIGHPERFORMANCE SEARCH
MULTI-MODEL
FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)
INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)
FAST RELIABLE FLEXIBLE
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
18
Unevennumberofsymmetricnodes
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
bull CloudProvider
bull Highermarginbyndash Idleness
bull CloudLock-in
CloudProvidershavedifferentincentives
16
bull DBAASProvider
bull Highermarginbybetterresourceutilizationndash Multi-tenancyndash ReducingRAMndash CPUutilization
RedisEnterpriseAUniquePrimaryDatabase
HIGHESTPERFORMANCELINEARSCALING
HIGHAVAILABILITYWITHINSTANTFAILOVER
DURABILITYATMEMORYSPEEDS
ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)
BUILT-INHIGHPERFORMANCE SEARCH
MULTI-MODEL
FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)
INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)
FAST RELIABLE FLEXIBLE
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
18
Unevennumberofsymmetricnodes
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
RedisEnterpriseAUniquePrimaryDatabase
HIGHESTPERFORMANCELINEARSCALING
HIGHAVAILABILITYWITHINSTANTFAILOVER
DURABILITYATMEMORYSPEEDS
ACTIVE-ACTIVEGEODISTRIBUTION(CRDT-BASED)
BUILT-INHIGHPERFORMANCE SEARCH
MULTI-MODEL
FLEXIBLEDEPLOYMENTOPTIONS(CLOUDON-PREMHYBRID)
INTELLIGENTTIEREDDATAACCESS(RAMampFLASHMEMORY)
FAST RELIABLE FLEXIBLE
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
18
Unevennumberofsymmetricnodes
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
18
Unevennumberofsymmetricnodes
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
19
Singlemasterdatabase
M
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
20
AnHAdatabase
SM
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
21
AClusteredDatabase
M1 M2 M3
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
Howdokeysgetassignedtopartitions
22
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
Howdokeysgetassignedtopartitions
23
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
RedisEnterpriseCluster
Node1 Node2 NodeN(oddnumber)
24
AHighlyAvailableClusteredDatabase
M1 M2 M3S3 S1 S2
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
RedisEnterpriseNode
25
ClusterManager
EnterpriseLayer
OpenSourceLayer
RESTAPIZerolatencyproxy
RedisShards
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
RedisEnterpriseSharedNothingSymmetricArchitecture
ClusterManagementPath
NodeWatchdogClusterWatchdog
Node1 Node2 NodeN(oddnumber)
RedisShardsampProxies
DataPath
26
Data-PathandControlManagementPathSeparation
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
27
asadatagrid
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
MicroservicesArchitectureandPolyglotPersistence
28
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
TheCostofPolyglotPersistence
AuthenticationAuthentication
Authentication
KeyValue
K V
KeyValue
K V
KeyValue
K V
Customers
KeyValue
K V
Graph
CustomersKeyValue
K V
Graph
Catalog
RDBMS Cache
Search
SessionStore
SessionStore
SessionStore
Document
Document
Document
API
API
API
API
API
API
API
API
API
FraudDetection
API
ColumnarSearch
Fulfilment API
RDBMS
Increasedapplicationcomplexity CostlycommunicationApplicationdoesheavyliftinginsharingdatakeepingdatasetsinsync
HighoperationalburdenHighercostofownershipDifferentdatabaseshavespecializedadministrativescalingavailability
requirements
Sub-optimalResourceUsage HighercostDedicatingpodsserversforeachtypeofdatabasereducesdeploymentefficiency
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
API API API API APIAPI API
RedisEnterpriseAMulti-modelDatabaseforMicroservices
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
Authentication Customers Catalog Search FraudDetection SessionStore
SearchGraphKeyValue
RDBMS
KeyValueCache
RDBMS
Cache
Fulfilment
KeyValue Document
Built-inMessageBroker
Built-inPub-SubStreamsforeventsynchacrossdatastores
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
Whatarewemissing
bull Howtoconsumemessagesinthisldquobuiltinmessageborkerrdquo
bull GivenashardeddatabasehowcanIrunanalyticalqueries
bull MultiModeldatabasendash Singlecopyincoredatatypesndash Intermodulecommunicationndash ComponentXdoingtranslationsbetweenmodules
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
33
RedisGears
Introducing
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
RedisGearsisaServerlessengineformulti-modelandclusteroperationsinRedissupportingbotheventdriven
aswellasbatchoperations
WhatisRedisGears
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
GearsCoordinator MapReducerGearsExecuter
GearsinfrastructureiswritteninC
C-API
Soon Soon
HighPerformanceArchitecture
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
RedisGearsallowstodefineapipeofoperations
bull Returningvaluefromoneoperationispassedtothenextoperationinthepipe
bull Lastoperationreturningtheresulttotheuserbull Firstoperationiscalledlsquoreaderrsquo-responsibleforprovidingdata
ndash Keysreader-readkeysfromRedisndash Streamreader-readstreamsfromRedisndash Pythonreader-allowtousertowritehisownreadersinpython
ScriptingwithRedisGears
36
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
bull Map
bull FlatMap
bull Filterbull Groupby+Reducebull Aggregatebull Sortbull Limit
bull ForEachbull Distinct
SupportedOperations
37
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
Reader(Flat)Mapper
UsingRedisGearsndash(Flat)Mapping
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
UsingRedisGears-Filtering
Reader Filterrecordwith1doc
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
UsingRedisGears-Aggregate
Reader CountAggregator
1
1
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
Demo
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
GearshasastreamingAPItoallowtotriggergearsexecutiononeventsndash RedisStreamevents-Triggeranexecutionwheneveranewdataentersasteamndash RedisKeysevents-Triggeranoperationwheneverakeyistouched
UseCase1ndashStreamProcessing
RedisTimeSeriesRedisStreams
Everysec
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
BecauseofRedisGearsrsquoflexibility(itsactuallyrunningpython)youcanachieveinternalmoduleintegrationwithitndash ReadfromhashesandindexinRediSearchRedisGraphndash ReadRedisJSONdataandpasstoRedisTimeSeriesndash hellip
UseCase2ndashaMultiModelEngine
RediSearch
RedisHashesEveryupdate
RedisGraph
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
createthebuilderbuilder=GearsBuilder()filtereventsonkeyall_keysbuilderfilter(lambdaxx[key]=all_keys)addthekeystoall_keyssetbuildermap(lambdaxexecute(saddall_keysx[key]))registertheexecutiononkeyspacenotificationbuilderregister()
Buildagearthatcreatesmaintainsasetofallkeyswithinredis
Recipe1ndasheventriggering
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
createthepipebuilderKeysOnlyReaderisaperformanceimprovementonlypipingthekeysbuilder=GearsBuilder(KeysOnlyReader)getfromeachhashthegenresfieldbuildermap(lambdaxexecute(hgetxgenres))filterthosewhodonothavegenresbuilderfilter(lambdaxxisnotNone)splitgenresbycommabuilderflatmap(lambdaxxsplit())countforeachgenrethenumberoftimesitappearsbuildercountby()starttheexecutionbuilderrun(movie)
Buildagearthatcountshowoftenagenreisusedwithinasetofmovies
Recipe2ndashmapreducing
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
createthebuilderwithaStreamReaderbuilder=GearsBuilder(StreamReader)extracteachfieldvaluepairfromthemessageandincreasethepipegranularitybuilderflatmap(lambdax[(a[0]a[1])forainxitems()])filteroutthestreamIditselfbuilderfilter(lambdaxx[0]=streamId)makesurethegearsdatalivesinthecorrectshardbuilderrepartition(lambdaxx[0])applyeachfieldvaluepairtoakeybuilderforeach(lambdaxexecute(setx[0]x[1]))registeronnewmessagesonthestreaminputStreambuilderregister(inputStream)
Buildagearthatconsumesastreamandupdateskeysaccordingly
Recipe3ndashstreamprocessing
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
ExampleTriggerExplained
47
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
ExampleTriggerExplained
48
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
ExampleTriggerExplained
49
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
ExampleTriggerExplained-Flatmap
50
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
ExampleTriggerExplained-Repartition
51
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
ExampleTriggerExplained-executeCommand
52
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
Demo
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
DemoSetup
54
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
bull httpsgithubcomRedisGearsAnimalRecognitionDemo
bull redisfoundmycat
Challenge
55
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
RediSearch(GA) redisearchio
RedisBloom(GA) redisbloomio
RedisTimeSeries redistimeseriesio
RedisJSON(GA) redisjsonio
RedisAI redisaiio
RedisGraph(GA) redisgraphio
RedisModules
56
RedisGearsredisgearsio
Thankyoupieterredislabscom
57
Thankyoupieterredislabscom
57