KDB+/R Integration
-
Upload
rory-winston -
Category
Software
-
view
271 -
download
1
Transcript of KDB+/R Integration
KDB+ AND R INTEGRATIONRORY WINSTON
KDB+ AND R INTEGRATION
AGENDA
▸ Why integrate R/kdb+
▸ Basic R integration via CSV import/export
▸ R integration via qserver
▸ Integrating R math functionality into kdb+
▸ Generating kdb+ data with R
KDB+ AND R INTEGRATION
WHAT IS R?
▸ DSL For Statistical Techniques and Data Analysis
▸ Integrated programming and EDA (Exploratory Data Analysis) environment
▸ Open-Source
▸ Thousands of contributed packages
KDB+ AND R INTEGRATION
WHY USE R?
▸ Comprehensive Environment
▸ Tooling
▸ Command-Line
▸ IDE
▸ Web
▸ Thousands of Packages
▸ Rich Reporting/Graphing/Analysis Capabilities
KDB+ AND R INTEGRATION
PART ONE: CSV IMPORT/EXPORT
▸ CSV import/export
▸ In kdb+:
▸ save`:table.csv
▸ In R:
▸ read.csv(‘table.csv’,header=TRUE)
EXAMPLE #1CSV IMPORT/EXPORT
KDB+ AND R INTEGRATION
CSV IMPORT / EXPORT
▸ Simplest approach
▸ Interchange with any other system
▸ No type information
▸ Manual overhead
▸ Can be a slow/laborious process
KDB+ AND R INTEGRATION
PART TWO: THE QSERVER LIBRARY
▸ Available from
▸ http://code.kx.com/wsvn/code/cookbook_code/r/
▸ Consists of:
▸ A native library containing kdb+ connectivity
▸ An R wrapper
▸ Binaries for Windows/Linux/OSX
KDB+ AND R INTEGRATION
THE QSERVER LIBRARY
qserver
kdb+
laye
r
R la
yer
KDB+ AND R INTEGRATION
BASIC QSERVER OPERATIONS
▸ Connect to a kdb+ instance:
▸ open_connection(host,port,credentials)
▸ Returns a handle
▸ Execute a query:
▸ execute(handle,query)
▸ Close connection:
▸ close_connection(handle)
EXAMPLE #2QSERVER
KDB+ AND R INTEGRATION
QSERVER INTEGRATION
▸ Most Efficient
▸ Can Manage Multiple Connections
▸ Can Be Wrapped Into An R Library
▸ Easy to Extend With Custom R Code
KDB+ AND R INTEGRATION
PART THREE: LEVERAGING THE R MATH LIBRARY
▸ R Comes With A Standalone Math Library
▸ Contains A Large Range Of Base Statistical/Math Routines
▸ E.g. Base Probability Functions/RNG
▸ Can Be Packaged Into a Library And Reused in kdb+
▸ I have packed a library and a set of routines on:
▸ https://github.com/rwinston/kdb-rmathlib
KDB+ AND R INTEGRATION
PART THREE: LEVERAGING THE R MATH LIBRARY
▸ Example In R:
▸ rnorm(1000,5,2)
▸ Generates 1000 random variables with a mean of 5 and standard deviation of 2
▸ In q, using the rmathlib integration:
▸ norm[1000;5;2]
▸ R’s math library contains hundreds of useful base functions
EXAMPLE #3RMATHLIB
KDB+ AND R INTEGRATION
USING R TO GENERATE Q CODE
▸ Generate q code from R
▸ Similar in principle to CSV export
▸ See
▸ http://www.theresearchkitchen.com/archives/776
EXAMPLE #4Q CODE GENERATION
THANKS