PyCuda: Come sfruttare la potenza delle schede video nelle applicazioni python
92
PyCUDA: Harnessing the power of GPU with Python
-
Upload
pycon-italia -
Category
Technology
-
view
1.318 -
download
2
description
Fabrizio Milo
Transcript of PyCuda: Come sfruttare la potenza delle schede video nelle applicazioni python
PyCUDA: Harnessing the power of GPU with Python
PyCon 4 – Florence 2010 – Fabrizio Milo
1. Why a GPU ? 2. How does It works ? 3. How do I Program it ? 4. Can I Use Python ?
Talk Structure
PyCon 4 – Florence 2010 – Fabrizio Milo
1. Why a GPU ? 2. How does It works ? 3. How do I Program it ? 4. Can I Use Python ?
Talk Structure
PyCon 4 – Florence 2010 – Fabrizio Milo
WHY A GPU ?
PyCon 4 – Florence 2010 – Fabrizio Milo
APPLICATIONS & DEMOS
PyCon 4 – Florence 2010 – Fabrizio Milo
Why GPU?
PyCon 4 – Florence 2010 – Fabrizio Milo
1. Why a GPU ? 2. How does it works ? 3. How do I Program it ? 4. Can I Use Python ?
Talk Structure
PyCon 4 – Florence 2010 – Fabrizio Milo
How does it works ?
PyCon 4 – Florence 2010 – Fabrizio Milo
Cache
ALU
Control
ALU
ALU
ALU
DRAM
CPU
PyCon 4 – Florence 2010 – Fabrizio Milo
DRAM
GPU
PyCon 4 – Florence 2010 – Fabrizio Milo
DRAM
Cache
ALU Control
ALU
ALU
ALU
DRAM
CPU GPU
PyCon 4 – Florence 2010 – Fabrizio Milo
CUDA
PyCon 4 – Florence 2010 – Fabrizio Milo
Compute Unified Device Architecture
PyCon 4 – Florence 2010 – Fabrizio Milo
CUDA
A Parallel Computing Architecture for NVIDIA GPUs
Direct X Compute
PyCon 4 – Florence 2010 – Fabrizio Milo
Execution Model
CUDA Device Model
PyCon 4 – Florence 2010 – Fabrizio Milo
EXECUTION MODEL
PyCon 4 – Florence 2010 – Fabrizio Milo
Thread
Smallest unit of logic
PyCon 4 – Florence 2010 – Fabrizio Milo
A Block
A Group of Threads
PyCon 4 – Florence 2010 – Fabrizio Milo
A Grid
A Group of Blocks
PyCon 4 – Florence 2010 – Fabrizio Milo
One Block can have many threads
PyCon 4 – Florence 2010 – Fabrizio Milo
One Grid can have many blocks
PyCon 4 – Florence 2010 – Fabrizio Milo
DEVICE MODEL The hardware
PyCon 4 – Florence 2010 – Fabrizio Milo
Scalar Processor
PyCon 4 – Florence 2010 – Fabrizio Milo
Scalar Processor
PyCon 4 – Florence 2010 – Fabrizio Milo
Many Scalar Processors
PyCon 4 – Florence 2010 – Fabrizio Milo
+ Register File
PyCon 4 – Florence 2010 – Fabrizio Milo
+ Shared Memory
PyCon 4 – Florence 2010 – Fabrizio Milo
Multiprocessor
PyCon 4 – Florence 2010 – Fabrizio Milo
Device
PyCon 4 – Florence 2010 – Fabrizio Milo
Real Example: 10-Series Architecture
" 240 Scalar Processor (SP) cores execute kernel threads " 30 Streaming Multiprocessors (SMs) each contain " 8 scalar processors " 1 double precision unit " Shared memory
PyCon 4 – Florence 2010 – Fabrizio Milo
Software Hardware
Thread
Scalar Processor
Thread Block Multiprocessor
Grid Device
PyCon 4 – Florence 2010 – Fabrizio Milo
Global Memory
PyCon 4 – Florence 2010 – Fabrizio Milo
Global Memory
PyCon 4 – Florence 2010 – Fabrizio Milo
Host - Device
RAM
Global Memory CPU
PyCon 4 – Florence 2010 – Fabrizio Milo
Host – Multi Device
RAM
CPU
PyCon 4 – Florence 2010 – Fabrizio Milo
1. Why a GPU ? 2. How does It works ? 3. How do I Program it ? 4. Can I Use Python ?
PyCon 4 – Florence 2010 – Fabrizio Milo
Software Hardware
Thread
Scalar Processor
Thread Block Multiprocessor
Grid Device
PyCon 4 – Florence 2010 – Fabrizio Milo
__global__ void multiply_them( float *dest, float *a, float *b )
{ const int i = threadIdx.x; dest[i] = a[i] * b[i];}
Kernel
Thread
PyCon 4 – Florence 2010 – Fabrizio Milo
__global__ void multiply_them( float *dest, float *a, float *b )
{ const int i = threadIdx.x; dest[i] = a[i] * b[i];}
Kernel
Thread
PyCon 4 – Florence 2010 – Fabrizio Milo
__global__ void multiply_them( float *dest, float *a, float *b )
{ const int i = threadIdx.x; dest[i] = a[i] * b[i];}
Kernel
Block
PyCon 4 – Florence 2010 – Fabrizio Milo
__global__ void kernel( … ){ const int idx =
blockIdx.x * blockDim.x + threadIdx.x;…
}
Kernel
Grid
PyCon 4 – Florence 2010 – Fabrizio Milo
.bin
NVCC
How do I Program it ?
GCC
.cubin CPU GPU
Kernel Main Logic
PyCon 4 – Florence 2010 – Fabrizio Milo
.bin
NVCC
How do I Program it ?
GCC
.cubin
CPU
GPU
Kernel Main Logic
..bin .cubin
PyCon 4 – Florence 2010 – Fabrizio Milo
Host - Device
RAM
Global Memory CPU
PyCon 4 – Florence 2010 – Fabrizio Milo
RAM
Global Memory CPU
PyCon 4 – Florence 2010 – Fabrizio Milo
Allocate Memory
cudaMalloc( pointer, size )
PyCon 4 – Florence 2010 – Fabrizio Milo
Copy to device
cudaMalloc( pointer, size )
cudaMemcpy( dest, src, size, direction)
PyCon 4 – Florence 2010 – Fabrizio Milo
Kernel Launch
cudaMalloc( pointer, size )
cudaMemcpy( dest, src, size, direction)
Kernel<<< # blocks, # threads >> (*params)
PyCon 4 – Florence 2010 – Fabrizio Milo
Get Back the Results
cudaMalloc( pointer, size )
cudaMemcpy( dest, src, size, direction)
Kernel<<< # blocks, # threads >> (*params)
cudaMemcpy( dest, src, size, direction)
PyCon 4 – Florence 2010 – Fabrizio Milo
Error Handling
If(cudaMalloc( pointer, size ) != cudaSuccess){ handle_error()}
PyCon 4 – Florence 2010 – Fabrizio Milo
And soon it becomes …
If(cudaMalloc( pointer, size ) != cudaSuccess){ handle_error()}
if (cudaMemcpy( dest, src, size, direction ) == cudaSuccess) {}
If (Kernel<<< # blocks, # threads >> (*params) != cudaSuccess){ handle_error()}
If( cudaMemcpy( dest, src, size, direction) != cudaSuccess) { }
PyCon 4 – Florence 2010 – Fabrizio Milo
And soon it becomes …
If(cudaMalloc( pointer, size ) != cudaSuccess){ handle_error()}
if (cudaMemcpy( dest, src, size, direction ) == cudaSuccess) {}
If (Kernel<<< # blocks, # threads >> (*params) != cudaSuccess){ handle_error()}
If( cudaMemcpy( dest, src, size, direction) != cudaSuccess) { }
If(cudaMalloc( pointer, size ) != cudaSuccess){ handle_error()}
if (cudaMemcpy( dest, src, size, direction ) == cudaSuccess) {}
If (Kernel<<< # blocks, # threads >> (*params) != cudaSuccess){ handle_error()}
If( cudaMemcpy( dest, src, size, direction) != cudaSuccess) { }
If(cudaMalloc( pointer, size ) != cudaSuccess){ handle_error()}
if (cudaMemcpy( dest, src, size, direction ) == cudaSuccess) {}
If (Kernel<<< # blocks, # threads >> (*params) != cudaSuccess){ handle_error()}
If( cudaMemcpy( dest, src, size, direction) != cudaSuccess) { }
If(cudaMalloc( pointer, size ) != cudaSuccess){ handle_error()}
if (cudaMemcpy( dest, src, size, direction ) == cudaSuccess) {}
If (Kernel<<< # blocks, # threads >> (*params) != cudaSuccess){ handle_error()}
If( cudaMemcpy( dest, src, size, direction) != cudaSuccess) { }
If(cudaMalloc( pointer, size ) != cudaSuccess){ handle_error()}
if (cudaMemcpy( dest, src, size, direction ) == cudaSuccess) {}
If (Kernel<<< # blocks, # threads >> (*params) != cudaSuccess){ handle_error()}
If( cudaMemcpy( dest, src, size, direction) != cudaSuccess) { }
If(cudaMalloc( pointer, size ) != cudaSuccess){ handle_error()}
if (cudaMemcpy( dest, src, size, direction ) == cudaSuccess) {}
If (Kernel<<< # blocks, # threads >> (*params) != cudaSuccess){ handle_error()}
If( cudaMemcpy( dest, src, size, direction) != cudaSuccess) { }
PyCon 4 – Florence 2010 – Fabrizio Milo
PyCon 4 – Florence 2010 – Fabrizio Milo
1. Why a GPU ? 2. How does It works ? 3. How do I Program it ? 4. Can I Use Python ?
PyCon 4 – Florence 2010 – Fabrizio Milo
= PYCUDA
+
& ANDREAS KLOCKNER
PyCon 4 – Florence 2010 – Fabrizio Milo
PyCuda Philosopy
Provide Complete Access
PyCon 4 – Florence 2010 – Fabrizio Milo
PyCuda Philosopy
AutoMatically Manage
Resources
PyCon 4 – Florence 2010 – Fabrizio Milo
PyCuda Philosopy
Check and Report Errors
PyCon 4 – Florence 2010 – Fabrizio Milo
PyCuda Philosopy
Cross Platform
PyCon 4 – Florence 2010 – Fabrizio Milo
PyCuda Philosopy
Allow Interactive
Use
PyCon 4 – Florence 2010 – Fabrizio Milo
PyCuda Philosopy
NumPy Integration
PyCon 4 – Florence 2010 – Fabrizio Milo
NUMPY - ARRAY
PyCon 4 – Florence 2010 – Fabrizio Milo
import numpy
my_array = numpy.array([1,] * 100)
1 1 1 1 1 1
99 0
PyCon 4 – Florence 2010 – Fabrizio Milo
import numpy
my_array = numpy.array([1,] * 100)
my_array[3] = 0
0 1 1 1 1 1
PyCon 4 – Florence 2010 – Fabrizio Milo
PyCuda: Workflow
PyCon 4 – Florence 2010 – Fabrizio Milo
PyCuda: Workflow
PyCon 4 – Florence 2010 – Fabrizio Milo
PyCuda: Workflow
PyCon 4 – Florence 2010 – Fabrizio Milo
Memory Allocation
cuda.mem_alloc( size_bytes )
PyCon 4 – Florence 2010 – Fabrizio Milo
Memory Copy
gpu_mem = cuda.mem_alloc( size_bytes )
cuda.memcpy_htod( gpu_mem, cpu_mem )
PyCon 4 – Florence 2010 – Fabrizio Milo
Kernel
gpu_mem = cuda.mem_alloc( size_bytes )
cuda.memcpy_htod( gpu_mem, cpu_mem )
SourceModule(“””__global__ void multiply_them( float *dest, float *a,
float *b ){ const int i = threadIdx.x; dest[i] = a[i] * b[i];}”””)
PyCon 4 – Florence 2010 – Fabrizio Milo
Kernel Launch
mod = SourceModule(“””__global__ void multiply_them( float *dest, float *a,
float *b ){ const int i = threadIdx.x; dest[i] = a[i] * b[i];}”””)
multiply_them = mod.get_function(“multiply_them”)multiply_them ( *args, block=(30, 64, 1))
PyCon 4 – Florence 2010 – Fabrizio Milo
PyCon 4 – Florence 2010 – Fabrizio Milo
PyCon 4 – Florence 2010 – Fabrizio Milo
PyCon 4 – Florence 2010 – Fabrizio Milo
DEMO Hello Gpu
PyCon 4 – Florence 2010 – Fabrizio Milo
GPUARRAY
PyCon 4 – Florence 2010 – Fabrizio Milo
gpuarray
PyCon 4 – Florence 2010 – Fabrizio Milo
gpuarray.to_gpu(numpy array)
numpy array = gpuarray.get()
PyCuda: GpuArray
PyCon 4 – Florence 2010 – Fabrizio Milo
gpuarray.to_gpu(numpy array)
numpy array = gpuarray.get()
PyCuda: GpuArray
+, -, !, /, fill, sin, exp, rand, basic indexing, norm, inner product …
PyCon 4 – Florence 2010 – Fabrizio Milo
PyCuda: GpuArray: ElementWise
from pycuda.elementwise import ElementwiseKernel
PyCon 4 – Florence 2010 – Fabrizio Milo
PyCuda: GpuArray: ElementWise
from pycuda.elementwise import ElementwiseKernel
lincomb = ElementwiseKernel( ” float a , float !x , float b , float !y , float !z”, ”z [ i ] = a !x[ i ] + b!y[i ] ”
)
PyCon 4 – Florence 2010 – Fabrizio Milo
PyCuda: GpuArray: ElementWise
from pycuda.elementwise import ElementwiseKernel
lin comb = ElementwiseKernel( ” float a , float !x , float b , float !y , float !z”, ”z [ i ] = a !x[ i ] + b!y[i ] ”
)
c gpu = gpuarray. empty like (a gpu) lincomb (5, a gpu, 6, b gpu, c gpu)
assert la . norm((c gpu ! (5!a gpu+6!b gpu)).get()) < 1e!5
PyCon 4 – Florence 2010 – Fabrizio Milo
Meta-Programming
__kernel_template__ = “””__global__ void kernel( args ){
for (int i=0; i={{ iterations }}; i++){ {{operations}}}
}”””
See for example jinja2
PyCon 4 – Florence 2010 – Fabrizio Milo
Meta-Programming
PyCon 4 – Florence 2010 – Fabrizio Milo
Meta-Programming
Generate Source !
PyCon 4 – Florence 2010 – Fabrizio Milo
Performances ?
PyCon 4 – Florence 2010 – Fabrizio Milo
DEMO mandelbrot
PyCon 4 – Florence 2010 – Fabrizio Milo
PyCuda: Documentation
PyCon 4 – Florence 2010 – Fabrizio Milo
PyCuda
WebSite: http://mathema.tician.de/software/ pycuda
License: X Consortium License
(no warranty, free for all use)
Dependencies: Python 2.4+, numpy, Boost
PyCon 4 – Florence 2010 – Fabrizio Milo
In the Future …
OPENCL
PyCon 4 – Florence 2010 – Fabrizio Milo
THANK YOU & HAVE FUN !
PyCon 4 – Florence 2010 – Fabrizio Milo
?
