PyCoRAM: Python-Verilog高位合成とメモリ抽象化によるFPGAアクセラレータ向けIPコア開発フレームワーク (FPGAX #05)
FPGAアクセラレータの作り方 (IBM POWER+CAPI編)
-
Upload
mr-vengineer -
Category
Devices & Hardware
-
view
1.145 -
download
1
Transcript of FPGAアクセラレータの作り方 (IBM POWER+CAPI編)
![Page 1: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/1.jpg)
FPGAアクセラレータの作り方
IBM POWER+CAPI編
@Vengineer2016/10/7
![Page 2: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/2.jpg)
Vengineer DEATH
無限ゲームのなか
いつものように、
よろしくお願いします。
@Vengineer に居ます
![Page 3: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/3.jpg)
Overview
![Page 4: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/4.jpg)
http://www.nallatech.com/wp-content/uploads/IBM_CAPI_Users_Guide_1-2.pdf
一見すると、CPUとFPGAだがFPGAの中に
ある
PSLと
POWER8の中にある
CAPPが重要
IBM CAPIとは
Coherent Accelerator Processor Interface
![Page 5: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/5.jpg)
The Coherent Accelerator Processor Interface (CAPI) is a general term for the infrastructure of attaching acoherent accelerator to an IBM POWER® system. The main application is executed on the host processorwith computation-heavy functions executing on the accelerator. The accelerator is a full peer to the hostprocessor, with direct communication with the application. The accelerator uses an unmodified effectiveaddress with full access to the real address space. It uses the processor’s page tables directly with pagefaults handled by system software.
Coherent Accelerator Processor Interface
![Page 6: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/6.jpg)
http://www.nallatech.com/wp-content/uploads/IBM_CAPI_Users_Guide_1-2.pdf
POWER8(CPU)は、MMUで仮想アドレス => 物理アドレスにしてメモリにアクセス
POWER8とAcceleratorのメモリ共有
Acceleratorは、PSL/PCIe/CAPPで仮想アドレス => 物理アドレスにしてメモリにアクセス
![Page 7: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/7.jpg)
http://openpowerfoundation.org/wp-content/uploads/2015/03/Finamore-Nick_OPFS2015_Altera_031215_final.pdf
Host Memory & Device Memory
Altera SDK for OpenCL
![Page 8: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/8.jpg)
http://openpowerfoundation.org/wp-content/uploads/2015/03/Finamore-Nick_OPFS2015_Altera_031215_final.pdf
Coherent Memory
Xeon - QPI − FPGACCI-extended
![Page 9: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/9.jpg)
コピー不要 &デバイスドライバ不要
メモリ移動 メモリ移動終了待ち起動
終了待ち起動
http://www.nallatech.com/wp-content/uploads/Ent2014-CAPI-on-Power8.pdf
![Page 10: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/10.jpg)
Hardware
![Page 11: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/11.jpg)
AFU
POWER8とFPGAの関係
CAPP
PCIe FPGA
PHB : PCIe Host BridgeCAPP : Coherently Attached Processor Proxy
PSL : Power Service LayerAFU : Accelerator Function Unit
PHB
CPU
POWER8
PSL
![Page 12: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/12.jpg)
AFU
PSL : POWER Service Layer
CAPP
PCIe FPGA
PSLは、IBMが開発
CAPI対応FPGAボードベンダから提供される (IBMから供給)
PHB
CPU
POWER8
PSL
![Page 13: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/13.jpg)
PSL : POWER Service Layer
The PSL provides the translation and interrupt services that the AFU needs.
This is what the kernel interacts with. For example, if the AFU needs to read a particular effective address, it sends that address to the PSL, the PSL then translates it, fetches the data from memory and returns it to the AFU. If the PSL has a translation miss, it interrupts the kernel and the kernel services the fault. The context to which this fault is serviced is based on who owns that acceleration function.
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/plain/Documentation/powerpc/cxl.txt
![Page 14: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/14.jpg)
AFU
AFU : Accelerator Function Unit
CAPP
PCIe FPGA
AFUは、ユーザが開発
HDL(Verilog HDL/VHDL)でなく、HLSなどを使ってC/C++でも可能
PHB
CPU
POWER8
PSL
![Page 15: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/15.jpg)
Software
![Page 16: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/16.jpg)
AFU
libcx
PCIe FPGA
Applicationは、libcxl (ユーザ空間) API =>Linux CXLデバイスドライバ経由でFPGAのAFUにアクセスする
FPGA(AFU)独自のデバイスドライバは不要
PSL
CXL
Application
POWER8
libcxl
![Page 17: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/17.jpg)
Coherent Accelerator Interface
https://github.com/ibm-capi/libcxl
CXL : linux device driver https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/plain/Documentation/powerpc/cxl.txt
![Page 18: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/18.jpg)
libcxl API
open : デバイスのオープン
ioctl : CXL_IOCTL_START_WORK
CXL_IOCTL_GET_PROCESS_ELEMENT
mmap : mmio空間へのアクセス
read : AFUからのイベントを読み込む
![Page 19: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/19.jpg)
AFU
Application
PCIe FPGA
Applicationは、libcxl (ユーザ空間) API =>Linux CXLデバイスドライバ経由でFPGAのAFUにアクセスする
FPGA(AFU)独自のデバイスドライバは不要
PSL
CXL
Application
POWER8
libcxl
![Page 20: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/20.jpg)
Applicationのテンプレートコード
// AFUの確認
struct cxl_afu_h *afu_h = cxl_afu_next(NULL);// AFUを獲得
afu_h = cxl_afu_open_h(afu_h,CXL_VIEW_DEDICATED);// AFUの起動
cxl_afu_attach(afu_h, (uint64_t) wed);// 必要ならMMIOをマップ
cxl_mmio_map(afu_h, CXL_MMIO_BIG_ENDIAN);
// ここで何かをやる
cxl_mmio_unmap(afu_h); // MMIOをアンマップ
cxl_afu_free(afu_h); // AFUを開放
![Page 21: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/21.jpg)
CAPI Solution FLow
http://www.nallatech.com/wp-content/uploads/Ent2014-CAPI-on-Power8.pdf
![Page 22: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/22.jpg)
本題のHDLシミュレーション環境
![Page 23: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/23.jpg)
PSLSE ModelSim ASE 10.4d
Power Service Layer Simulation Engine
PLITCP/IP
AFU (HDL)TCP/IP
PSLSE用のlibclxには、
一部実装されていない関数がある
https://github.com/ibm-capi/pslse
Application
libcxl
![Page 24: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/24.jpg)
githubから環境をダウンロード
% git clone https://github.com/ibm-capi/pslse
% lsNOTICES afu_driver pslseQUICK_START common sample_appREADME debug testSIMULATOR_SETUP libcxl vcs_include
% cd afu_driver
% lssrc verilog
![Page 25: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/25.jpg)
ModelSim ASEの環境設定
% export ALTERA_HOME={ModelSim ASEのディレクトリ}
% export VPI_USER_H_DIR=${ALTERA_HOME}/include
VPI_USER_H_DIRは、利用するインクルードファイルの場所を指定する
![Page 26: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/26.jpg)
DPI-Cライブラリのビルド
% cd src ; BIT32=y make [CC] afu_driver.o [CC] afu_driver.d [SED] afu_driver.d [CC] psl_interface.o [CC] psl_interface.d [SED] psl_interface.d [CC] veriuser.sl [CC] libdpi.so
ModelSim ASE 10.4d (Linux版)は32ビット版なのでマクロ(BIT32)を有効にする必要があるREADME.32bitを参照のこと
ライブラリ
![Page 27: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/27.jpg)
HDLのコンパイル
% cd verilog// ライブラリの生成% vlib work// WORKライブラリへマッピング% vmap work// トップテストベンチをWORKライブラリにコンパイル% vlog -sv top.v-- Compiling module top
Top level modules: topEnd time: 14:13:01 on Jul 24,2016, Elapsed time: 0:00:00Errors: 0, Warnings: 0
トップテストベンチ
![Page 28: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/28.jpg)
シミュレーション開始
% vsim -c -L afu -sv_lib ../src/libdpi top ¥ -do "run -all"# Loading work.top# ** Error: (vsim-19) Failed to access library 'afu' at "afu".# No such file or directory. (errno = ENOENT)# ** Error: (vsim-3033) top.v(699): Instantiation of 'afu' failed. The design unit was not found.# Time: 0 ps Iteration: 0 Instance: /top File: top.v# Searched libraries:# ** Error: (vsim-19) Failed to access library 'afu' at "afu".# No such file or directory. (errno = ENOENT)# /home/uip/ibm-capi/pslse/afu_driver/verilog/work# Error loading designError loading design# End time: 14:18:35 on Jul 24,2016, Elapsed time: 0:00:00# Errors: 3, Warnings: 0
work.topをロード中にafuが無いというエラーが発生
![Page 29: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/29.jpg)
( ̄Д ̄;) ガーン
AFUのサンプルがない
![Page 30: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/30.jpg)
いつものように
Google君にきいた
![Page 31: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/31.jpg)
CAPIStreaming Framework
![Page 32: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/32.jpg)
capi-streaming-framework
AFU framework for streaming applications with CAPI connected FGPAs.
% git clone https://github.com/mbrobbel/capi-streaming-framework.git
% cd capi-streaming-framework
% git submodule update --init
http://slides.com/mbrobbel/capi-streaming-framework
![Page 33: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/33.jpg)
PSLSE ModelSim ASE 10.4d
AFUは、VHDLで実装されている
Applicationmemcpy
PLITCP/IP
Top (SystemVerilog)AFU (VHDL)
libclx
TCP/IP
ModelSim ASE 10.4d では、
Verilog HDL(SystemVerilog)/VHDLの混在シミュレーションが出来るよ
http://blogs.yahoo.co.jp/verification_engineer/69928166.html
![Page 34: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/34.jpg)
pslse Build
% export VPI_USER_H_DIR=${ALTERA_HOME}/include
% make pslse-build
makeコマンドが、(cd sim/pslse/afu_driver/src && make clean && BIT32=y make)を実行して、
libvpi.so, libcxl.a, libcxl.so を作ってくれた
最初からこちらでやればいいんだった。orz
![Page 35: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/35.jpg)
pslse Buildでのエラー
[CURL ../common/misc/cxl.h] [CC] libcxl.oIn file included from libcxl.h:21:0, from libcxl.c:32:../common/misc/cxl.h:140:2: error: unknown type name 'u8' u8 data[]; ^make[1]: *** [libcxl.o] Error 1
sim/pslse/libcxl/Makefile.vars で、
CFLAGS += -Wall -Wunused-but-set-variable -I$(CURDIR) -I$(COMMON_DIR) -fPIC
CFLAGS += -Du8=__u8を追加して対応した
![Page 36: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/36.jpg)
Application Build
% make sim-buildmkdir -p host/app/sim-buildc++ host/app/src/example.cpp -o host/app/sim-build/example sim/pslse/libcxl/libcxl.a -O3 -Wall -m64 -Isim/pslse/common -Isim/pslse/libcxl -lrt -lpthread -D SIM
こちらも作ってくれた。orz
![Page 37: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/37.jpg)
Application Buildでのエラー
In file included from sim/pslse/libcxl/libcxl.h:21:0, from host/app/src/example.cpp:9:sim/pslse/common/misc/cxl.h:140:2: error: 'u8' does not name a type u8 data[]; ^make: *** [sim-build] Error 1
Makefile で、
CFLAGS = -O3 -Wall -m64を
CFLAGS = -O3 -Wall -m64 -Du8=__u8にて、対応した
![Page 38: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/38.jpg)
cxl.hのエラーの原因
2016-06-28cxl: Add mechanism for delivering AFU driver specific events
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/include/uapi/misc/cxl.h?id=b810253bd9342f863a86ec7dfff4a5a7a0394d2f
にて追加され、Linux v4.8-rc1で取り込まれたのが原因
![Page 39: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/39.jpg)
sim/pslse/libcxl/Makefile内の
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/plain/include/uapi/misc/cxl.h
を
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.gi
t/plain/include/uapi/misc/cxl.h ?h=v4.7
と、Linux v4.7からコードを獲得するようにすることでエラーが無くなる
cxl.hのエラーの回避
![Page 40: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/40.jpg)
Simulation Run
% make vsim-run <= #1 HDL Simulationcd sim && vsim -do vsim.tcl……# Compiling top levelModelSim> s <= シミュレーションを開
始
……# Loading ./pslse/afu_driver/src/veriuser.slで待ちになるので、違うターミナルで
# make pslse-run <= #2 PSLSE
もう一つ違うターミナルで
# make sim-run ARGS=128 <= #3 Application
![Page 41: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/41.jpg)
3つのプロセスで実現
AFU is done.memcpy successful.
INFO:127.0.0.1 client disconnect from afu0.0 context 0INFO:Sending reset to AFUDEBUG:afu0.0:JOB code=0x80 ea=0x0000000000000000DEBUG:afu0.0:JOB doneINFO:Stopping clocks to afu0.0
HDL Simulation EnvironmentModelSim ASE 10.4d
PSLSE
Application
![Page 42: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/42.jpg)
Application
![Page 43: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/43.jpg)
Application : memcpy
struct cxl_afu_h *afu;afu = cxl_afu_open_dev((char*) (DEVICE));cxl_afu_attach(afu, (__u64) wed0);cxl_mmio_map(afu, CXL_MMIO_BIG_ENDIAN);
// ポーリングでAFUの終了待ちwhile (!wed0->status) { cxl_mmio_read64(afu, MMIO_ADDR, &rc);}
cxl_mmio_unmap(afu_h); // MMIOをアンマップ
cxl_afu_free(afu_h); // AFUを開放
capi-streaming-framework/host/app/src/example.cpp
![Page 44: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/44.jpg)
cxl_afu_open_dev
#define DEVICE "/dev/cxl/afu0.0d"
// 戻り値のafuは、ファイル識別子のようなもの
struct cxl_afu_h *afu;
// cxl_afu_open_dev関数でデバイスをオープンする
afu = cxl_afu_open_dev ((char*) (DEVICE));
https://github.com/ibm-capi/pslse/wiki/Programming-with-libcxl
![Page 45: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/45.jpg)
cxl_afu_attachstruct wed *wed0 = NULL;posix_memalign ((void **) &(wed0), CACHELINE_BYTES, sizeof(struct wed));wed0->status = 0;wed0->size = copy_size;wed0->source = source;wed0->destination = destination;
// デバイスと接続し、動作を開始
cxl_afu_attach(afu, (__u64) wed0);
https://github.com/ibm-capi/pslse/wiki/Programming-with-libcxl
![Page 46: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/46.jpg)
cxl_afu_attach
struct wed { __u8 volatile status; // 7 downto 0 __u8 wed00_a; // 15 downto 8 __u16 wed00_b; // 31 downto 16 __u32 size; // 63 downto 32 __u64 *source; // 127 downto 64 __u64 *destination; // 191 downto 128 __u64 wed03; // 255 downto 192 …. __u64 wed15; // 1023 downto 960};
https://github.com/ibm-capi/pslse/wiki/Programming-with-libcxl
![Page 47: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/47.jpg)
cxl_mmio_map// MMIOをマップする
cxl_mmio_map (afu, CXL_MMIO_BIG_ENDIAN)) < 0);
https://github.com/ibm-capi/pslse/wiki/Programming-with-libcxl
![Page 48: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/48.jpg)
ポーリングによる処理待ち
uint64_t rc;
// ポーリングによる処理待ち
while (!wed0->status) { cxl_mmio_read64(afu, MMIO_ADDR, &rc); printf("Response counter: %lu\n", rc);}
https://github.com/ibm-capi/pslse/wiki/Programming-with-libcxl
![Page 49: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/49.jpg)
参考:cxl_mmio_xxxx
64ビットアクセスと32ビットアクセス
cxl_mmio_read64(afu, 0, &data64)cxl_mmio_write64(afu, 8, data64)
cxl_mmio_read32(afu, 20, &data32)cxl_mmio_write32(afu, 28, data32)
https://github.com/ibm-capi/pslse/wiki/Programming-with-libcxl
![Page 50: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/50.jpg)
参考:割り込みによる処理待ち
struct cxl_event intr;
// ブロッキング処理の場合
cxl_read_event(afu, &intr); <= ここがブロッキング
handle_event(intr); // interrupt handler
while (cxl_pending_event(afu)) {
cxl_read_event(afu, &intr);
handle_event(intr); // interrupt handler
}
https://github.com/ibm-capi/pslse/wiki/Programming-with-libcxl
![Page 51: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/51.jpg)
参考:割り込みによる処理待ち
struct cxl_event intr;
// ノン・ブロッキング処理の場合
while (cxl_pending_event(afu)) {
cxl_read_event(afu, &intr);
handle_event(intr); // interrupt handler
}
https://github.com/ibm-capi/pslse/wiki/Programming-with-libcxl
![Page 52: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/52.jpg)
cxl_mmio_unmap/cxl_afu_free
// MMIOのアンマップ
cxl_mmio_unmap (afu);
// デバイスの開放
cxl_afu_free (afu);
https://github.com/ibm-capi/pslse/wiki/Programming-with-libcxl
![Page 53: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/53.jpg)
libcxl => pslse
![Page 54: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/54.jpg)
libcxl <=> pslse
cxl_afu_open_dev _pslse_connect _pslse_open
_psl_loop
HDL Simulationサーバー機能
capi-streaming-framework/sim/pslse/libcxl/libcxl.c
cxl_afu_attachcxl_mmio_attachcxl_mmio_readxxx
pthread生成
![Page 55: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/55.jpg)
_pslse_loop
// コマンド待ちのループ処理
while (afu->opened) { _delay_1ms(); // Send any requests to PSLSE over socket if (afu->int_req.state == LIBCXL_REQ_REQUEST) _req_max_int(afu); if (afu->attach.state == LIBCXL_REQ_REQUEST) _pslse_attach(afu); if (afu->mmio.state == LIBCXL_REQ_REQUEST) {
capi-streaming-framework/sim/pslse/libcxl/libcxl.c
![Page 56: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/56.jpg)
_pslse_loop続き
switch (afu->mmio.type) { case PSLSE_MMIO_MAP: _mmio_map(afu); break; case PSLSE_MMIO_WRITE64: _mmio_write64(afu); break; case PSLSE_MMIO_WRITE32: _mmio_write32(afu); break; case PSLSE_MMIO_READ64: case PSLSE_MMIO_READ32: /*fall through */ _mmio_read(afu); break; Default: break; }
capi-streaming-framework/sim/pslse/libcxl/libcxl.c
![Page 57: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/57.jpg)
Top Testbench
![Page 58: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/58.jpg)
Simulation : Top Testbench
top
AFUPLIベースモデル
afu_driverpslse
libcxl
App
capi-streaming-framework/sim/pslse/afu_driver/verilog/top.v
![Page 59: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/59.jpg)
Top Testbenchの続き
PLIベースモデル:afu_driverの実態 => PLI関数
$afu_init; // サーバー機能の初期化
$register_clock(...); // クロックの登録
$register_control(...); // 各信号線の登録
$register_mmio(...);$register_command(...);$register_rd_buffer(...);$register_wr_buffer(...);$register_response(...);
![Page 60: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/60.jpg)
PLIベースモデル : afu_driver//サーバー機能の初期化
$afu_init; => afu_init
// クロックの登録
$register_clock(); => register_clock
// 各信号線の登録
$register_control(); => register_control$register_mmio(); => register_mmio$register_command(); => register_command$register_rd_buffer(); => register_rd_buffer$register_wr_buffer(); => register_wr_buffer$register_response(); => register_response
capi-streaming-framework/sim/pslse/afu_driver/src/afu_driver.c
![Page 61: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/61.jpg)
AFU
![Page 62: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/62.jpg)
AFU : memcpy RTL (VHDL)
AFUFRAME
ControlMMIO
DMA CU
CAPI Streaming Framework
capi-streaming-framework/accelerator/rtl
PSL
![Page 63: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/63.jpg)
CU : memcpy
CU
FIFO
FSM
cu_in cu_out
type cu_in is record cr : cr_in; start : std_logic; wed : wed_type; id : unsigned(DMA_ID_WIDTH - 1 downto 0); read : dma_read_response; write : dma_write_response;end record;
type cu_out is record done : std_logic; read : dma_read_request; write : dma_write_request;end record;
capi-streaming-framework/accelerator/rtl/cu.vhd
![Page 64: FPGAアクセラレータの作り方 (IBM POWER+CAPI編)](https://reader031.fdocuments.net/reader031/viewer/2022022414/5873d2861a28ab9d168b673d/html5/thumbnails/64.jpg)
おしまい