Hardware/Software Co-design of Schedulers for Real Time ...
Transcript of Hardware/Software Co-design of Schedulers for Real Time ...
![Page 1: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/1.jpg)
Hardware/Software Co-design of Schedulers for
Real Time Systems
Jorge Ortiz
CommitteeDavid Andrews, Chair
Douglas NiehausPerry Alexander
![Page 2: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/2.jpg)
Presentation OutlineBackground
Prior work in hybrid co-designFPGA hardware/software co-designScheduling
RTFPGA Project OverviewDesign approach & functionalityDesign implementation plan
Scheduler ImplementationHybrid co-design for KURT-Linux event schedulerSystem properties
Testing and AnalysisFunctionality validation of designData Stream testingConclusions and future work
![Page 3: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/3.jpg)
Contributions
DesignMitchell Trope, Sweatha RaoImplementationSweatha Rao, Jorge OrtizTestingMitchell Trope, Sweatha Rao, Jorge Ortiz
![Page 4: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/4.jpg)
Background
![Page 5: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/5.jpg)
Prior Work in Hybrid Co-design
Average-case enhancementsWorst-case scenariosPlatform architectures for real time systemsSystems that support real-time constraints
![Page 6: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/6.jpg)
System Co-DesignParallel systems
Multi-processor, FPGA, System-on-Chip, ASIC, Reconfigurable systems
Hybrid systemsExploit recursive patternsIncrease quality of serviceMeet real-time constraints
Real time operating systemsHardware support for the operating systemOS function migration
![Page 7: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/7.jpg)
FPGA Hardware/Software Co-design
Desired system propertiesTimeliness, concurrency, liveness, interfaces, heterogeneity, reliability, reactivity, predictability and safetyAverage-case, worst case scenarios
System flexibility and performanceReconfigurableNo loss in performance
![Page 8: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/8.jpg)
Scheduling
How to allocate which resources to whom and for how longHandling real-time eventsDesire for finer granularity for servicing interruptsOverhead costs due to interrupt handling and event schedulingDesign trade-offs
![Page 9: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/9.jpg)
RTFPGA Project Overview
![Page 10: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/10.jpg)
Project Motivation
Provide higher resolution and finer granularity for event handlingMinimize overhead processingMigrating key functions into hardwareTime keeping hardware supportHardware/software co-design of event scheduler for real-time hybrid operating system
![Page 11: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/11.jpg)
Design ApproachMigrate shadow timer into FPGAImplement scheduler bookkeeping operationsProvide processing for enqueuing and dequeuing eventsAllow entering, sorting and deletion of eventsEnable access to all queued eventsMove scheduling algorithm into the hardware
![Page 12: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/12.jpg)
Design Functionality
FPGA stores event information in local queueRun scheduling algorithmIdentify the next event to runSends event time to a match registerFree-counting hardware clock provides interrupt generation
![Page 13: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/13.jpg)
Design Implementation PlanInterface ported KURT-Linux to FPGA componentsCreate FPGA-mapped registersCreate basic event queue storage, interface and functionalityImplement event schedulerDeliver FIQ interrupts to CPUSolve concurrency issues of hardware-based implementation
![Page 14: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/14.jpg)
Scheduler Implementation
![Page 15: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/15.jpg)
Hybrid Co-design for Kurt-Linux Event Scheduler
Forward event requests from OS to hardware through command registerIdentify request type and event information:execution time and reference pointerService request on event schedulerRaise interrupt flag upon execution timeKURT-Linux runs appropriate ISR with retrieved event information
![Page 16: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/16.jpg)
Data Flow Chart in the RTFPGA Scheduler
Request to FPGASend request to schedulerIdentify requestAdd or delete in Block RAM storageSchedule events in parallelQueue expired events for CPU to read
![Page 17: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/17.jpg)
RTFPGA Scheduler Modules
![Page 18: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/18.jpg)
RTFPGA Scheduler Modules
Register MapperInterface to all registersInterface to internal functionality in the FPGA board Modular and portableTwo operational blocks - read and writeTiming, event information, scheduler commands, debugging information.
![Page 19: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/19.jpg)
RTFPGA Scheduler Modules
Memory ManagerControl storage and handling of Block RAM-implemented event queueBlock RAM Address generation for event additionService Scheduler requestsDirty bits indicate usage of Block RAM addresses
![Page 20: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/20.jpg)
RTFPGA Scheduler Modules
Block RAMDual-read/write port synchronous Block RAM FPGA storage memoryFirst port for Memory Manager requestsSecond port for continuously polling information in BRAM addresses for scheduling and searching purposes
![Page 21: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/21.jpg)
RTFPGA Scheduler Modules
UtimeShadow of KURT Utime clock implementationSet for microsecond resolutionProvide match register for next scheduled event timeCreate FIQ interrupts to CPU
![Page 22: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/22.jpg)
RTFPGA Scheduler ModulesQueue Minimum
Earliest Deadline First algorithm Reads event values output by Block RAM Output information for next event to be scheduledSend appropriate control signals
EDF pseudo-code
STARTSET next_event = max valueLOOPIF last deleted value = next_event
GOTO STARTIF last expired event = next_event
GOTO STARTIF no elements in queue THEN GOTO STARTELSE
READ new data from BRam event queueIF data = event (dirty bit is set)
READ scheduled timeIF scheduled time < next_event
next_event = scheduled timeEND IF
END IFEND IFGOTO LOOP
![Page 23: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/23.jpg)
RTFPGA Scheduler Modules
Queue DeleteContinuously receive polled information from Block RAMPerform a deletion using linear search & comparisonFound or lost signal sent back to Memory ManagerCheck for concurrency with interrupt generation and event queue popping
![Page 24: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/24.jpg)
RTFPGA Scheduler Modules
FIFO Block RAMFIFO queue for expired events that have not been read yet by CPUKeep track of number of expired eventsDe-queue until empty
![Page 25: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/25.jpg)
Inter-Module Interfaces
![Page 26: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/26.jpg)
System Properties
High interdependency of modulesConcurrency of deletions and interrupts solved by linear approachHighly cohesive scheduling algorithmTight coupling due to system requirements
![Page 27: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/27.jpg)
Testing and Analysis
![Page 28: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/28.jpg)
Functionality validation of design
Initial setup bootstraps FPGA timer shadowCheck for:
Event additionEvent deletionEvent scheduling
![Page 29: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/29.jpg)
Event Addition Timing Diagram
![Page 30: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/30.jpg)
Event Deletion Timing Diagram
![Page 31: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/31.jpg)
EventSchedulingTimingDiagram
![Page 32: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/32.jpg)
Integrated Scheduler Test1. Initial Set up2. Add Event1, Add Event2,
Add Event3, Add Event4, Add Event5
3. Find Minimum4. Delete Event2, Delete
Event35. Find Minimum6. Read current time from
FPGA timer registers (jiffy, jiffy_u)
7. Write event_time = jiffy + offset
8. Write ref_ptr = loop variable
9. Repeat the above three steps for loop variable = 0 to 2
10. Wait on jiffy + large offset
11. WHILE fifo_ref_ptr != x"00000000" (queue not empty)
12. Read fifo_ref_ptr(confirm all the three timers expired)
13. END WHILE14. Read jiffy & jiffy_u
![Page 33: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/33.jpg)
Data Stream Kernel Interface Tests
0
1000
2000
3000
4000
5000
0 2 4 6 8 10 12 14 16 18 20 22
Execution time delay (us)
Num
ber o
f eve
nts
With Hardw are Components
Without
Timer Execution Delay for Software RTOS and Hardware/Software RTOS
![Page 34: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/34.jpg)
Data Stream Kernel Interface TestsThe Software-based RTOS showed expected better performance
Lag between CPU and FPGA timers of 2-4 microsecondsFIQ handler schedules an IRQ, downgrading the interrupt priority
![Page 35: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/35.jpg)
Data Stream Kernel Interface Tests[root@george dski]$./dskihists -i 900 -f 4 -h 1 -n 100Waiting for 900 seconds...Name of the histogram: HIST_TIMER_EXEC_DELAY Number of events logged: 10000 Minimum Value: 0 Maximum Value: 7137329 Lower Bound: 0 Upper Bound: 100
Num of Buckets: 102
![Page 36: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/36.jpg)
Stability and Bounded Execution Time
Maximum execution delay for a software-based timer was in the range of millions of microsecondsIn the hardware version, all events in all tests were executed within the time delay range of 23 microsecondsHardware delay attributed to startup delays, worst case scenario checking, interrupt servicing, communication costs across ports
![Page 37: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/37.jpg)
Event handling under different loads
Increase of 6.4% for events handled with a zero microsecond delay1.7% increase for all events handled within one microsecond12.6% increase after the serial port polling delay between 10-12 microseconds
FPGA Event handling
0
20
40
60
80
100
120
1 3 5 7 9 11 13 15 17 19 21 23 25
Time in microseconds
% o
f Eve
nts
Han
dled
10K events100 K Events
![Page 38: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/38.jpg)
Device Utilization (V2P)
EDF scheduling functionLogic utilization: 11% SLICES, 17% of LUT, 22% Block RAM, Gate Count of 405,637Device utilization: 24% SLICES
Total FPGA SLICE Utilization83% or2510 out of 3008
![Page 39: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/39.jpg)
Conclusions and Future Work
![Page 40: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/40.jpg)
System behaviorWith Hardw are Components Aggregate
performance characteristics of both systems are comparableRobust base for hardware based event scheduler correctness, behavior and performance
0
2000
4000
6000
8000
10000
12000
1 3 5 7 9
11
13
15
17
19
21
23
Execution Delay (us)
Nu
mb
er
of
Ev
en
ts
Without
![Page 41: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/41.jpg)
System characteristics
Modular and Portable, IP encapsulatedFlexible and reconfigurableCOTS Linux environmentTested in different architectures, ADI Engineering's 80200EVB and the Xilinx Virtex-II Pro
![Page 42: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/42.jpg)
Future Work
Thread SchedulingHardware-based ThreadsHybrid ThreadsSmarter scheduler functionalityOnly necessary interruptions to CPU
![Page 43: Hardware/Software Co-design of Schedulers for Real Time ...](https://reader031.fdocuments.net/reader031/viewer/2022013101/61d20e873bd7131be9228523/html5/thumbnails/43.jpg)
Questions