Benchmarking for [Physical] Synthesis

download Benchmarking  for [Physical] Synthesis

of 31

  • date post

  • Category


  • view

  • download


Embed Size (px)


Benchmarking for [Physical] Synthesis. Igor Markov and Prabhakar Kudva The Univ. of Michigan / IBM. In This Talk …. Benchmark ing vs benchmarks Benchmarking exposes new research Qs Why industry should care about benchmarking - PowerPoint PPT Presentation

Transcript of Benchmarking for [Physical] Synthesis

  • Benchmarking for [Physical] SynthesisIgor Markov and Prabhakar KudvaThe Univ. of Michigan / IBM

  • In This Talk Benchmarking vs benchmarksBenchmarking exposes new research QsWhy industry should care about benchmarkingWhat is (and is not) being done to improve benchmarking infrastructureNot in this talk, but in a focus groupIncentives for verifying published workHow to accelerate a culture change

  • BenchmarkingDesign benchmarksData model / representation; InstancesObjectives (QOR metrics) and constraintsAlgorithms, methodologies; ImplementationsSolvers: dittoEmpirical and theoretical analyses, e.g.,Hard vs easy benchmarks (regardless of size)Correlation between different objectivesUpper / lower bounds for QOR, statistical behavior, etcDualism between benchmarks and solversFor more details, see

  • Industrial BenchmarkingGrowing size & complexity of VLSI chipsDesign objectives Area / power / yield / etcDesign constraintsTiming / FP + fixed-die partitions / fixed IPs / routability / pin access / signal integrityCan the same algo excel in all contexts?Sophistication of layout and logic motivate open benchmarking for Synthesis and P&R

  • Design TypesASICsLots of fixed I/Os, few macros, millions of standard cellsDesign densities : 40-80% (IBM)Flat and hierarchical designsSoCsMany more macro blocks, coresDatapaths + control logicCan have very low design densities : < 20%Micro-Processor (P) Random Logic Macros(RLM)Hierarchical partitions are LS+P&R instances (5-30K)High placement densities : 80%-98% (low whitespace)Many fixed I/Os, relatively few standard cellsNote: Partitioning w Terminals DAC`99, ISPD `99, ASPDAC`00

  • Why Invest in BenchmarkingAcademiaBenchmarks can identify / capture new research problemsEmpirical validation of novel researchOpen-source tools/BMs can be analyzed and tweakedIndustryEvaluation and transfer of academic researchSupport for executive decisions (which tools are relatively week & must be improved)Open-source tools/BMs can be analyzed and tweakedWhen is an EDA problem (not) solved?Are there good solver implementations?Can they solve existing benchmarks?

  • Participation / Leadership NecessaryActivity 1: Benchmarking platform / flowsActivity 2: Establishing common evaluatorsStatic timing analysisCongestion / yield predictionPower estimationActivity 3: Standard-cell librariesActivity 4: Large designs w bells & whistlesActivity 5: Automation of benchmarking

  • Activity 1: Benchmarking PlatformBenchmarking platform: a reasonable subset ofdata modelspecific data representations (e.g., file formats)access mechanisms (e.g., APIs)reference implementation (e.g., a design database)design examples in compatible formatsBase platforms available (next slide)More participation necessaryregular discussionsadditional tasks / features outlined

  • Common Methodology PlatformSynthesis (SIS, MVSIS)Placement(Capo, Dragon, Feng Shui, mPl,)Common Model(Open Access?)Blif Bookshelf formatBlue Flow exists, Common model hooks: To be Done

  • Placement Utilities input in the GSRC Bookshelf formatFormat convertersLEF/DEF BookshelfBookshelf Kraftwerk (DAC98 BP, E&J)BLIF(SIS) BookshelfEvaluators, checkers, postprocessors and plottersContributions in these categories are welcome

  • Placement Utilities (contd)Wirelength Calculator (HPWL)Independent evaluation of placement resultsPlacement PlotterSaves gnuplot scripts ( .eps, .gif, )Multiple views (cells only, cells+nets, rows,)Probabilistic Congestion Maps (Lou et al.)Gnuplot scriptsMatlab scripts better graphics, including 3-d fly-by views.xpm files ( .gif, .jpg, .eps, )

  • Placement Utilities (contd)Legality checkerSimple legalizerLayout GeneratorGiven a netlist, creates a row structureTunable %whitespace, aspect ratio, etcAll available in binaries/PERL at Most source codes are shipped w Capo

  • Activity 2: Creating EvaluatorsContribute measures/analysis tools for:Timing AnalysisCongestion/YieldPowerAreaNoise.

  • Challenges for Evaluating Timing-Driven OptimizationsQOR not defined clearlyMax path-length? Worst set-up slack?With false paths or without?...Evaluation methods are not replicable (often shady)Questionable delay models, technology paramsNet topology generators (MST, single-trunk Steiner trees)Inconsistent results: path delays < gate delaysPublic benchmarks?...Anecdote: TD-place benchmarks in Verilog (ISPD `01)Companies guard netlists, technology parametersCell libraries; area constraints

  • Metrics for Timing + ReportingSTA non-trivial: use PrimeTime or PKSDistinguish between optimization and evaluationEvaluate setup-slack using commercial tools Optimize individual nets and/or pathsE.g., net-length versus allocated budgetsReport all relevant dataHow was the total wirelength affected?Were per-net and per-path optimizations successful?Did that improve worst slack or did something else?Huge slack improvements reported in some 1990s papers, but wire delays were much smaller than gate delays

  • Benchmarking Needs for Timing Opt.A common, reusable STA methodology High-quality, open-source infrastructureFalse paths; realistic gate/delay modelsMetrics validated against phys. synthesisThe simpler the better, but must be good predictorsBuffer insertion profoundly impacts layoutThe use of linear wirelength in timing-driven layout assumes buffers insertion (min-cut vs quadratic)Apparently, synthesis is affected too

  • Vertical BenchmarksTool flow Two or more EDA tools, chained sequentially (potentially, part of a complete design cycle)Sample contexts: physical synthesis, place & route, retiming followed by sequential verificationVertical benchmarksMultiple, redundant snapshots of a tool flow sufficient info for detailed analysis of tool performanceHerman Schmit @CMU is maintaining a resp. slot in the VLSI CAD BookshelfSee flat gate-level netlistsLibrary information ( < 250nm)Realistic timing & fixed-die constraints

  • Infrastructure NeedsNeed common evaluators of delay / powerTo avoid inconsistent / outdated resultsRelevant initiatives from Si2OLA (Open Library Architecture)OpenAccess For more info, see http://www.si2.orgStill: no reliable public STA toolSought: OA-based utilities for timing/layout

  • Activity 3: Standard-cell LibrariesLibraries carry technology informationImpact of wirelength delays increases in recent technology generationsCell characteristics must be compatibleSome benchmarks in the Bookshelf use 0.25m and 0.35m librariesGeometry info is there, + timing (in some cases)Cadence test library?Artisan libraries?Use commercial tools to create librariesProlific, Cadabra,

  • Activity 4: Need New BenchmarksTo Confirm / Defeat Tool TuningData on tuning from the ISPD03 paper Benchmarking for Placement, Adya et al.Observe thatCapo does well on Cadence-Capo, grid-like circuitsDragon does well on IBM-Place (IBM-Dragon)FengShui does well on MCNC benchmarksmPL does well on PEKOThis is hardly a coincidenceMotivation for more / better benchmarksP.S. Most differences above have been explained, all placers above have been improved

  • Activity 4: Large Benchmark has large designsMay be a good starting point use vendor tools to create blif files (+post results)Note: there may be different ways to convert A group of design houses (IBM, Intel, LSI, HP) is planning a release of new large gate-level benchmarks for layoutProbably no logic information

  • Activity 5: Benchmarking AutomationRigorous benchmarking is laborious. Risk of errors is highHow do we keep things simple / accessible?Encapsulate software management in an ASPWeb uploads for binaries and source in tar.gz w MakefilesWeb uploads for benchmarksGUI interface for NxM simulations; tables created automaticallyGUI interface for composing tool-flows; flows can be saved/reusedDistributed back-end includes job schedulingEmail notification of job completionAll files created are available on the Web (permissions & policies)Anyone can re-run / study your experiment or interface with it

  • Follow-on Action PlanLooking for volunteers to -test Bookshelf.exeParticularly, in the context of synthesis & verificationContact: Igor Create a joint benchmarking group from industry and academiaContact: Prabhakar Regular discussionsDevelopment based on common infrastructure