Post on 15-Jul-2018
©2017MIPIAlliance,Inc.
Shiou Mei Huang, Automotive ApplicationsMayank Mangla, ADAS Imaging ArchitectTexas Instruments
ADAS High Bandwidth Imaging Implementation Strategies
©2017MIPIAlliance,Inc.
AutomotiveImagingApplications
2
CompiledwithGoogleSearchImages
FrontCam
SV/Park Assist
Radar/Lidar
DMS
Fusion
CMS
SmartRearCam
©2017MIPIAlliance,Inc.
Challenges• HighBandwidthRequirements
– Fasterframerate– Highersensorresolution– Highdynamicrange(multipleexposuresperpixel)
• Multi-CameraSystems– Numerousimage/videoformats– Singleprocessortoprocessvideoinputsfromall
cameras– Needtodistinguishamongstvideostreams– Multi-modalfusion
www.ti.com/ADAS
• SoC DesignConstraints– Package:Minimalpincount– Routing:Signalintegrity– Lowercableusage
• AutomotiveQuality&Safety– Needtomaintainsignalintegrityfor
interfacedistance>20ft
©2017MIPIAlliance,Inc.
ExampleSolutions• FPDLinkDeserializer
– Quadhubchip– EachcameraconnectedthroughFPD-Linkcable– VCtagforindividualcamerastreams– Mux4camerasintoasingleCSI-2℠ stream
• VisionProcessor– RxmodulefullycompliantwithMIPICSI-2℠ spec1.0
• Future:32VirtualChannels– Upto6.0Gbps throughput– HWsupporttoparseCSI-2℠ streams– Highspeed,multichannelimageprocessor– Visionaccelerators:HWIPs+ProgrammableCore
4
www.ti.com/ADAS
DISPLAY
©2017MIPIAlliance,Inc.
WhyMIPICSI-2℠inAutomotive
5
www.ti.com/ADAS
§ Lowpower
§ Highspeedtransfers
§ Lowpincounto Flexibilityinchoosing1/2/3/4lane(s)
§ StandardizedvsProprietaryprotocolso Easytointerfaceawiderangeoftransmitters
andreceivers
§ Virtualchannels§ Allowsconnectionsofmultipledevicestothe
samebus
§ Errordetectionandrecoveryo Synccodeso ECCcodeso 16-bitsCRCChecksum
§ Safetyo Safetyrelatedmeta-dataalongwithpixeldata
©2017MIPIAlliance,Inc.
MIPICSI-2℠ LaneScalability
6
www.ti.com/ADAS
RECEIVER
Ø Inhighbandwidthscenarios,CSI-2℠ canbeconfiguredtosplitbytestreaminto1/2/3/4datalanes
Ø OnTx sidethestreamisdistributedacrossmultiplelanes.OnRxsideincomingpacketsaremergedintoasinglestream
Ø Eachlaneoperatesindependently,includingSoT frame,StartPacketcodeandEndPacketcode
©2017MIPIAlliance,Inc.
VirtualChannelsandDataTypes
7
www.ti.com/ADAS
Ø EachCSI-2℠ packethasa“DataIdentifier”field,whichspecifiesthepayloadDataType(DT)andVirtualChannel(VC)numberitcarries
Ø Differentdatatypesfromdifferentsourcescanbemergedintoasinglestream
Ø CSI-2℠ RxusesDTandVCfieldstodistinguishamongstdifferentpacketsandprocessthemaccordingly
Ø Examples:• Pixeldatavsembeddeddata• Datafromdifferentcamerasmultiplexedintoasingle
CSI-2℠ stream• Multi-exposureWDRwhereL/M/Spixelsarecomingin
asinglestream.
©2017MIPIAlliance,Inc.
Example– SurroundView
8
www.ti.com/ADAS
Ø 4xFullHDCameras– Throughputrequirement>3GbpsØ EachcamerasendsahighresolutionRAWstreamatahigh
framerateØ SurroundViewApplication(SRV)
4:1Deserializer Hub(DS90UB96x)• Synchronizesvideoinputsacrossmultiplecameras• Aggregatescamerastreamsfromfoursidesofthevehicle• TagseachcamerastreamwithuniqueVCandmetadata• Multiplexesintoasinglestream• SendsoverCSI-2℠ interfacetothehostprocessor
HostProcessor(TDAx)• Receivestheincomingstream• Parseseachpacket,identifiesthecamerastreambasedonVC• Obtains4independentvideostreaminseparatebufferqueues• AppliescomplexSRVimageprocessingalgorithms
©2017MIPIAlliance,Inc.
SurroundViewSolution
11
www.ti.com/ADAS
ChallengesØ HighDataRate– Upto3GbpsØ HighPinCount– TosupportmultiplecamerasØ LongChannelLength– Cameraslocatedfarawayfromthe
processorØ Sync– Allcamerasmustbesynchronizedtoeliminatemotion
artifacts
MIPICSI-2℠ totherescueØ 4-laneinterface forHighSpeedDataTransferØ SupportforVirtualChannelsandDataTypes
• Allowsmultiplecamerastreamstobemuxed intoasingleCSI-2℠ stream
• ProcessorcanidentifyaCSI-2℠ packetusingVCandDTinfo
Ø CSI-2℠ enabledSerDes chipsallowlongchannellengthoverFPDLink• Actas4-camerahub• Implementssyncacrossallcameras
©2017MIPIAlliance,Inc.
Example– Multi-ModalFusion
13
www.ti.com/ADAS
Lighting Rain
SnowFog/Smog
Night
Dirt
PassiveSensorCamera
ActiveSensorRadar,Lidar,Ultrasound
InfrastructureHigh-precisionMapV2XCommunication
MidandLongRangeAdaptiveCruiseControl,Emergencybraking,Fully/HighlyAutonomousDriving
ShortRangeandUltraShortRangeBlindSpot,CollisionAvoidance,LaneChangeAssist,PedestrianDetection,ParkAssist
ProximitySensorsOccupantDetection,GestureRecognition,DriverMonitoring
©2017MIPIAlliance,Inc.
FusionSolution
14
www.ti.com/ADAS
ChallengesØ HighDataRate– 1Gbps/Camera+2.4Gbps/RadarØ HighPINCount– TosupportlargenumberofdevicesØ ImagingFormatVariations– Needastandardprotocolforimaging
andnonimagingdevices
MIPICSI-2℠ totherescueØ 4-Laneinterface forHighSpeedDataTransfer.Ø AbilitytoMUXCamera/Radar/LIDARØ ProcessorcanidentifythesourceusingVCandDTØ CSI-2℠ specificationeasilyextendstononimagingdevices
likeRadar.Ø NodesignchangeneededattheprocessorRx
interfaceØ Serializer andDeserializer chipsforFPDlink
transmissioncanprocessRadarstreamexactlylikecamera.
©2017MIPIAlliance,Inc.
Fusion– SLAM(SimultaneousLocalizationandMapping)
15
www.ti.com/ADAS
6x-10xCameras
6x-10xRadar
1x-4xLIDARs
8x-12xUltrasonic
GPSIMU
SensorProcessing
SensorProcessing
SensorProcessing
SensorProcessing
SensorProcessing
Perception
Perception
Perception
Perception
Perception
FUSION• SensorFusion• Localization• Mapping
Maps
PLANNINGANDCONTROL• Pathplanning• Motionplanning• Vehiclecontrols
• Acceleration• Brake• Steering
FEEDBACKTODRIVER• Visualization/Display• Warnings
Drivermonitoring
Thermal/IR
©2017MIPIAlliance,Inc.
SummaryMIPICSI-2℠ isgainingpopularityinAutomotiveindustrybecauseofthefollowingbenefits
• HighBandwidth:Total6.0Gbps.Allows4x1080pcamerasonasingleProcessor• Lowpincount: Scalablefrom6- 12• MultiCamera/MultiModalArchitecture:EfficientuseofVirtualChannelandDataTypes• Quality:SerDes solutionsconvertsimagesignalfromdigitaltoanalogform.FPDlinkcabletransportsanalog
signaloverlongdistancewithoutdegradation• Safety:TDAxx processorshasadvancedcapabilityforerrordetection,recovery,andembedsafetydata
alongwithpixeldata• Future:32virtualchannels
WithmoreImagesensors,RadarandotherdevicesadoptingCSI-2℠,thetrendisexpectedtocontinueandgainmomentumintheyearstocome.
16
www.ti.com/ADAS