Alveo Card Out-of-Band Management Specification for Server BMC … · 2020-06-27 · Server BMC....

Alveo Card Out-of-BandManagement Specificationfor Server BMC

User Guide

UG1363 (v1.1) February 14, 2020

https://www.xilinx.com

Revision HistoryThe following table shows the revision history for this document.

Section Revision Summary02/14/2020 Version 1.1

General updates Revised outline of document

Table 1: Slave Adresses and Corresponding Protocols Revised table to remove slave address 0x64 and 0x66andinclude slave address 0x50.

Chapter 3: IPMI FRU Implementation Changed title

Temperature Limits Revised table to be card specific.

Alveo PCIe Information Included to capture the PCIe information for Alveo U200,U250, U280, and U50 cards.

Satellite Controller Firmware Version Added to provide information on the latest SC firmware.

07/17/2019 Version 2019.1

Initial release. N/A

Revision History

UG1363 (v1.1) February 14, 2020 www.xilinx.comOOB Management Specification 2Send Feedback


https://www.xilinx.com/about/feedback/document-feedback.html?docType=User_Guides&docId=UG1363&Title=Alveo%20Card%20Out-of-Band%20Management%20Specification%20for%20Server%20BMC&releaseVersion=1.1&docPage=2

Table of ContentsRevision History...............................................................................................................2

Chapter 1: Introduction.............................................................................................. 5Satellite Controller.......................................................................................................................5

Chapter 2: Card Management Interfaces......................................................... 7Out-of-Band Communication.................................................................................................... 8

Chapter 3: IPMI FRU Implementation...............................................................13Block Write................................................................................................................................. 14Block Read..................................................................................................................................15

Chapter 4: I2C/SMBus Implementation and Protocol Recap ..............16Read Byte................................................................................................................................... 17Read Word..................................................................................................................................17Block Read..................................................................................................................................17Block Write Block Read............................................................................................................. 18

Chapter 5: I2C/SMBus Commands.......................................................................19Command Code Definition.......................................................................................................19FRU Data.....................................................................................................................................20Block Write................................................................................................................................. 21Block Read..................................................................................................................................21Maximum DIMM Temperature................................................................................................21Board Temperature...................................................................................................................22Board Power Consumption......................................................................................................22(MSP432) Firmware Version.....................................................................................................23FPGA Die Temperature............................................................................................................. 24Maximum QSFP Temperature................................................................................................. 24

Chapter 6: PLDM Implementation...................................................................... 26Terminus Locator PDR.............................................................................................................. 26Numeric Sensor PDR.................................................................................................................27




Sensor Auxiliary Names PDR................................................................................................... 32Temperature Limits...................................................................................................................34Sample PLDM Transaction....................................................................................................... 34

Appendix A: Additional Resources and Legal Notices............................. 38Xilinx Resources.........................................................................................................................38Documentation Navigator and Design Hubs.........................................................................38Alveo™ PCIe Information..........................................................................................................38Satellite Controller Firmware Version.....................................................................................39References..................................................................................................................................40Please Read: Important Legal Notices................................................................................... 40




Chapter 1

IntroductionThis document describes out-of-band (OOB) support available for the U200, U250, U280, andU50 Alveo™ Data Center cards. OOB support is provided by satellite controller firmware thatruns on TI's MSP432 MCU. The underlying protocol supported is platform level data model(PLDM) over the management component transport protocol (MCTP) over the systemmanagement bus (SMbus).

Satellite ControllerThe satellite controller firmware runs on TI’s MSP432 device and the underlying RTOS isFreeRTOS. The satellite controller firmware is an essential component of Alveo cardmanagement, providing in-band and OOB communication mechanisms. The MSP432 device,sensor, and peripherals reside on the auxilary power domain.




Figure 1: Satellite Controller Block

LTC 3884

I2C0

Server BMCDebug UART

Console

FPGA

SMBus/I2C

UART

UART I2C1

DIMMDIMMDIMMDIMM

SE 98SE 98

EEPROM

LM96036

I2CMUX PCA9536

I2CMUX

QSFP 0

QSFP 1

SI 570

Ch 0

Ch 1

Ch 2

Ch 3

Ch 2

Ch 1

Ch 0

MSP 432 MCU

Satellite Controller

Out of Band ManagementFor Eng Bring-up Only Disabled in Release

Card Management Controller

CMC

Voltage Regulator

Host OS

PCIe

In-Band Management

Fan Controller for FGPA

Board temperature

sensors

For logs & factory data

I/O expander for BSL

DIMM temperature

sensors

OSC

Temperature sensor

Temperature sensor

SE 98I2C2

X23545-112119

Chapter 1: Introduction




Chapter 2

Card Management InterfacesXilinx® Alveo™ Data Center cards achieve card management using two interfaces:

• OOB communication channel: The satellite controller communicates with the serverbaseboard management controller (BMC) via SMBus/I2C interface to provide OOB cardmanagement functionalities.

• In-band communication channel: The card management controller (CMC) communicates withthe host server via the PCIe® interface to provide in-band management features. The CMCfirmware, running in MicroBlaze™, and satellite controller firmware, running in MSP432,communicates via the UART channel using a Xilinx proprietary protocol. All sensor data ispassed on to the CMC by the satellite controller firmware through this in-band channel.

The following figure shows the high-level block functional diagram of Alveo cards.




Figure 2: Satellite Communication Firmware (inside MSP432) and Card ManagementController (inside FPGA)

XILINX FPGA

Main Power Domain

3.3V AUX Power Domain User DSA

UART

AXI Bridge

PCIe Shell

Xilinx DSASerialRX/TX

In-B

and

Chan

nel

Out

-of-B

and

Chan

nel

Devices Always On Devices

UART Satellite Controller

MSP432

I2C

PCIe SMBus

PCIe Host Baseboard Management Controller

I2C

PCIe Edge Connector

X23546-112119

Out-of-Band CommunicationWhen installed in a server, the satellite controller firmware communicates with server BMC. Themain purpose of OOB communication is to respond to requests that originate from server BMC.It uses this information to take action related to power and thermal management (i.e., to ramp-upfans or send requests to throttle down power consumption). The MSP432 and the sensors andperipherals it accesses reside on the AUX 3.3V always-on power domain.

OOB communication occurs via the physical medium of SMBus/I2C. The following table liststhree I2C slave addresses, each supporting different protocols/features.

Chapter 2: Card Management Interfaces




Table 1: Slave Adresses and Corresponding Protocols

I2C Slave Address (7-bit)I2C Slave

Address (8-bit)Protocol/Features Supported

0x50 0xA0 IPMI FRU data0x65 0xCA I2C Commands0x67 0xCE PLDM/MCTP

I2C FRU Only CommandSatellite controller firmware supports IPMI field replaceable unit (FRU) data read at I2C slaveaddress addr: 0x50 (0xA0 in 8-bit). For FRU data access, 2-byte addressing mode is supportedand the contents of FRU data are explained in UG1378 Alveo FRU Specfication.

I2C/SMBus CommunicationSatellite controller firmware supports I2C/SMBus protocol based OOB communication at I2CSlave address addr: 0x65 (0xCA in 8-bit). It provides support for server BMC that does notaccept PLDM or distributed management task force (DMTF) specifications. The followinginformation is exposed via I2C/SMBus protocol:

• FRU data information

• Thermal sensors such as FPGA, board, maximum DIMM, and maximum QSFP.

• Board power consumption

• MSP432 firmware version number

The following is a comprehensive list of all OOB commands supported by satellite controllerfirmware adhering to the PLDM over MCTP over SMBus Protocol.



https://www.xilinx.com/support/documentation/boards_and_kits/accelerator-cards/2019_1/ug1378-alveo-fru-data-specification.pdf



Table 2: Supported Custom I2C/SMBus Commands

CommandCode

SensorName

SMBusTransactio

n Type

Number ofResponse

BytesRequest Response

80h for write00h for read

FRU data Block writeread

Write: ACKwill be sent

by I2C driver.

Command code: 0x80Data Byte 0: FRU offset LSbyteData byte 1: FRU offset MSbyteData byte 2: length (< 256)Example: 0x01 0x10 0x20=>Get 32 bytes of FRU addressstarting at 0x0110

The FRU read request arrivein two or more parts:Part 1: Request to read thecommon header.Part 2 or later: Request toread actual FRU contents.

Note: Default requestcombination (i.e., when writeData bytes 0, 1 and 2 are all0x0), entire FRU contents willbe returned.

Read; lengthis set by write

command.

Command code: 0x0 Requested length of FRUdata from the start offset willbe returned.

01h MaximumDIMM

temperature

Read bytes 1 Command code: 0x01Data Byte(s): N/A

Byte 0: maximum DIMMtemperature value

Note: MSP432 calculates themaximum DIMMtemperature of all DIMMspresent and provide singlesensor information.

02h Boardtemperature

Read bytes 1 Command code: 0x02Data byte(s): N/A

Byte 0: Board temperature

03h Board powerconsumption

Read words 2 Command code: 0x03Data byte(s): N/A

uint_16 value;Byte 0: LS byteByte 1: MS byte

04h MSP432firmwareversion

(satellitecontroller)


Byte 0: Version;Byte1: Major revision;Byte2: Minor revision;Byte3: 0x0 (Reserved)

05h FPGA dietemperature


Byte 0: FPGA temperature

06h MaximumQSFP

temperature


Byte 0: Maximum QSFPtemperature

Note: MSP432 calculates theMAX QSFP temperature of allQSFP modules present andprovides single sensorinformation.

See Chapter 3: IPMI FRU Implementation for more Implementation level details.





PLDM Over MCTP Over SMBus ProtocolSatellite controller firmware supports the MCTP/PLDM protocol via I2C Slave address 0x67(0xCE in 8-bit). OOB implementation adheres to PLDM Base Specification (DSP240) and PLDMfor Platform Monitoring and Control Specification (DSP0248).

The following figure illustrates the PLDM over MCTP over SMBus binding specification stack.

Figure 3: PLDM Over MCTP Over SMBus Binding Specification Stack

Management Controller(BMC or Host Processor)

PLDM

Management Component Transport Protocol (MCTP)

MCTP over SMBus/I2CBinding

(MSP 432)

NC-SIMCTP

Control Protocol

SMBus/I2C

Application Layer

Protocol Layer

Transport Layer

Physical Layer

Device

X23542-011620

The following sensor readings are reported via PLDM OOB:

• FPGA temperature (fan controller remote temperature)

• Board temperature (fan controller local temperature )



https://www.dmtf.org/sites/default/files/standards/documents/DSP0240_1.0.0.pdf





• QSFP0 temperature

• QSFP1 temperature

The following PLDM commands are supported in the satellite controller firmware:

Table 3: List of Supported PLDM Commands and Descriptions

Command ID DescriptionSetTID 0x01 Sets terminus ID (TID) for a PLDM terminus.

GetTID 0x02 Returns the present TID setting for a PLDM terminus.

GetPLDMVersion 0x03 Returns the PLDM base specification versions that the PLDMterminus supports, as well as the PLDM type specificationversions supported for each PLDM type.

GetPLDMTypes 0x04 Enables management controllers to discover the PLDM typecapabilities supported by the PLDM terminus and get a listof the supported PLDM types.

GetPLDMCommands 0x04 Enables management controllers to discover the PLDMcommand capabilities supported by the PLDM terminus fora specific PLDM type and version, as a responder.

GetSensorReading 0x11 Returns the present reading and threshold event statevalues from a numeric sensor, as well as the operating stateof the sensor itself

GetSensorThresholds 0x13 Returns the present threshold settings for a PLDM numericsensor.

GetPDRRepositoryInfo 0x50 Returns information about the size and number of recordsin the PDR repository of a particular PLDM terminus andtime stamps that indicate the last time an update to therepository occurred.

GetPDR 0x51 Returns individual PDRs from a PDR repository. The recordis identified by the PDR record handle value that is passed inthe request. The command can also be used to dump all thePDRs within a PDR repository.

These PLDM commands are categorized into type 0 and type 2, as detailed in the following table.

Table 4: Supported Type 0 and Type 2 PLDM Commands

PLDM Type 0 Commands PLDM Type 2 CommandsSetTID (0x01) SetTID (0x01)

GetTID (0x02) GetTID (0x02)

GetPLDMVersion (0x03) GetSensorReading (0x11)

GetPLDMTypes (0x04) GetSensorThresholds (0x12)

GetPLDMCommands (0x05) GetPDRRepositoryInfo (0x50)





Chapter 3

IPMI FRU ImplementationSatellite controller firmware exposes FRU data via a dedicated I2C slave address 0x50 (0xA0 in8-bit). All FRU data is compliant with Intelligent Platform Management Interface (IPMI) FRUspecification [Ref 1]. Satellite controller firmware emulates the traditional EEPROM's FRU datawithin the firmware to enable server BMCs that are traditionally used to interface with a non-private EEPROM that resides in the same I2C bus, along with satellite controllers.

Accessing this FRU data follows I2C/SMBus block write block read, where block write provides a2-byte FRU offset (address byte 0 or LS Byte and address byte 1 MS Byte) and block readretrives FRU data. The SMBus transaction, with repeated start option, will be used to fetch allFRU data.

The maximum response bytes per transaction is 256 bytes, as set by the underlying I2C driver.This implies that to fetch a FRU data length of 300 bytes, the server BMC is expected to sendtwo repeated START transactions. For the first transaction, the satellite controller firmware sends256 FRU bytes. For the second transaction, 44 FRU bytes + 212 bytes of 0xFF are sent.

Figure 4: Random Read

Random Read

SDA Line

Device AddressStart

MSB

Write

Device Address

Start

1st, 2nd Word Address n

Read Stop

NO ACK

DATA nACKACKLSB ACK

R/W

X23543-112119

Format is as follows:

START, SA+W, addr-byte0, addr-byte1, RepeatedSTART, SA+R, BYTE0, BYTE1…… BYTEN, STOP

Where:

addr-byte0][addr-byte1] are FRU offsets (block writes)

and:

[BYTE0][BYTE1]…[BYTEN] are FRU data response (block reads)




Note: 2-byte FRU offset follows [LS Byte] [MS Byte].

Example of Read FRU Data Starting at Offset 0x0

• Block Write Operation: [N] 0x64 0x00 0x00

[I2C Bus Number = N ] [I2C Slave = 0x64] [FRU Offset LS Byte] [FRUOffset MS Byte]

• Block Read Operation: [N] 0x64

[I2C Bus Number = N ] [I2C Slave = 0x64]

Example of Read FRU Data Starting at Offset 50

• Block Write Operation: [N] 0x64 0x32 0x00

[I2C Bus Number = N ] [I2C Slave = 0x64] [FRU Offset LS Byte] [FRUOffset MS Byte]

• Block Read Operation: [N] 0x64

[I2C Bus Number = N ] [I2C Slave = 0x64]

Block WriteTable 5: Block Write, Server BMC Request

Server BMC RequestData Bytes [Byte 0] [Byte 1] [Byte 0] – FRU Offset LSB

[Byte 1] – FRU Offset MSB

Table 6: Block Write, Xilinx® Alveo™ Card Response

Xilinx Alveo Card ResponseData Bytes ACK sent by I2C Driver

Chapter 3: IPMI FRU Implementation




Block ReadTable 7: Block Read, Server BMC Request

Server BMC RequestData bytes NA [Byte 0] – FRU offset LSB

[Byte 1] – FRU offset MSB

Table 8: Block Read, Xilinx Alveo Card Response

Xilinx Alveo Card ResponseData bytes [Byte 0] [Byte 1] …. [Byte 255]] 256-byte FRU data

Chapter 3: IPMI FRU Implementation




Chapter 4

I2C/SMBus Implementation andProtocol Recap

The latest sensor information is stored locally in MSP432 satellite controller firmware and isexposed on-demand to server BMC via the OOB channel I2C/SMBus, at slave address 0x65. SMBus v2.0 Specification is followed for this implementation. Each sensor data is associated withan I2C command code as mentioned in Table 15: Supported I2C/SMBus Commands

Table 9: Key to Protocol

1 7 1 1 8 1 1

S Slave Address W A Data Byte A P

Table 10: SMBus Packet diagram element Key

Key DescriptionS Start Condition

Sr Repeated Start Condition

R Read (bit value of 1)

W Write (bite value of 0)

x Shown under a field indicates that the field is required to have the value of x

A Acknowledge (this bit position may be 0 for an ACK or 1 for a NACK

P Stop Condition

PEC Packet Error Code

□ Master-to-slave

■ Slave-to-master

... Continuation of protocol

Chapter 4: I2C/SMBus Implementation and Protocol Recap


http://smbus.org/specs/smbus20.pdf



Read ByteTable 11: Read Byte

1 7 1 1 8 1 1 7 1 1 8 1 1

S Slave Address W A Command Code A S Slave Address R A Data Byte A P

1

Read WordTable 12: Read Word

1 7 1 1 8 1 1 7 1 1 8 1 8 1 1

S Slave Address W A Command Code A S Slave Address R A Data Byte (Low) A Data Byte (High) A P

1

Block ReadTable 13: Block Read

1 7 1 1 8 1 1 7 1

S Slave Address W A Command Code A S Slave Address R ...

8 1 8 1 ... 8 1 1

Data Byte 1 A Data Byte 2 A ... Data Byte N A P

1





Block Write Block ReadTable 14: Block Write Block Read

1 7 1 1 8 1 8 1 ... 8 1 ...

S Slave Address R/W A Data Byte 1 A Data Byte 2 A ... Data Byte N A ...

1 7 1 1 8 1 8 1 ... 8 1

S Slave Address R/W A Data Byte 1 A Data Byte 2 A ... Data Byte N P





Chapter 5

I2C/SMBus CommandsTable 15: Supported I2C/SMBus Commands

Command orRegistered Value Command Name SMBus Transaction Type Number of Resp Bytes

0x80 FRU writeFRU read

Block writeBlock read

0Variable per request0x00

0x01 Maximum DIMM temperature Read byte 10x02 Alveo card temperature Read byte 10x03 Alveo card power consumption Read word 20x04 FW version (MSP432) Block read 40x05 Alveo FPGA die temperature Read byte 10x06 Maximum QSFP temperature Read byte 1

Note: Xilinx recommends waiting for 1–2 ms between any two I2C transactions. Without the delay,uninterrupted I2C operation isn’t guaranteed.

Details of each command are given in Command Code Definition.

Command Code DefinitionThe command codes are detailed in Table 15: Supported I2C/SMBus Commands. The seventh bit(0 based) represents read or write operation.

Table 16: Command Code Definition

1 7

R/W Command Code

• Write Operation: R/W bit = 1

• Read Operation: R/W bit = 0




FRU DataServer BMC uses block write block read to read the FRU data from the Xilinx® FPGA card. Forblock write, the server BMC uses register value 0x80 to write the 2-byte offset and 1-byte FRUdata length. This is followed by a block read where only the 0x00 register value is used, withoutany data bytes.

Note: 2-byte FRU offset follows [LS Byte] [MS Byte]

Example of Read 8 bytes Starting at Offset 0x0

• Block Write Operation: [N] 0x65 0x80 0x00 0x00 0x08

[ I2C Bus Number = N ] [I2C Slave = 0x65] [Command Code] [LS ByteOffset] [MS Byte Offset] [FRU Data Len]

• Block Read Operation: [N] 0x65 0x00

[ I2C Bus Number = N ] [I2C Slave = 0x65] [Command Code]

Example of Read 120 bytes starting at offset 100


[ I2C Bus # = N ] [I2C Slave = 0x65] [Command Code] [LS ByteOffset] [MS Byte Offset] [FRU Data Len]



Default Values Example

If the write operation gives all 0x00 values (i.e., FRU offset = 0x0 and FRU length = 0x0), onlythe starting 256-byte FRU information is returned as default.


[ I2C Bus Number = N ] [I2C Slave = 0x65] [Command Code] [LS ByteOffset] [MS Byte Offset] [FRU Data Len]



Chapter 5: I2C/SMBus Commands




Block WriteTable 17: Block Write, Server BMC Request

Server BMC RequestCommand Code 0x80

Data bytes [Byte 0] [Byte 1] [Byte 0] – FRU offset LSB[Byte 1] – FRU offset MSB[Byte 2] – FRU data length

Table 18: Block Write, Xilinx Alveo™ Card Response

Xilinx Alveo Card ResponseData Bytes ACK sent by I2C Driver

Block ReadTable 19: Block Read, Server BMC Request

Server BMC RequestCommand code 0x00

Data bytes N/A

Table 20: Block Read, Xilinx Alveo Card Response

Xilinx Alveo Card ResponseData bytes [Byte 0] [Byte 1] …. [Byte N] FRU data with a variable length from given offset

Maximum DIMM TemperatureThe Alveo cards have DIMMs in them and the number of DIMMs varies with different models.The primary motivation for server BMC to read DIMM temperature is to provide closed-loopthermal monitoring. The best way to expose the DIMM temperature is to provide maximumvalue of all the DIMM temperature values. MSP432 firmware keeps track of temperature valuesinternally for all the DIMMs present in a Alveo FPGA card, exposing only the maximum DIMMtemperature value to the server BMC. Server BMC uses command code 0x01 to read the maxDIMM temperature value. The response data from the Xilinx FPGA card is 1-byte temperaturedata (twos complement) and the range is –128°C to 127°C.





Table 21: Maximum DIMM, Server BMC Request


Data bytes N/A

Table 22: Maximum DIMM, Xilinx Alveo Card Response

Xilinx Alveo Card ResponseData bytes [Byte 0] 1-byte temperature data (2’s complement) and the range is

–128 °C to 127 °C (degree Celsius).For example:[Byte 0] = 0xFE presents –2°C[Byte 0] = 0x23 presents 35°C

Board TemperatureServer BMC uses register 0x02 to read the board temperature value. The response data from theXilinx FPGA card is 1-byte temperature data (twos complement) and the range is –128°C to127°C.

Table 23: Board Temperature, Server BMC Request


Data bytes N/A

Table 24: Board Temperature, Xilinx Alveo Card Response

Xilinx Alveo Card ResponseData bytes [Byte 0] 1-byte temperature data (twos complement) and the range

is –128°C to 127°C.For example:[Byte 0] = 0xFE presents –2°C[Byte 0] = 0x23 presents 35°C

Board Power ConsumptionServer BMC uses register 0x03 to read the current board power consumption value. Theresponse data from the Xilinx FPGA card is 2-byte power consumption data (LSB first), unit is inwatts (W).





Table 25: Board Power Consumption, Server BMC Request


Data bytes N/A

Table 26: Board Power Consumption, Xilinx Alveo Card Response

Xilinx Alveo Card ResponseData bytes [Byte 0] 2-byte temperature data in Watts.

For example:[Byte 0] [Byte 1] = 0x32 0x00 presents 50W (0x0032)

[Byte 0] [Byte 1] = 0x20 0x01 presents 288W (0x0120)

(MSP432) Firmware VersionServer BMC uses register 0x04 to read the current MSP432 firmware version, which followsxx.yy.zz formatting. The response data from the Xilinx FPGA card is 4 bytes.

Table 27: (MSP432) Firmware Version, Server BMC Request


Data bytes N/A

Table 28: (MSP432) Firmware Version, Xilinx Alveo Card Response

Xilinx Alveo Card ResponseData bytes [Byte 0]

[Byte 1][Byte 2][Byte 3]

4-byte firmware version data – LSB first[Byte 0] – Firmware version[Byte 1] – Major revision[Byte 2] – Minor revision[Byte 3] - ReservedFor example:V4.7.1 = 0x00 0x01 0x07 0x04V8.13.9 = 0x00 0x08 0x0D 0x08





FPGA Die TemperatureServer BMC uses register 0x05 to read the FPGA die temperature value. The response data fromthe Xilinx FPGA card is 1-byte temperature data (twos complement) and the range is –128°C to127°C.

Table 29: Max QSFP Temperature, Server BMC Request


Data bytes N/A

Table 30: Max QSFP Temperature, Xilinx Alveo Card Response


is –128 to 127 °C.For example:[Byte 0] = 0xFE presents –2°C[Byte 0] = 0x23 presents 35°C

Maximum QSFP TemperatureThe Alveo card comes with network interface (i.e., QSFP or SFP-DD) modules. The number ofSFP modules vary depending on the model. The primary incentive for server BMC to read theSFP temperature is to provide closed-loop thermal monitoring. The most effective way to exposethe SFP temperature is to provide the maximum value of all the SFP temperature values.MSP432 firmware internally tracks temperature values for all the SFP modules present in a Alveocard, exposing only the maximum SFP temperature value to server BMC.

Server BMC uses register 0x06 to read the maximum QSFP temperature value. The responsedata from the Xilinx FPGA card is 1-byte temperature data (twos complement) and the range is –128°C to 127°C.

Table 31: Maximum QSFP Temperature, Server BMC Request


Data bytes N/A





Table 32: Maximum QSFP Temperature, Xilinx Alveo Card Response


is –128°C to 127°C.For example:[Byte 0] = 0xFE presents –2°C[Byte 0] = 0x23 presents 35°C





Chapter 6

PLDM ImplementationThe latest sensor information is stored locally in MSP432 satellite controller firmware and isexposed on-demand to server BMC via OOB protocol PLDM at slave address 0x67 (0xCE in8-bit).

Sensor information is stored and represented in the platform descriptor record (PDR). PDRsprovide semantic information for sensors and effects their relationship to the entities that aremonitored or controlled. The satellite controller firmware has nine PDRs in total:

• Terminus locator PDR

• Numeric sensor PDR for FPGA temperature

• Numeric sensor PDR for board temperature

• Numeric sensor PDR for QSFP0 temperature

• Numeric sensor PDR for QSFP1 temperature

• Sensor auxiliary names PDR for FPGA temperature

• Sensor auxiliary names PDR for board temperature

• Sensor auxiliary names PDR for QSFP0 temperature

• Sensor auxiliary names PDR for QSFP1 temperature

For more details on PDR and the different types, please refer to the PLDM for PlatformMonitoring and Control Specification.

Terminus Locator PDRThe terminus locator PDR provides information that associates a PLDM terminus handle withvalues that uniquely identify the device or software that contains the PLDM terminus. The fieldspopulated in Terminus Locator PDR are listed in the following table.

Table 33: Populated Terminus Locator Fields

Field ValuePLDM_TERMINUS_HANDLE 0x0000

VALIDITY 0x1

Chapter 6: PLDM Implementation






Table 33: Populated Terminus Locator Fields (cont'd)

Field ValueCONTAINER_ID 0x7f7f

TERMINUS_LOCATOR_TYPE UID (0x0)

TERMINUS_LOCATOR_VALUESIZE 17

TERMINUS_INSTANCE 1

DEVICE_UID 62 bf a1 a4 c2 a4 11 e8 be 1fe0 3f 49 e8 97 a4

Numeric Sensor PDRThe numeric sensor PDR is primarily used to describe the semantics of a PLDM numeric sensorand includes factors to convert raw sensor readings to normalized units.

Numeric Sensor PDR for FPGA TemperatureTable 34: Numeric Sensor PDR for FPGA Temperature Fields

Field ValuePLDM_TERMINUS_HANDLE PLDM_TERMINUS_HANDLE

SENSOR_ID SENSOR_ID

ENTITY_TYPE ENTITY_TYPE

ENTITY_INSTANCE_NUMBER ENTITY_INSTANCE_NUMBER

CONTAINER_ID CONTAINER_ID

SENSOR_INIT SENSOR_INIT

SENSOR_AUXILIARY_NAMES_PDR SENSOR_AUXILIARY_NAMES_PDR

BASE_UNIT BASE_UNIT

UNIT_MODIFIER UNIT_MODIFIER

RATE_UNIT RATE_UNIT

BASE_OEM_UNIT_HANDLE 0

AUX_UNIT NONE (0x0)

AUX_UNIT_MODIFIER 0

AUX_RATE_UNIT NONE (0x0)

REL DIVIDED_BY (0x0)

AUX_OEM_UNIT_HANDLE 0

IS_LINEAR True

SENSOR_DATA_SIZE SINT16 (0x3)

RESOLUTION 1

OFFSET 0





Table 34: Numeric Sensor PDR for FPGA Temperature Fields (cont'd)

Field ValueACCURACY +/– 0.0%

PLUS_TOLERANCE 3

MINUS_TOLERANCE 3

HYSTERESIS 0

SUPPORTED_THRESHOLDS UPPER_THRESHOLD_FATAL (0x2) UPPER_THRESHOLD_CRITICAL (0x1)UPPER_THRESHOLD_WARNING (0x0)

THRESHOLD_AND_HYSTERESIS_VOLATILITY

None

STATE_TRANSITION_INTERVAL 0.0 sec

UPDATE_INTERVAL 0.10000000149011612 sec

MAX_READABLE 127

MIN_READABLE –40

RANGE_FIELD_FORMAT SINT16 (0x3)

RANGE_FIELD_SUPPORT FATAL_HIGH (0x5) CRITICAL_HIGH (0x3) NORMAL_MIN (0x2) NORMAL_MAX(0x1)

NOMINAL_VALUE 0

NORMAL_MAX 88

NORMAL_MIN 3

WARNING_HIGH 88

WARNING_LOW 0

CRITICAL_HIGH 97

CRITICAL_LOW 0

FATAL_HIGH 107

FATAL_LOW 0

Numeric Sensor PDR for Board TemperatureTable 35: Numeric Sensor PDR for Board Temperature Fields


SENSOR_ID 0x0002

ENTITY_TYPE 0x0051 (P/L PHYSICAL (0x0), ENTITY_ID 0x0051)

ENTITY_INSTANCE_NUMBER 0x0002

CONTAINER_ID 0x7f7f

SENSOR_INIT NO_INIT (0x0)

SENSOR_AUXILIARY_NAMES_PDR True

BASE_UNIT DEGREES_C (0x2)

UNIT_MODIFIER 0





Table 35: Numeric Sensor PDR for Board Temperature Fields (cont'd)

Field ValueRATE_UNIT NONE (0x0)


AUX_UNIT NONE (0x0)

AUX_UNIT_MODIFIER 0




IS_LINEAR True


RESOLUTION 1

OFFSET 0

ACCURACY +/– 0.0%

PLUS_TOLERANCE 3

MINUS_TOLERANCE 3

HYSTERESIS 0

SUPPORTED_THRESHOLDS UPPER_THRESHOLD_FATAL(0x2) UPPER_THRESHOLD_CRITICAL (0x1)UPPER_THRESHOLD_WARNING (0x0)


None



MAX_READABLE 127

MIN_READABLE –40


RANGE_FIELD_SUPPORT FATAL_HIGH(0x5) CRITICAL_HIGH (0x3) NORMAL_MIN (0x2) NORMAL_MAX(0x1)

NOMINAL_VALUE 0

NORMAL_MAX 80

NORMAL_MIN –40

WARNING_HIGH 80

WARNING_LOW 0

CRITICAL_HIGH 85

CRITICAL_LOW 0

FATAL_HIGH 125

FATAL_LOW 0





Numeric Sensor PDR for QSFP0 TemperatureTable 36: Numeric Sensor PDR for QSFPO Temperature Fields


SENSOR_ID 0x0002

ENTITY_TYPE 0x0051 (P/L PHYSICAL (0x0), ENTITY_ID (0x0051)


CONTAINER_ID 0x7f7f




UNIT_MODIFIER 0

RATE_UNIT NONE (0x0)


AUX_UNIT NONE (0x0)

AUX_UNIT_MODIFIER 0




IS_LINEAR True


RESOLUTION 1

OFFSET 0

ACCURACY +/– 0.0%

PLUS_TOLERANCE 3

MINUS_TOLERANCE 3

HYSTERESIS 0

SUPPORTED_THRESHOLDS UPPER_THRESHOLD_FATAL (0x2) UPPER_THRESHOLD_CRITICAL (0x1)UPPER_THRESHOLD_WARNING (0x0)


None



MAX_READABLE 127

MIN_READABLE –40



NOMINAL_VALUE 0

NORMAL_MAX 80





Table 36: Numeric Sensor PDR for QSFPO Temperature Fields (cont'd)

Field ValueNORMAL_MIN –40

WARNING_HIGH 80

WARNING_LOW 0

CRITICAL_HIGH 85

CRITICAL_LOW 0

FATAL_HIGH 125

FATAL_LOW 0

Numeric Sensor PDR for QSFP1 Temperature FieldsTable 37: Numeric Sensor PDR for QSFP1 Temperature Fields


SENSOR_ID 0x0004

ENTITY_TYPE 0x0051 (P/L PHYSICAL (0x0), ENTITY_ID (0x0051)


CONTAINER_ID 0x7f7f




UNIT_MODIFIER 0

RATE_UNIT NONE (0x0)


AUX_UNIT NONE (0x0)

AUX_UNIT_MODIFIER 0




IS_LINEAR True


RESOLUTION 1

OFFSET 0

ACCURACY +/– 0.0%

PLUS_TOLERANCE 3

MINUS_TOLERANCE 3

HYSTERESIS 0





Table 37: Numeric Sensor PDR for QSFP1 Temperature Fields (cont'd)

Field ValueSUPPORTED_THRESHOLDS UPPER_THRESHOLD_FATAL (0x2) UPPER_THRESHOLD_CRITICAL (0x1)

UPPER_THRESHOLD_WARNING (0x0)


None



MAX_READABLE 127

MIN_READABLE –40



NOMINAL_VALUE 0

NORMAL_MAX 80

NORMAL_MIN –40

WARNING_HIGH 80

WARNING_LOW 0

CRITICAL_HIGH 85

CRITICAL_LOW 0

FATAL_HIGH 90

FATAL_LOW 0

Sensor Auxiliary Names PDRThe sensor auxiliary names PDR may be used to provide optional information that name thesensor.

Sensor Auxiliary Names PDR for FPGA TemperatureTable 38: Sensor Auxiliary Names PDR for FPGFA Temperature Fields


SENSOR_ID 0x0001

SENSOR_COUNT 1

NAME_STRING_COUNT 1

NAME_LANGUAGE_TAG en-US

SENSOR_NAME FPGA temperature





Sensor Auxiliary Names PDR for Board TemperatureTable 39: Sensor Auxiliary Names PDR for Board Temperature Fields


SENSOR_ID 0x0002

SENSOR_COUNT 1

NAME_STRING_COUNT 1


SENSOR_NAME Board temperature

Sensor Auxiliary Names PDR for QSFP0 TemperatureTable 40: Sensor Auxiliary Names PDR for QSFP0 Temperature Fields


SENSOR_ID 0x0003

SENSOR_COUNT 1

NAME_STRING_COUNT 1


SENSOR_NAME QSFP-0 temperature

Sensor Auxiliary Names PDR for QSFP1 TemperatureTable 41: Sensor Auxiliary Names PDR for QSFP1 Temperature Fields


SENSOR_ID 0x0004

SENSOR_COUNT 1

NAME_STRING_COUNT 1


SENSOR_NAME QSFP-1 temperature





Temperature LimitsTable 42: Temperature Limits

Sensor Name Warning Limit Critical Limit Fatal LimitU200/U250

Device temperature 88°C 97°C 107°C

Board temperature 80°C 85°C 125°C

QSFP temperature1 80°C 85°C 90°C

U50

Logical device temperature2 88°C 97°C 107°C

Board temperature 80°C 85°C 125°C

QSFP temperature1 80°C 85°C 90°C

Notes:1. QSFP temperature limits may vary based on model and manufacturer. Refer to the following data sheet specific to the

card:• Alveo U200/U250• Alveo U280• Alveo U50

2. Logical device temperature is combination of device die temperature and the HBM temperature.

Note: The satellite firmware automatically shuts down the FPGA when the FPGA or QSFP temperatureexceeds the fatal limit.

Sample PLDM TransactionThis section examines a sample PLDM request and response message. For this example, the BMCon the server has an I2C address of 0x20 and the satellite controller has an I2C address of 0xCE.

PLDM RequestThe PLDM request originates from the server BMC and satellite firmware running in the MSP432receives this request via I2C port at address 0xCE.

The MCTP packet encapsulation and the different fields are explained in MCTP SMBus/I2CTransport Binding Specification.



https://www.xilinx.com/support/documentation/data_sheets/ds962-u200-u250.pdf

https://www.xilinx.com/support/documentation/data_sheets/ds963-u280.pdf






Figure 5: PLDM Request

7 6 5 4 3 2 1 0

Destination Slave Address

7 6 5 4 3 2 1 0

Command Code = MCTP = 0Fh

7 6 5 4 3 2 1 0

Byte Count

7 6 5 4 3 2 1 0

Source Slave Address0 1

MessageHeader

MessageData

PEC

MCTP Reserved Destination Endpoint ID Source Endpoint ID SOM

Header Version

EOMPacket

Sequence Number

TO Message Tag

IC Message Type Message Integrity Check

+0 +1 +2 +3

Byte 1 >

Byte 5 >

Byte 9 >

Byte N >

X23544-112219

A request sent from the BMC to satellite controller will look like the following:

Table 43: Request sent from Server BMC to Satellite Controller

Destination Slave Address Command Code Byte Count Source Slave Address67 0F 0C 21

MCTP Reserved Destination Source SOM, EOM, Pkt#HDR Version Endpoint ID Endpoint ID TO, MSG Tag

1 0 0 C8IC, MSG Type MSG header Message data, Message Integrity check

80, 02, 11, 01, 00, 001PEC29

The part in blue is the PLDM message that can be decoded, as explained in PLDM BaseSpecification.

Table 44: PLDM Message Payload

Byte 1 Byte 2 Byte 3 Byte 4

7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0

Rq

D RSVD

Instance ID HdrVer

PLDM Type PLDM Command Code PLDM Completion Code*

PLDM Message Payload (Zero or more bytes)

*The PLDM completion code is present only in PDM response messages.







The part in blue in the previous message decoded using the PLDM message scheme looks likethe following figure:

Table 45: PLDM Message Scheme

Byte 1 Byte 2 Byte 3 Byte 4Rq, D, rsvd, Instance Hdr Ver, PLDM PLDM Command PLDM Completion

ID Type Code Code80 2 11

PLDM Message Payload01, 00, 00

The PLDM completion code field applies to only PLDM responses and not PLDM requests.PLDM Command code 0x11 corresponds to the GetSensorReading and the Payload can be nowdecoded as detailed in the following table:

Table 46: PLDM Completion Codes

Type Request Data Value In Our Examplesuint16 Sensor ID 0x0001

bool8 rearmEventState 0x00

Now the satellite controller knows that the server BMC is requesting sensor reading with SensorID 0x01.

PLDM ResponseThe PLDM response from satellite firmware to server BMC is explained in this section.

The following table details the response for GetSensorReading:

Table 47: GetSensorReading Response

Type Request Data Value In Our Examplesenum8 completionCode 0x00

enum8 sensorDataSize 0x02

enum8 sensorOperationalState 0x00

enum8 sensoreventMessageEnable 0x00

enum8 presentState 0x01

enum8 previousState 0x00

enum8 eventState 0x01

uint16 presentReading 0x002A

The response that gets plugged into the PLDM message scheme looks like this:





Table 48: Response in PDLM Message Scheme

Byte 1 Byte 2 Byte 3 Byte 4Rq, D, rsvd, Instance Hdr Ver, PLDM PLDM Command PLDM Completion

ID Type Code Code0 2 11 0

PLDM Message Payload02, 00, 00, 01, 00, 01, 2A, 00

The PLDM message encapsulated inside MCTP response looks like this:

Table 49: PDLM Message in MCTP Response

Destination Slave Address10

Command Code0F

Byte Count12

Source Slave AddressCF

MCTP Reserved Destination Source SOM, EOM, Pkt#HDR Version Endpoint ID Endpoint ID TO, MSG Tag

1 0 0 C0IC, MSG Type MSG header Message data, Message Integrity check

1 00, 02, 11, 00, 02, 00, 00, 01, 00, 01, 2A, 00PECB4

The server BMC decodes the MCTP response it receives to know the sensor readings.





Appendix A

Additional Resources and LegalNotices

Xilinx ResourcesFor support resources such as Answers, Documentation, Downloads, and Forums, see XilinxSupport.

Documentation Navigator and Design HubsXilinx® Documentation Navigator (DocNav) provides access to Xilinx documents, videos, andsupport resources, which you can filter and search to find information. To open DocNav:

• From the Vivado® IDE, select Help → Documentation and Tutorials.

• On Windows, select Start → All Programs → Xilinx Design Tools → DocNav.

• At the Linux command prompt, enter docnav.

Xilinx Design Hubs provide links to documentation organized by design tasks and other topics,which you can use to learn key concepts and address frequently asked questions. To access theDesign Hubs:

• In DocNav, click the Design Hubs View tab.

• On the Xilinx website, see the Design Hubs page.

Note: For more information on DocNav, see the Documentation Navigator page on the Xilinx website.

Alveo™ PCIe InformationThe following table captures the PCIe information for Alveo U200, U250, U280, and U50 cards.

Appendix A: Additional Resources and Legal Notices


https://www.xilinx.com/support

https://www.xilinx.com/support

https://www.xilinx.com/cgi-bin/docs/ndoc?t=design+hubs

https://www.xilinx.com/cgi-bin/docs/rdoc?t=docnav



Table 50: Alveo PCIe Information

Card/Shell Vendor ID Device ID Subsystem VID Subsystem DIDU200 Golden 0x10EE 0xD000 0x10EE 0x000E

U200 XDMA and 2RP 0x10EE 0x5000

0x5001

0x5002

0x5003

0x10EE 0x000E

U200 QDMA 0x10EE 0x5010

0x5011

0x5012

0x5013

0x10EE 0x000E

U250 Golden 0x10EE 0xD004 0x10EE 0x000E

U250 XDMA & 2RP 0x10EE PF0=0x5004PF1=0x5005PF2=0x5006PF3=0x5007

0x10EE 0x000E

U250 QDMA 0x10EE PF0=0x5014PF1=0x5015PF2=0x5016PF3=0x5017

0x10EE 0x000E

U280 Golden 0x10EE 0xD00C 0x10EE 0x000E

U280 XDMA & 2RP 0x10EE 0x500C

0x500D

0x500E

0x500F

0x10EE 0x000E

U50 Golden 0x10EE 0xD020 0x10EE 0x000E

U50 XDMA & 2RP 0x10EE PF0=0x5020PF1=0x5021PF2=0x5022PF3=0x5023

0x10EE 0x000E

Satellite Controller Firmware VersionNot all features described in this document are available in older satellite controller firmware.Ensure the latest firmware is being used.

• U200/U250— 4.3.7

• U280— 4.3.6

• U50/U50DD— 5.0.25





ReferencesThese documents provide supplemental material useful with this guide:

1. Intelligent Platform Management Interface FRU Specification

2. Alveo FRU Data Specification (UG1378)

3. PLDM base specification

4. PLDM for platform monitoring and control specification

5. SMBus 2.0 Specification

6. MCTP SMBus/I2C Transport Binding Specification

7. Alveo U200 and U250 Data Center Accelerator Cards Data Sheet (DS962)

8. Alveo U280 Data Center Accelerator Cards Data Sheet (DS963)

9. Alveo U50 Data Center Accelerator Cards Data Sheet (DS965)

Please Read: Important Legal NoticesThe information disclosed to you hereunder (the "Materials") is provided solely for the selectionand use of Xilinx products. To the maximum extent permitted by applicable law: (1) Materials aremade available "AS IS" and with all faults, Xilinx hereby DISCLAIMS ALL WARRANTIES ANDCONDITIONS, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING BUT NOT LIMITED TOWARRANTIES OF MERCHANTABILITY, NON-INFRINGEMENT, OR FITNESS FOR ANYPARTICULAR PURPOSE; and (2) Xilinx shall not be liable (whether in contract or tort, includingnegligence, or under any other theory of liability) for any loss or damage of any kind or naturerelated to, arising under, or in connection with, the Materials (including your use of theMaterials), including for any direct, indirect, special, incidental, or consequential loss or damage(including loss of data, profits, goodwill, or any type of loss or damage suffered as a result of anyaction brought by a third party) even if such damage or loss was reasonably foreseeable or Xilinxhad been advised of the possibility of the same. Xilinx assumes no obligation to correct anyerrors contained in the Materials or to notify you of updates to the Materials or to productspecifications. You may not reproduce, modify, distribute, or publicly display the Materialswithout prior written consent. Certain products are subject to the terms and conditions ofXilinx's limited warranty, please refer to Xilinx's Terms of Sale which can be viewed at https://www.xilinx.com/legal.htm#tos; IP cores may be subject to warranty and support terms containedin a license issued to you by Xilinx. Xilinx products are not designed or intended to be fail-safe orfor use in any application requiring fail-safe performance; you assume sole risk and liability foruse of Xilinx products in such critical applications, please refer to Xilinx's Terms of Sale which canbe viewed at https://www.xilinx.com/legal.htm#tos.



https://www.intel.com/content/dam/www/public/us/en/documents/specification-updates/ipmi-platform-mgt-fru-info-storage-def-v1-0-rev-1-3-spec-update.pdf

https://www.xilinx.com/cgi-bin/docs/bkdoc?k=accelerator-cards;d=ug1378-alveo-fru-data-specification.pdf



http://smbus.org/specs/smbus20.pdf


https://www.xilinx.com/cgi-bin/docs/ndoc?t=data_sheets;d=ds962-u200-u250.pdf


https://www.xilinx.com/cgi-bin/docs/ndoc?t=data_sheets;d=ds965-u50.pdf

https://www.xilinx.com/legal.htm#tos





AUTOMOTIVE APPLICATIONS DISCLAIMER

AUTOMOTIVE PRODUCTS (IDENTIFIED AS "XA" IN THE PART NUMBER) ARE NOTWARRANTED FOR USE IN THE DEPLOYMENT OF AIRBAGS OR FOR USE IN APPLICATIONSTHAT AFFECT CONTROL OF A VEHICLE ("SAFETY APPLICATION") UNLESS THERE IS ASAFETY CONCEPT OR REDUNDANCY FEATURE CONSISTENT WITH THE ISO 26262AUTOMOTIVE SAFETY STANDARD ("SAFETY DESIGN"). CUSTOMER SHALL, PRIOR TO USINGOR DISTRIBUTING ANY SYSTEMS THAT INCORPORATE PRODUCTS, THOROUGHLY TESTSUCH SYSTEMS FOR SAFETY PURPOSES. USE OF PRODUCTS IN A SAFETY APPLICATIONWITHOUT A SAFETY DESIGN IS FULLY AT THE RISK OF CUSTOMER, SUBJECT ONLY TOAPPLICABLE LAWS AND REGULATIONS GOVERNING LIMITATIONS ON PRODUCTLIABILITY.

Copyright

© Copyright 2019-2020 Xilinx, Inc. Xilinx, the Xilinx logo, Alveo, Artix, Kintex, Spartan, Versal,Virtex, Vivado, Zynq, and other designated brands included herein are trademarks of Xilinx in theUnited States and other countries. The PowerPC name and logo are registered trademarks ofIBM Corp., and used under license. All other trademarks are the property of their respectiveowners.





Alveo Card Out-of-Band Management Specification for Server BMC … · 2020-06-27 · Server BMC....

Documents

Transcript of Alveo Card Out-of-Band Management Specification for Server BMC … · 2020-06-27 · Server BMC....