System management on VPX: Leveraging VITA 46.11 for VPX plug-in modules

4In November 2013, after several years of work, VITA 46.11 was adopted as a Draft Standard for Trial Use in the VPX community. The draft standard defines components, interfaces, mechanisms, and general infrastructure to support the implementation of an interoperable management subsystem within VPX that enables features such as inventory, sensor, and diagnostic management, as well as system configuration and recovery. In the absence of VITA 46.11, such features, if present in VPX systems, would be implemented as part of the application(s), rather than as a standardized layer serving the application(s). VITA 46.11 aims to enable, for the system management aspects of applications, many of the same benefits that COTS hardware has long provided: lower cost and quicker application development and system upgrades.

VITA 46.11 addresses two main levels of system management. The lowest level is the individual front-loading plug-in module, where the integrated management entity is called an Intelligent Platform Management Controller (IPMC). The next level up is the overall chassis, including all the modules and other management-visible facilities (such as fan trays) it contains. The management entity at this level is the Chassis Manager, which supervises the chassis and its constituent elements, in addition to representing the chassis to upper layers of management. In VITA 46.11, those upper management layers are represented by a logical System Manager.

This article focuses on the IPMC level of VITA 46.11, introducing the functionality and responsibilities of this level. It also covers possible approaches to implementing this level using a generic hardware + firmware component that provides the core of an IPMC, suitable for adaptation to the needs of a particular module and associated application(s).

How do the system management levels in VITA 46.11 communicate?

Figure 1 shows these levels and the System Manager interface, which is typically implemented over an Ethernet link to the Chassis Manager. Each module in a VITA 46.11 compliant system includes an IPMC, which monitors the operation of that module and represents the management aspects of the module to the Chassis Manager. The main connection between the Chassis Manager and the IPMCs in the chassis is the I2C-based System IPMB (Intelligent Platform Management Bus), which is optionally dual redundant. Figure 1 represents the logical architecture; VITA 46.11 can be applied in any compatible VPX-based system.

Figure 1: Overall architecture for VITA 46.11 system management.
(Click graphic to zoom)

How does VITA 46.11 leverage ATCA management (and its IPMI foundation)?

The VITA 46.11 working group chose to leverage the management infrastructure in PICMG’s AdvancedTCA (ATCA) architecture rather than start from scratch. This choice accelerated development of the standard and will hasten availability and maturity of VITA 46.11-compliant products because the management controllers can be derived from corresponding mature ATCA controllers.

Effective collaboration with PICMG was a key factor in implementing this approach. PICMG graciously agreed that substantial portions of the VITA 46.11 standard could be derived (with attribution) from the corresponding sections of the ATCA specification. PICMG also agreed to share materials related to its management interoperability testing, which has occurred in dozens of PICMG-organized interoperability workshops, starting in 2002. The VITA 46.11 working group has already started adapting ATCA interoperability test plans for use with VITA 46.11.

ATCA’s management layer is based on the Intelligent Platform Management Interface (IPMI), which is an openly published management architecture that is widely used in the PC and server industries for hardware management (IPMB is part of the IPMI architecture). Similarly, VITA 46.11 is based on IPMI.

IPMI provides a rich infrastructure for defining and monitoring analog and digital sensors in an implementation-independent and consistent way. These facilities allow independently implemented elements of a VPX system from different suppliers (including chassis vendors, module vendors, and system integrators) to be monitored by a single Chassis Manager. That Chassis Manager has a unified view of the state of the chassis and all the analog and digital sensors that its elements include.

How are VITA 46.11 IPMCs different from ATCA IPMCs?

One key difference results from the distinct needs of VPX applications for system management. For some VPX applications, the traditional approach of integrating system management into the application is preferred. Therefore, VITA 46.11 management is optional, unlike the mandatory ATCA management layer.

Even among applications that need a separate management layer, the extent and sophistication of the needed layer varies. In response, VITA 46.11 defines two tiers of functionality. Tier 1 IPMCs and Chassis Managers have minimal facilities, but can be simpler and less expensive to implement. Tier 2 components deliver more functionality in the management layer, allowing greater reuse of those facilities across applications.

In addition, VITA 46.11 omits some aspects of the ATCA architecture that are not relevant to VPX. For instance, VPX modules are not designed to be hot swapped in a live system, so the extensive facilities supporting such operations in ATCA are not needed, simplifying the management layer for VPX.

Also, VITA 46.11 defines a number of VPX-specific facilities to optimize the management layer for VPX needs. For instance, there is a collection of mandatory sensors that allows that layer to quickly and consistently summarize the overall status of all the modules in a chassis, including their voltage and temperature status. Also, the IPMI commands that are directly leveraged from ATCA have their own set of VSO IPMI command numbers, so that VSO can evolve those commands independently from PICMG, if necessary.

Finally, there are some valuable supplementary facilities for ATCA IPMCs that have not yet been explicitly mapped into a VPX context. One of these is an implementation-independent framework for upgrading the firmware of IPMCs in a system. The framework is defined in a PICMG specification called HPM.1. It can be straightforwardly implemented, as is, in VPX IPMCs. Perhaps the VITA 46.11 working group will do an explicit mapping in the future, but in the meantime, it seems reasonable to take advantage of the existing HPM.1 architecture, which is essentially universally implemented in ATCA IPMCs.

Can VPX IPMCs be connected to an on-module Ethernet plane?

Many VITA 46.11-compliant VPX modules will have connections to one or more Ethernet-based communication planes in the chassis. It would be desirable to allow the IPMCs on those modules to connect directly to such planes, complementing their System IPMB connectivity. Fortunately, there is a well-established ecosystem and governing open specifications that enable an IPMC to share access to Network Controllers (NCs) that interface to an in-chassis Ethernet on behalf of the main payload of a module. This area is not addressed in the current revision of VITA 46.11, but these facilities can potentially be applied in VPX modules where this need exists.

One key specification in this area is NC-SI, the Network Controller Sideband Interface specification, which is maintained by the Distributed Management Task Force ( If an NC that provides Ethernet connectivity for the main module payload is NC-SI-compliant, the module’s IPMC can make a “sideband interface” connection to the NC using a 10/100 Mbps RMII connection. A PICMG specification called HPM.2 can be used to establish interoperable integration of LAN attach features into a VPX IPMC, enabling IPMC firmware upgrades, serial over LAN, and other applications to benefit from the LAN attach connection. Figure 2 shows this arrangement for a LAN-attached IPMC.

Figure 2: With standards-based extensions beyond VITA 46.11, a LAN-attached IPMC can share one or more Ethernet-based communication planes with payload CPU(s).
(Click graphic to zoom)

How can VPX modules be equipped quickly with compliant, interoperable IPMCs?

One key to cost-effective IPMC integration into a VPX module is to start with an existing implementation and integrate a generic IPMC core to match the specific needs of a particular VPX module design. This approach, which is widely and successfully used in the ATCA ecosystem, allows an IPMC implementer to defer to a specialized partner for thorough knowledge of the relevant governing specifications and the corresponding interoperability testing.

One example of this class of generic, but adaptable, IPMC cores is the Pigeon Point BMR-A2F-VPX IPMC solution, which is a reference hardware/firmware design based upon the SmartFusion (A2F) intelligent mixed signal FPGA that is already widely used in the ATCA domain. Leveraging the commonality between VPX and ATCA management, multiple VPX module vendors have already been delivering VPX modules that integrate an ATCA-derived IPMC, including Pigeon Point’s corresponding ATCA solution, BMR-A2F-ATCA.

Figure 3 provides a block diagram for a SmartFusion-based generic IPMC core. IPMC firmware runs on the integrated Cortex-M3 ARM32 microcontroller while the integrated FPGA fabric customizes the hardware for system management needs and optionally implements board-specific FPGA logic. The microcontroller subsystem also includes needed interfaces, such as I2C ports, serial ports, and (if relevant to a given module) an Ethernet interface for LAN attach purposes. SmartFusion’s analog subsystem reduces the need for external components to address analog sensing needs.

Figure 3: Example generic core of a VITA 46.11 IPMC, suitable for cost-effective adaption to the needs of a specific VPX module.
(Click graphic to zoom by 2.9x)

The benefits of a system management layer for VPX are gaining increasingly wide recognition. Adopting a proven management solution for that subsystem of a VPX module can save time and money and preserve scarce development resources for value added functionality of the module. With careful implementation choices, VITA 46.11 based system management for VPX can deliver a standardized implementation of the hardware platform management layer that saves development and integration time and improves interoperability. The architecture of this layer also preserves the flexibility for system developers and integrators to decide how to partition the system management function between the standardized layer and their application subsystems.

Mark Overgaard is the Founder and CTO of Pigeon Point Systems. He is a leader in the VITA 46.11 working group and actively contributed to the development of VITA 46.11. Within PICMG, he chairs the HPM.x subcommittee and participates in numerous others.

Pigeon Point Systems 760-757-2304