Implementing system management in OpenVPX

The high-energy campaign to complete the OpenVPX standard has been complemented by a parallel effort to define a generic system management layer for use in OpenVPX and other VPX systems. The resulting VITA 46.11 architecture and corresponding implementation considerations are presented.

System management in OpenVPX (VITA 65) refers to the combination of software, hardware, and firmware responsible for administrative tasks associated with maintaining an OpenVPX system. Such functions include sensor monitoring, hardware inventory management, and firmware installation/upgrades. Historically, this set of functions, which is present in some form within any substantial VME-based system, has been implemented as part of the application layer. There has not been a distinct, VITA-architecture-defined layer that handles it.

With increased emphasis on interoperability, reduced integration effort, and time-to-market in the OpenVPX initiative, this layer needs to be architected and specified so that system integrators can combine platform elements for their applications as quickly and efficiently as possible, while implementing the level of such management that is suitable for their applications.

These challenges are being addressed within the VITA Standards Organization (VSO) through development of VITA 46.11, the System Management on VPX standard, which is currently in the early draft state. OpenVPX allocates pins in backplane slots and on modules for system management connections and mandates VITA 46.11 compliance on those pins if they are used. VITA 46.11 can be applied to any compatible VPX-based architecture.

The following discussion introduces the VITA 46.11 architecture, including the levels of management and the tiers within those levels that it defines. Possible approaches to implementing VITA 46.11 at the module level are also presented.

Levels of management for OpenVPX in VITA 46.11

VITA 46.11 identifies three levels of management: module, chassis, and system. At the module level, an Intelligent Platform Management Controller (IPMC) handles the local module management responsibilities, representing that module to the Chassis Manager. Using an I2C-based Intelligent Platform Management Bus (IPMB) link, the Chassis Manager monitors the collection of modules in a chassis and represents the entire chassis to a System Manager. The System Manager is a logical entity that is typically linked to the Chassis Manager via some higher-speed connection such as Ethernet; it monitors and supervises the operation of one or more chassis that combine to form an OpenVPX-based system.

VITA 46.11 defines the responsibilities and interfaces of the IPMC and Chassis Manager blocks, but defers definition of the System Manager to the application. Figure 1 shows this architecture with two example IPMC-equipped modules and the Chassis Manager monitoring them.

21
Figure 1: Overall architecture for VITA 46.11 system management

Here is a simple example of how the VITA 46.11 facilities could contribute to the operation of a real OpenVPX system. Each OpenVPX module in the system would have one or more temperature sensors, perhaps monitoring key temperature-sensitive sites on that module. For each of those sensors, the module developer or system integrator would define temperature thresholds of increasing severity for higher temperatures, based on the temperature operating profile for the device(s) at a given site on the module. Any temperature measurement that crosses one of those thresholds would trigger an event message to the Chassis Manager. By monitoring and integrating such event messages, the Chassis Manager could identify the need to change the speed of the fan(s) for all or a subset of the modules in the chassis, for instance, and monitor the effects of such changes on the temperatures in the chassis.

Like AdvancedTCA, VITA 46.11 leverages the Intelligent Platform Management Interface (IPMI), which is widely used in the PC and server industry for hardware platform management facilities. For example, IPMI provides a rich infrastructure for defining and monitoring analog and digital sensors in an implementation-independent and consistent way. These facilities allow independently implemented elements of an OpenVPX system from different suppliers (including chassis vendors, module vendors, and system integrators, for instance) to be monitored by a single Chassis Manager that has a unified view of the state of the chassis and all the analog and digital sensors that its elements include.

Functional tiers provide architectural flexibility

One challenge for VITA 46.11 system management is to provide the appropriate extent of these services to fit the needs of a given application. Different applications and different system integrators can have very different views regarding the partitioning of management functions between application layers and underlying infrastructure layers. VITA 46.11 addresses these challenges by defining functionality tiers for both the IPMC and the Chassis Manager: tentatively three tiers for each level. This approach allows chassis and module suppliers, as well as their customers, to choose the appropriate tier level for the management infrastructure layer, while still gaining the interoperability and cost effectiveness that result from standardization.

For instance, the tier 1 IPMC provides minimal management functionality, such as inventory data and a few simple sensors, but is designed to interoperate successfully on a module in a chassis with other modules that include more sophisticated (higher tier) IPMCs and a Chassis Manager. Furthermore, the tier 1 IPMC is being defined so that it can be implemented with no firmware at all – potentially just with logic in a flash-based FPGA, for instance.

A tier 1 IPMC could be a good choice for a simple module or for a module where avoiding firmware might simplify formal certification. Alternatively, modules with tier 1 IPMCs might be chosen by a system integrator who decides that the great majority of the system management layer should be implemented as part of the application, not by an underlying infrastructure.

Tier 1 IPMCs have disadvantages, however. For instance, their support for analog and digital sensors is severely restricted: As currently planned, tier 1 IPMCs will support only a handful of simple sensors, and the Chassis Manager will have to poll every one of those sensors to get updates on state changes. Higher-tier IPMCs will have full-function sensor capabilities (among other substantial benefits), enabling much more effective operation of the platform management layer.

Leveraging AdvancedTCA’s hardware platform management layer

Another key decision in the VITA 46.11 initiative has been to leverage the widely used hardware platform management layer in PICMG’s proven AdvancedTCA framework. This decision allows the OpenVPX community to take advantage of AdvancedTCA’s years of specification development and field experience, while still adapting the AdvancedTCA management architecture to the different needs and constraints that characterize OpenVPX applications. For instance, OpenVPX modules, by architectural choice, are not hot-swappable in a live system, which allows considerable simplifications in the system management architecture.

Adapting the AdvancedTCA management architecture for the OpenVPX context yields a second benefit as well: minimizing the needed investments for OpenVPX vendors who also develop products in the AdvancedTCA form factor. Such vendors can potentially spread the benefits of management investments across both communities.

Implementation options for IPMC on an OpenVPX module

One way to implement a VITA 46.11 management controller, especially a higher-tier IPMC (that is, above tier 1), is with a generic microcontroller. The CPU in such a device implements the controller firmware and uses the available integrated peripherals (such as voltage and temperature monitors) to provide key management data.

If the microcontroller includes an Ethernet port, it can potentially connect with in-chassis Ethernet, such as an OpenVPX Control Plane, for substantial performance benefits in IPMC firmware upgrades and other operations.

Another way to implement a full-function VITA 46.11 IPMC is with an intelligent mixed signal FPGA, such as Actel’s SmartFusion device. The microcontroller and analog subsystems of such an FPGA implement the IPMC firmware and analog sensors, possibly with significantly more capabilities in the analog area; the 10/100 Mbps Ethernet interface can implement a LAN connection. Figure 2 provides a high-level block diagram of an example VITA 46.11 full-function IPMC based on an intelligent mixed signal FPGA.

22
Figure 2: Example VITA 46.11 IPMC based on a SmartFusion intelligent mixed signal FPGA
(Click graphic to zoom by 1.9x)

The FPGA fabric in such an IPMC adds customizability, both for management architecture additions – such as an IPMI-defined interface between the IPMC and a main processor on the module or extra I2C ports – and for board-specific logic that might otherwise require a separate programmable logic device.

Implementing an example VITA 46.11 IPMC

Creating a compliant and interoperable VITA 46.11 IPMC is no small project, however. One challenge is that the standard itself is not yet complete. In addition, the specifications leveraged by VITA 46.11 (such as AdvancedTCA hardware platform management and IPMI) encompass more than 1,000 pages of management-focused specification content. Most OpenVPX module developers who want to support VITA 46.11 choose to use an existing management controller implementation, such as one from the Pigeon Point Board Management Reference (BMR) family. If the existing controller supports the requirements of an AdvancedTCA IPMC, the likelihood is high that it can support VITA 46.11 IPMC requirements with some level of firmware updates once the VITA 46.11 standard is finalized.

In addition to the normal management functions implemented by the microcontroller and analog and FPGA subsystems of the example intelligent mixed signal FPGA, this IPMC can implement advanced facilities based on a connection with a network controller that is part of the payload domain of the module (that is, the portion of the module not focused on system management). Through a connection to an Ethernet-based OpenVPX Control Plane, for instance, such an IPMC can support the following:

  • Serial Over LAN (SOL): Access via a Control Plane LAN to serial ports that are either on the IPMC or on the major processor(s) implemented on the module: Such LAN-based serial port access can be a big benefit compared to attaching physical serial cables to every serial port of interest in a system.
  • Upgrading IPMC firmware or IPMC-accessible programmable logic devices via LAN: For modules and systems that implement such upgrades, using a LAN connection can yield 15x or more speedups, compared to doing them via the IPMB links.

System management for OpenVPX is catching on

The benefits of a system management layer for OpenVPX are gaining increasingly wide recognition. Adopting a proven management solution for that subsystem of an OpenVPX module can save time and money and preserve scarce development resources for value-added functionality of the module. With careful implementation choices, VITA 46.11-based system management for OpenVPX can deliver a standardized implementation of the hardware platform management layer that saves development and integration time, plus offers improved interoperability. This layer also preserves the flexibility for system developers and integrators to decide how to partition the system management function between the standardized layer and their application subsystems.

Mark Overgaard is founder and President of Pigeon Point Systems (PPS), now an Actel company. Mark is a leader in PICMG technical subcommittees, including those addressing hardware platform management of AdvancedTCA, AdvancedMC, and MicroTCA. He also actively participates in the VITA 46.11 working group. Prior to founding PPS in 1997, Mark was VP, Engineering at Lynx Real-Time Systems and TeleSoft. He can be contacted at mark@pigeonpoint.com.

Pigeon Point Systems 831-438-1565 www.pigeonpoint.com