The integration of Intel Xeon D processors on COM-HPC Server-on-Modules from manufacturers such as congatec allows edge server installations to break free from the tight thermal constraints of air-conditioned server rooms. For the first time, they can now be installed anywhere where massive data throughput with lowest possible latencies is required – all the way to deterministic real time.
The world’s first COM-HPC Server-on-Modules with Intel Xeon D processors free edge servers from the shackles of air-conditioned server rooms. congatec offers the Intel Xeon D processor on COM-HPC Server Size E and Size D as well as COM Express Type 7. (© congatec) ▶
Edge servers process data at the edge of communication networks instead of in central clouds. This enables interaction with clients of all kinds without delay or in real time but presents manufacturers of server, networking and storage technologies with major challenges. Until now, they used to develop standardized rack solutions for their systems, with active ventilation concepts and powerful air-conditioning technology to control the thermal management of the racks and the air-conditioning in the server rooms. However, such an approach is often no longer a good fit for the edge server technology of today.
The American Society of Heating, Refrigerating and Air-Conditioning Engineers, or ASHRAE for short, has looked thoroughly at the question of how to best install edge server performance in harsh environments. So, from the perspective of the associated heating, cooling, ventilation and air-conditioning companies, there are already quite plausible recommendations on how to design edge data centers with high-performance air-conditioning and the best possible insulation to protect them from heat and cold.
The modular Server-on-Module approach makes it easy to develop dedicated edge servers with application-specific interface layouts using custom-designed carrier boards. (© congatec)
Freeing edge servers from the shackles of air-conditioning
However, ASHRAE proposes a maximum permissible temperature fluctuation of 20°C within one hour and a maximum of 5°C in 15 minutes for edge data centers. This requires complex air-conditioning technology and is therefore very difficult to implement. But not only that; adhering to these guidelines is near impossible, especially during maintenance work in edge data centers that are smaller than a telephone booth, because such solutions must be opened for maintenance at any ambient temperature. It is simply not possible to slip into such systems via a climate control chamber and quickly close the door again before carrying out maintenance work in the fully air-conditioned edge server room.
Edge servers and data centers operating in harsh environments therefore need system designs that can cope with greater temperature fluctuations and a much wider temperature range than the 0‑40°C that is common for indoor IT. In industrial environments, embedded system designs can be exposed to ambient temperatures ranging from an arctic -40°C to a searing hot 85°C. Which means each component must be hardened.
Rugged designs reduce air-conditioning costs
The most neuralgic point in the design of edge server, networking and storage technologies is the choice of processor technology. The decision that stands and falls with this choice is whether to follow the ASHRAE recommendations and invest massively in air-conditioning technology and insulation, which comes with high investment and operating costs for secondary energy. Or whether to develop systems that don’t need any of that because they work reliably even in extreme temperatures and can therefore be implemented in harsh environments much more cheaply – from factory installations to outdoor communications, video surveillance and other critical infrastructure equipment to servers in mobile systems ranging from trains and airplanes to self-driving shuttle buses in smart cities.
Thanks to the new Intel Xeon D processors, there is now a very powerful server technology that is qualified for use in extreme temperature ranges from -40°C to 85°C. Even ultra high-performance server designs are no longer constrained by the tight thermal restrictions of air-conditioned server rooms. Ultimately, they can be deployed wherever latency-free massive data throughput is required at the edge of the Internet of Things and in Industry 4.0 factories.
Edge server with three COM-HPC modules for extreme real-time workloads (© congatec).
Reference design for machine-learning AI clustering
COM-HPC based multi-module designs
COM-HPC edge server designs are not limited to single-module concepts. The standard also explicitly supports multi-module carriers with heterogeneous COM-HPC module configurations that integrate e.g. FPGAs or GPGPU accelerators. A mix of COM-HPC Server and COM-HPC Client modules on one board is also possible. For example, congatec is currently working with the University of Bielefeld and Christmann IT on an edge server design that combines different COM-HPC modules on one carrier board to process extreme real-time workloads in a multi-system design for machine-learning AI clustering of high dimensional data (self-organizing maps).
High system design demands
However, a server processor alone does not make a rugged edge server. Meeting the system design demands for harsh environments also requires extensive know-how. Every single component used must be qualified for this environment, and special requirements also apply for the circuit board and board design. Examples are special coatings that protect against condensation water and other environmental influences, or a high level of protection against extraneous electromagnetic and high-frequency signals that could hamper device performance.
Intel Xeon D processor-based COM Express Type 7 and COM-HPC Server Size D modules don’t just differ in size but also in pinout. (© congatec)
Developers of embedded computing technologies such as congatec have decades of experience in the design of such systems. They have long been integrating standard PC technologies such as Intel Core processors into embedded systems in a way that is suitable for industrial use. They know the requirements and certification standards of a wide range of industries inside out; and they are used to designing their systems for long-term availability in order to meet industry requirements and to be able to supply OEM solutions with identical board configurations for 7, 10 or 15 years. They also know that industrial applications differ significantly from standard system designs for the office environment, because industrial applications always require a greater or lesser degree of customization, which makes modular designs that deploy Computer-on-Modules the ideal way to develop boards. They have also learned that standardization is key, which is why they helped to create globally recognized standards for such modules.
Reaching the goal faster with standards
With the new COM-HPC Server specification in place, and the launch of the Intel Xeon D processors, this bundled expertise has now been transposed to industrial edge server designs. For the first time, developers have access to real products. The advantage of these new standardized COM‑HPC Server-on-Modules is that developers can integrate them into their custom carrier boards as application-ready embedded computing logic. This means they don’t have to concern themselves with basic processor technology issues, but only deal with the application-specific positioning of the board components and execute the interfaces at the right place on the carrier board. To this end, the PICMG standardization committee recently released the COM-HPC Carrier Design Guide. It provides essential guidelines for building interoperable and scalable customer-specific embedded computing platforms based on the new standard and also makes it easy for developers to understand the logic behind the standard.
Knowledge is power
Standard motherboard/motherboard designs usually only support standard interfaces onboard that are executed at the rear of the board (rear I/Os). As this doesn’t take industrial requirements into account, their suitability as edge servers for the Internet of Things is limited. And as a rule, they are neither designed for the extended temperature range of -40°C to +85°C nor do they guarantee long-term availability of 7 to 15 years. With Server-on-Modules, however, it is possible to use the mechanics of these form factors and design a carrier board that executes the desired interfaces where they are required. (© congatec)
To allow developers to dive quickly, easily and efficiently into the new design rules, congatec has opened an online and on-site training academy for COM-HPC Server and Client designs. Here, developers can get an expert-led introduction to the new world of high-end embedded and edge computing designs based on the new Computer-on-Module standard. The training program covers all mandatory and recommended design basics and best practice schematics of COM-HPC carrier boards and accessories such as high-end fanless cooling solutions for server designs up to 100 Watt or more. The feature-rich evaluation carrier boards for COM-HPC Server modules serve as a reference platform for learning how to implement Intel Xeon D processors. They leverage the full feature set of the standard, and developers can use them as platforms for further application development.
The congatec academy aims to train developers in all fundamental COM-HPC design basics – from PCB layer principles, power management rules and signal integrity requirements to component selection. Sessions with a special focus on communication interfaces provide guidance on how to avoid pitfalls in the challenging design of high-speed serial communications: from PCIe Gen 4 and 5 to USB 3.2 Gen 2 and USB 4 with Thunderbolt on USB-C to 100 Gigabit Ethernet, and also including the management of sideband signals for 10G / 25G / 40G / 100G Ethernet KR interfaces, which in COM-HPC must be deserialized on the carrier board. During these sessions it is also explained how best practice designs utilize interface standards such as eSPI, I²C and GPIOs.
An introduction to x86 firmware implementation – ranging from embedded BIOS to Board Management Controller and Module Management Controller features – complements the design-in training. And last but not least, there are sessions on verification and test strategies that address all challenges from initial carrier board design verification to mass production testing. With such a comprehensive training program, the congatec academy aims to make the design of rugged edge server technology as easy as possible. Needless to say, the company can also provide interested OEM customers with complete system designs that leverage their new COM-HPC Server modules and large partner network.
Accelerating edge server workloads
However, the new COM-HPC Server-on-Modules in Size E and Size D with BGA-mounted Intel Xeon D processors (code name Ice Lake D) impress not just with support for the extended temperature range from -40°C to 85°C. They also break many of the previous bottlenecks caused by edge server restrictions and will significantly accelerate the coming generation of real-time micro server workloads in harsh environments and extended temperature ranges. Improvements include up to 20 cores, up to 1TB memory on up to 8 DRAM sockets at 2933MT/s, up to 47 PCIe lanes per module in total and 32 PCIe Gen 4 lanes with double throughput per lane, and up to 100 GbE connectivity and TCC/TSN support with optimized power consumption thanks to 10nm manufacturing. Video storage and analytics servers also benefit from integrated Intel AVX-512, VNNI and OpenVINO support for AI-based data analytics.
A milestone for real-time edge server designs
In fact, the market launch of the Ice Lake D based COM-HPC Server-on-Modules sets a triple milestone: First, because extended temperature range support means Intel Xeon D Server-on-Modules are no longer limited to standard industrial applications but also target outdoor and automotive environments. Second, the world’s first COM-HPC Server-on-Modules increase the number of available cores to 20 for the first time; with up to 8 DRAM sockets, this provides massively more memory bandwidth than Server-on-Modules based on other PICMG specifications. Third, these new server modules are real-time capable, both in terms of processor cores and TCC/TSN-enabled real-time Ethernet, which is essential for digitalized IIoT and Industry 4.0 projects.
The Intel Xeon D 27xx HCC processor configurations of COM-HPC Server Size E modules (200 mm × 160 mm) from congatec
The Intel Xeon D 17xx LCC processor configurations of COM-HPC Server Size D (160mm x 160mm) and COM Express Type 7 (95 mm × 120 mm) modules from congatec
To be able to implement server balancing and server consolidation services for deterministic real-time edge server installations, where diverse real-time applications operate independently of each other on a single edge server, it is helpful if the platforms support real-time-capable virtual machines, as the RTS Hypervisor from Real-Time Systems does, for example. This allows Industry 4.0 factories to host heterogeneous real-time applications on a single server platform at the edge of their private 5G networks, and to allocate exclusive system resources to the individual processes. Server-on-Modules from congatec are pre-qualified for such services. Custom installations with all the necessary parameterizations can be included in the standard services that congatec offers for the new COM-HPC modules.
The modules further impress with a comprehensive server-grade feature set: For mission-critical designs, they offer powerful hardware security functions such as Intel Boot Guard, Intel Total Memory Encryption – Multi-Tenant (Intel TME-MT) and Intel Software Guard Extensions (Intel SGX). For best RAS capabilities, the processor modules integrate the Intel ME Manageability Engine and support remote hardware management features such as IPMI and redfish. In fact, there is another PICMG specification that ensures the interoperability of such implementations, and the training program of the congatec academy also covers this aspect.
Server-on-Module options for Intel Xeon D
The new modules will come as a High Core Count (HCC) and a Low Core Count (LCC) variant featuring different flavors of the Intel Xeon D processor series.
The conga-HPC/sILH COM-HPC Server Size E modules will be available with 5 different Intel Xeon D 27xx HCC processors with a choice of 4 to 20 cores, 8 DIMM sockets for up to 1 TByte of 2933 MT/s fast DDR4 memory with ECC, 32x PCIe Gen 4 and 16x PCIe Gen 3 as well as 100 GbE throughput plus real-time capable 2.5 Gbit/s Ethernet with TSN and TCC support at a processor base power of 65 to 118 Watt.
The COM-HPC Server Size D and COM Express Type 7 modules will come with 5 different Intel Xeon D 17xx LCC processors with a choice of 4 to 10 cores. While the conga-B7Xl COM Express Server-on-Module supports up to 128 GB DDR4 2666 MT/s RAM via up to 3 SODIMM sockets, the conga‑HPC/SILL COM-HPC Server Size D module offers 4 DIMM sockets for up to 256 GB of 2933 MT/s fast DDR4 RAM or 128 GB with ECC UDIMM RAM. Both module families offer 16x PCIe Gen 4 and 16x PCIe Gen 3 lanes. For fast networking, they provide up to 50 GbE throughput and TSN/TCC support via 2.5 Gbit/s Ethernet at a processor base power of 40 to 67 Watt.
The modules can be pre-ordered now and application ready evaluation samples – with rugged cooling solutions to match the processor TDP – are available immediately. Cooling solutions range from powerful active cooling with heat pipe adapter to fully passive cooling solutions for best mechanical resilience against vibration and shocks. The latter also alleviate thermal stress in applications that must endure short bursts of extreme temperature fluctuations. On the software side, the new modules come with comprehensive board support packages for Windows, Linux and VxWorks, and RTS hypervisor technology.
Author: Andreas Bergbauer,
Product Line Manager COM-HPC
