HPE - A00046945en - Us - HPE Synergy Power Management Overview
HPE - A00046945en - Us - HPE Synergy Power Management Overview
Technical Paper
Abstract
This document is for individuals who want to gain additional details on how HPE Synergy manages power usage in an
HPE Synergy 12000 frame.
Notices
The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise
products and services are set forth in the express warranty statements accompanying such products and services.
Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable
for technical or editorial errors or omissions contained herein.
Confidential computer software. Valid license from Hewlett Packard Enterprise required for possession, use, or copying.
Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and
Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license.
Links to third-party websites take you outside the Hewlett Packard Enterprise website. Hewlett Packard Enterprise has no
control over and is not responsible for information outside the Hewlett Packard Enterprise website.
Acknowledgments
Intel®, Itanium®, Optane™, Pentium®, Xeon®, Intel Inside®, and the Intel Inside logo are trademarks of Intel Corporation
or its subsidiaries.
AMD and the AMD EPYC™ and combinations thereof are trademarks of Advanced Micro Devices, Inc.
Microsoft® and Windows® are either registered trademarks or trademarks of Microsoft Corporation in the United States
and/or other countries.
Java® and Oracle® are registered trademarks of Oracle and/or its affiliates.
Power estimation.............................................................................................................. 12
3
Introduction
Power management for HPE Synergy encompasses several capabilities that provide a simplified customer experience. It
allows the use of the maximum amount of power that can be safely used for the frame. Also, it provides protection from
power-related failures. This technical paper familiarizes the reader with various aspects of HPE Synergy power
management.
Power modes
HPE Synergy supports three power modes:
Configuring the proper power mode in HPE OneView allows the frame link module to appropriately manage power in the
frame according to the power feed and power supply configuration. The customer must ensure that the power feed
cabling and PDU configuration power feeds are correctly connected to the power supplies appropriate for the selected
power mode. The default power mode is redundant power feed. If the user wants to change the power mode or other
power supply settings, the power mode selection can be made in the HPE OneView Enclosure Group power configuration.
• Depending on the type of power supplies installed in the frame the following comments apply:
• HPE Synergy Flex Slot Power Solution (HPE Synergy Dual Slot Power Supply Adapter with HPE
1800-2200W Flex Slot Titanium Hot Plug Power Supplies):
HPE Synergy supports only 6+6 combination of power supplies. Half of the installed power
supplies are required to be connected to one power feed. The other half are required to be
connected to the second power feed.
A dual slot power supply adapter accommodates the installation of two flex slot power
supplies. All dual slot power supply adapters (6) must be installed in the frame and all flex
slot power supplies (12) must populate the dual slot power supply adapters.
Power modes 4
One flex slot power supply from each dual slot power supply adapters must be connected to
each power feed.
For more information, see HPE Synergy Configuration and Compatibility Guide or the HPE
Synergy 12000 Frame Setup and Installation Guide (https://www.hpe.com/info/synergy-
docs).
• HPE Synergy standard form factor power supplies:
HPE Synergy supports 1+1, 2+2, and 3+3 combinations of power supplies. Half of the
installed power supplies are required to be connected to one power feed. The other half
are required to be connected to the second power feed.
There are no restrictions on which power supply bays are populated with power supplies.
There are also no restrictions on which power supplies are connected to which of the two
power feeds. This flexibility allows customers flexibility in power cabling and power supply
configuration.
There are recommended configurations that provide the best overall cooling for the power
supplies and most efficient distribution of power. For more information, see HPE Synergy
Configuration and Compatibility Guide (https://www.hpe.com/info/synergy-docs).
• The amount of power available for use is referred to as the power capacity. N is the total number of installed power
supplies divided by two. For example, if there are six 2650W Synergy standard form factor power supplies installed in
the frame, N=3 and the power capacity of the frame is 3 x 2650W=7950W. Similarly, if there are 12 flex slot power
supplies configured for 2200W capacity installed in the frame, N=6 and the power capacity of the frame is 6 x 2200W =
13200W. Flex slot power supplies are used with dual slot power supplies. If an example lists 12 flex slot power supplies,
then 6 dual slot power supply adapters are being used to support those power supplies because two flew slot power
supplies go in a dual slot power supply adapter.
NOTE: The 1800W-2200W flex slot power supply must be configured based on the power feed and power
distribution input voltage and rated input current values. Configuration is done in HPE OneView. In the example
above, flex slot power supplies used the configured wattage value of 2200 watts.
The power capacity is N times the wattage of an individual power supply. The computation of power capacity does
not sense the actual power feed connectivity, so it is important that half of the flex slot power supplies (bays 1A-6A)
are connected to one feed and the other half of the flex slot power supplies (bays 1B-6B) are connected to the other
feed.
• Only an even number of total power supplies installed in a frame is meaningful due to the symmetric nature of N+N
redundant power feed power management. If an odd number of power supplies are installed, the value of N for N+N
power capacity calculation is equal to:
• HPE Synergy Frame Link Module firmware versions 4.00.00 and later:
The total number of power supplies in the frame divided by 2 and rounded up to the next higher
number of power supplies. For example, if there are five 2650W power supplies, N=3 and the redundant
power capacity is 7950W as if there are three 2650W power supplies on each power feed. Although the
missing power supply is included in calculating the redundant power capacity of the frame, one power
feed does not have the required number of power supplies to support power feed redundancy. One
should be installed to get redundancy. Note that all operational power supplies always share equally in
supplying the power required by components in the frame.
• For HPE Synergy Frame Link Module firmware versions prior to 4.00.00:
The total number of power supplies in the frame divided by 2 and rounded down to the next lower
number of power supplies. For example, if there are five 2650W power supplies, N=2 and the redundant
power capacity is 5300W as if there are only two 2650W power supplies on each power feed. Although
the odd power supply is not included in calculating the power capacity of a system, all operational power
supplies always share equally in supplying the power required by components in the frame.
• Allocation of power above the determined redundant power capacity in the frame may be allowed depending on the
power supplies used. This oversubscription of the redundant power capacity is called power capacity boost. Details on
power capacity boost are described in the Power Capacity Boost section.
Power modes 5
Redundant power supply mode
Redundant power supply mode is primarily used when all the power supplies are connected to the same power feed. When
in this mode, HPE Synergy holds the equivalent of one power supply of power in reserve. A failure of a single power supply
will not risk the uptime of the frame due to insufficient power. The following apply specifically to redundant power supply
mode.
• Depending on the type of power supplies installed in the frame the following comments apply:
• HPE Synergy Flex Power Solution (HPE Synergy Dual Slot Power Supply Adapter with HPE
1800 -2200W Flex Slot Titanium Hot Plug Power Supplies):
HPE Synergy supports 10+2 combination of power supplies which is derived from
the required presence of six dual slot power supply adapters in the frame with two
flex slot power supplies installed in each adapter. The frame will remain
operational without server throttling even if two flex slot power supplies or a
single dual slot power supply adapter fails.
All dual slot power supply adapters must be installed in the frame and all dual slot
power supply adapter bays must be populated with flex slot power supplies.
Power capacity is defined as the amount of power provided by N power supplies,
where
N = [Total number of installed power supplies] - 2
Power capacity = N x [wattage of an individual power supply]
Power capacity boost is not available in base power mode. See the Power Capacity Boost section for more details. Server
throttling is disabled in this mode. While the frame is operating in this base power mode there are no built-in power
management safeguards other than limiting total power allocation.
Power modes 6
Power allocation
The frame link module is responsible for managing the use of power within the frame. All components (modules -
compute, interconnect, storage, appliances, and frame link modules / components - front panel, power supplies, and
system fans) that require power provide information to the frame link module that represents the maximum amount of
power that they can reasonably use. The frame link module maintains a power allocation total for the entire frame, based
upon installed and powered components. The difference between the power capacity of frame, along with any power
capacity boost watts, and the allocated power total is the amount of power available to be used to power on additional
components.
Some components will automatically be allowed to power on (see Power On Policy), while other components must
request permission to power on. Components that must request power on permission provide a power value that the
frame link module will check against the available power in the frame. If the available power in the frame is equal to or
greater than the power on request value, the component is allowed to power on.
When a component is granted power on permission, the available power of the frame is reduced by the amount allocated
for that component. When a component is powered down or removed, the amount of power that had been allocated for it
is released and made available to allocate to other components in the frame. Whenever there is a change to the amount of
available power, the power requirements of any components installed in a frame that have not been powered and are
requesting power on permission, or components that have been powered off and are requesting power on permission will
be re-evaluated.
Since user applications vary widely and may not stress all components to their maximum power usage, the actual power
used by any component might be lower than the amount of power that is allocated for it. Allocating power based upon a
reasonable maximum power value allows the frame to support a wide range of workloads. This maximum estimation is
necessary to prevent even the most aggressive workloads from causing frame brownouts or crashes. The key point is
that power allocation is necessarily done to the maximum power draw potential of the components in the frame and not
done relative to actual power utilization since this varies widely and is not absolutely predictable.
In redundant power supply mode, the frame link module allocates power only to the redundant power capacity of the
frame. Similarly, in base power mode, the frame link module allocates power only to the total power capacity of the frame.
However, in redundant power feed mode, the frame link module may allocate power beyond the capacity of the frame if
there are power capacity boost watts available. For more information, see the Power Capacity Boost.
Power on policies
The power on policies determines:
After powering on, the power associated with these components will be accounted for by the frame link module and
deducted from the available power to be granted to other components in the frame.
Interconnect modules
Interconnect modules provide a power request value to the frame link module while operating on auxiliary power. The
power request value for an interconnect module represents the maximum amount of power that the module could use.
Storage modules
Storage modules, such as the HPE Synergy D3940 Storage Module, allow drives to be hot-plugged into or out of the drive
drawer without removing the storage module itself from the frame. Power is allocated to the storage module as if it were
fully configured. This rule ensures that there is always sufficient power available to support a full complement of drives.
HPE Synergy uses the power associated with the highest wattage drives that are supported by the HPE Synergy D3940
Storage Module. By allocating power associated with high wattage drives, any drives supported in the HPE Synergy
D3940 Storage Module will be allowed to power on.
Compute modules
Because the configuration can vary widely, the powering on of compute modules is handled differently than interconnect
or storage modules. HPE Synergy compute modules provide power request values representative of the actual hardware
configuration. Since compute modules support a wide variety of operating systems and applications, the power request
values for these modules represent the maximum power levels the configuration could possibly use and is not necessarily
representative of the actual power utilization associated with every application workload or operating system.
The compute module provides the initial power on request to the frame link module once the iLO of the compute module
can operate on auxiliary power. HPE iLO determines many aspects of the compute module configuration to make the
initial power on request but depending on the generation of the compute module some configuration information, such
as the type of CPUs and memory installed, cannot be determined until those devices are powered as the compute module
boots. After the compute module is granted power on permission and has booted, the compute module can determine
the exact configuration of processors and memory, and a potentially more accurate second power request is provided to
the frame link module to reflect the actual configuration of the module. If the second allocation number is lower than the
initial power on request, some power allocation will be freed up for other components.
With Gen10 (iLO 5 v2.65 or later) and Gen10 Plus compute modules, the actual CPU and memory configuration from the
prior boot will be used in the initial power request. If the system configuration is reset by removing the battery and then re-
installing the system battery, this behaviour will change as follows:
• For Gen10 compute modules, the highest-powered CPU and memory configuration that the compute module can
support, based on the actual number of CPUs installed, is assumed in the initial power request.
• For Gen10 Plus compute modules with iLO 5 firmware v2.65 or later, the power allocation of the actual CPU
configuration is used in the initial power request. The highest-powered memory configuration that the compute module
can support is assumed for the initial power request.
For Gen11 compute modules, the power allocation of the actual CPU and memory configuration is used in the initial power
request even after a reset to factory defaults.
Because HPE Synergy supports a range of compute module configuration options with varying maximum power draw
potential, and the highest-powered CPU/DIMM configuration is assumed for the initial power request, it is not unusual for
the power capacity of the frame to initially be allocated to only a subset of the installed compute modules when multiple
modules are being powered on simultaneously. When this event happens, there may be a small number of compute modules
that are initially denied power. However, when the compute modules that have been granted power fully boot and provide
their second, lower, power allocation value, the amount of available power will increase and the compute modules that were
initially denied power will then be able to power on as well.
Power capacity boost 8
This readjusting of available power might take several minutes to complete and might require an additional press of the
power button (either virtual or physical) for compute modules that were initially denied power on.
NOTE: Improvements have been made across compute module generations to minimize the initial over-allocation of
power.
Beginning with Gen10 Plus compute modules, certain compute module CPU, memory, drive cage configurations may
require increased cooling. As a result, additional infrastructure power must be allocated to account for the increased
power utilization of the fans in the corresponding frame cooling zone. When the first such compute module requests
power on permission from the frame link module in a frame cooling zone, the additional fan power allocation will be added
by the frame link module. This additional power is allocated for all fans in the corresponding cooling zone. Frame cooling
zones encompass compute module bays 1 through 6 and corresponding fans 1 through 5, and compute module bays 7
through 12 and corresponding fans 6 through 10. Beginning with Gen11 compute modules, higher capacity fans are
required, and these fans could result in allocation of additional power depending on the compute module configurations.
The frame link module may also hold a compute module in an unpowered state for a period. For example, to ensure that a
compute module does not boot without fabric connectivity, the frame link module may hold a compute module in an
unpowered state until the interconnects that it is connected to become available. To prevent an indefinite hold on the
ability for a compute module to power on, it will be allowed to boot if the interconnect module it is dependent upon does
not become available within approximately 15 minutes. Note that an alert will be asserted against the interconnect
module in this case.
HPE OneView may also hold a compute module in an unpowered state while performing discovery and required
configuration tasks such as profile assignment. Information about the status of HPE OneView running tasks is found in
the HPE OneView Activity log.
• While the frame is operating with one power feed failed or multiple power supplies failed.
• In redundant power feed mode, when power capacity boost watts have been used such that allocated power falls
below the redundant capacity of the frame and when a power feed fails. In this scenario, the remaining amount of
power from a single feed may be insufficient to keep the frame safely powered without throttling.
• In redundant power feed mode, when multiple power supplies fail and where the capacity of the remaining operational
power supplies exceeds the level of power allocated to the frame. Note, this action can occur anytime multiple power
supplies fail, regardless of whether a power feed failure occurred or not.
• In redundant power supply mode, when multiple power supplies fail and where the capacity of the remaining
operational power supplies falls below the level of power allocated to the frame. Note that for HPE Synergy
Frame Link Module firmware version 4.00.00 and later throttling will no longer occur in this scenario.
While compute modules are in a state of throttle, application performance may be impacted. The amount of impact will
vary based on the applications running on the compute modules, the compute module power utilization levels, and the
compute module generation. Throttling cannot be directly disabled. However, throttling can be avoided by reducing the
configuration to a level where the allocated power of the frame does not exceed the available redundant power capacity
of the frame. Note that the following enhancements have been made across compute module generations to minimize
the performance impacts of throttling due to oversubscription and the use of power capacity boost watts:
• For Gen10 Plus compute modules:
As a percentage of the overall full power allocation for the compute module. the number of power
capacity boost watts for each compute module has been lowered. This results in less oversubscription
of frame power and lighter levels of throttling which lessens the potential impact to performance during
throttling.
Smart Capping: The throttling mechanism used has been enhanced such that actual throttling of a
specific compute module will only occur when the instantaneous power draw of the compute module
actually exceeds the reduced power budget for the compute module when throttling is being
performed. Prior to this change, continuous throttling occurred under the same conditions despite
actual power draw of the compute module. With this enhancement, many workloads that typically use
less power than the full power allocation will experience less and potentially no throttling.
• For Gen11 compute modules and later:
Dynamic Smart Capping: Building on the Smart Capping capability, throttling levels have been further
minimized for Gen11 compute modules to the lowest levels possible based on the actual level of
oversubscription of the redundant power capacity. With this enhancement, the frame link module will
monitor the level of oversubscription and lower the power budget for each Gen11 compute module
during throttling by only the amount required to fit within the redundant power capacity of the frame.
The oversubscribed power is distributed between each Gen11 compute module proportional to its full
power allocation. This further minimizes the performance impacts experienced by each Gen11 compute
module during throttling as compared to previous generations of compute modules.
For example, if a frame has a power allocation level that only exceeds the redundant capacity by 100W
this 100W will be divided between all compute modules in the frame. This division is done
proportionally to each compute module’s full power allocation level and distributed as a reduced target
power budget. The target power budget must be met when throttling is active. Throttling will occur as
needed to meet the reduced power budget.
Power throttling 10
With this change, the power budget for each compute module during throttling will be reduced from the
full power allocation value by the minimum value necessary to make sure that the frame does not
exceed the redundant power capacity.
Power throttling does not occur when the frame is in base power mode. In frame link modules firmware 4.00.00 and prior,
power throttling could occur in power supply redundant mode when, due to multiple power supply failures, the power
capacity available in the frame dropped below the amount of power allocated to frame components. For frame link modules
firmware 4.00.00 and later, power throttling will no longer occur for power supply redundant mode.
Power throttling 11
Power estimation
Power usage can be estimated with the HPE Power Advisor tool. The power numbers provided by Power Advisor are only
estimates. Every workload uses compute module hardware differently and actual power usage can vary from Power Advisor
estimates. When a configuration of HPE Synergy components is modeled in Power Advisor, there are several important
power estimates that the tool provides, as shown in the following figure.
1. Utilization power: This number is the estimated average power for the modeled configuration. The default utilization
is 100%, but can be changed to represent lower utilizations as needed. It represents the real power as measured at the
input to the power supplies and is what would be provided by the rack PDUs. When the utilization is 100%, this value
should equal "Max load input power".
2. Max load input power: Max load input power is the estimated maximum amount of power that can be used by the
configuration. Like utilization power, this value represents the real power as measured at the input to the power
supplies and is always higher than the amount of 12V DC power that the power supplies are providing to the frame.
3. Allocation power: HPE Power Advisor uses the same power allocation algorithms that the actual HPE Synergy system
uses to determine allocated power. When this value is larger than the power capacity of the frame and the frame is in
power feed redundancy mode, one should expect that this frame will use power capacity boost watts and that
compute throttling will occur during a power feed failure.
Power estimation 12
HPE OneView power metrics
The following power metric parameters are displayed in HPE OneView, when viewing the frame (enclosure) power page.
• Power mode: User selectable as redundant power feed mode, redundant power supply mode, or base power mode.
The default is redundant power feed mode.
• Installed power supplies: The number of power supplies installed in the frame.
• Minimum power supplies: The minimum number of power supplies required to ensure that the frame remains
operational in the given power mode. Depending on the configuration, operation of the frame with only the minimum
number of power supplies might result in the frame being held in a degraded power state. This state will continue until
additional power supplies are added.
• Minimum power supplies for redundant power feed: The minimum number of power supplies required to ensure a
non-degraded operational frame state in the given power mode.
• Average power: The average power usage of the frame during the past 24 hours.
• Power capacity: N times the DC output watts of a single power supply, where (1) N = number of power supplies / 2
for redundant power feed mode, (2) N = number of power supplies - 1 for redundant power supply mode, and (3) N =
number of power supplies for base power mode.
• Power capacity boost: Amount of power that the frame will allocate above the capacity of the frame.
• Power allocated: Total DC watts allocated to all devices, interconnects, and infrastructure, including fans, link
modules, and appliances.
• Power available: Power capacity + capacity boost - power allocated. Note that when this value is zero or negative it
indicates that the frame power is currently oversubscribed as described in the sections above and that throttling of the
compute modules may occur on loss of power capacity due to power feed loss or power supply failures.
Environmental settings
The environmental setting for an HPE Synergy frame influences the amount of power that is allocated for cooling of the
frame components. However, they do not directly control the power management features. The options available are:
• Standard (35C/95F): Optimizes the frame to operate in environments with ambient temperatures up to 35 degrees
Celsius. This option is the default setting.
• ASHRAE A3 (40C/104F): Optimizes the frame to operate in environments with ambient temperatures up to 40
degrees Celsius. In this mode, HPE Synergy allocates an additional 150W to accommodate the system fans operating at
a slightly higher RPM versus standard mode.
• ASHRAE A4 (45C/113F): Optimizes the frame to operate in environments with ambient temperatures up to 45
degrees Celsius. In this mode, HPE Synergy allocates an additional 300W to accommodate the system fans operating at
a higher RPM versus standard mode.
• Telco: Optimizes the frame to operate in environments with standard ambient temperatures up to 40 degrees Celsius
(continuous). It also operates in extremes up to 55 degrees Celsius (no longer than 96 hours per occurrence). In this
mode, HPE Synergy allocates an additional 500W to accommodate the system fans. They operate at a high RPM to cool
the frame.
As the data center ambient temperature increases, the average speed of the fans will also increase. This action draws
more air through the frame, keeping hot components properly cooled. The system does not validate the environmental
setting. Take care to select the lowest environmental setting that is appropriate for the individual needs. Selecting too
high of a temperature setting will result in additional power being allocated for cooling that could otherwise be available
to compute, storage, or interconnect modules. Selecting too low an environmental setting could result in insufficient power
being allocated to the frame fans, which could lead to an overutilization of the power supplies or feeds.
Environmental settings 14
Compute configuration
The compute modules for Gen 9 and Gen 10 allocate infrastructure power only on environmental settings. In Gen 10 Plus
compute modules the configuration can also influence power allocation. For standard configurations the environmental
settings apply. For hard to cool configurations the power allocated to the fans is always 1000W at a chassis level. In Gen 11
compute modules, the introduction of high-performance fans, and harder to cool configurations the power allocated to the
fans can be 1500W.
• Note that when the flex slot power supplies are used they must populate all power supply bays in the frame. Mixing
this type of power supply with other types is not supported and can result in unexpected loss of power to the frame.
• All power modes require a minimum of two power supplies to be installed in the frame. Note that when using HPE
Synergy Dual Slot Power Supply Adapters with flex slot power supplies, all frame power supply bays must contain a
power supply adapter and all power supply adapter bays must contain flex slot power supplies.