SPEC CPU2017 Platform Settings for xFusion Systems

Operating System Tuning Parameters

ulimit -s <n>:
Sets the stack size to n kbytes, or unlimited to allow the stack size to grow without limit.
drop_caches:
Writing to this will cause the kernel to drop clean caches, as well as reclaimable slab objects like dentries and inodes. Once dropped, their memory becomes free.
cpupower:
The OS 'cpupower' utility is used to change CPU power governors settings. Available settings are:
SCALING_GOVERNOR:

Setting this environment variable to "performance" to enable cores to run at performance mode.

"scaling_governor" is a configuration file in Linux's "cpufreq" model. There are five mode in "scaling_governor" which are performance, powersave, userspace, ondemand, and conservative.

We use "cpupower frequency-set -g performance" to set this environment variable to "performance".

Transparent Hugepages (THP)
THP is an abstraction layer that automates most aspects of creating, managing, and using huge pages. It is designed to hide much of the complexity in using huge pages from system administrators and developers. Huge pages increase the memory page size from 4 kilobytes to 2 megabytes. This provides significant performance advantages on systems with highly contended resources and large memory workloads. If memory utilization is too high or memory is badly fragmented which prevents hugepages being allocated, the kernel will assign smaller 4k pages instead. Most recent Linux OS releases have THP enabled by default.
THP usage is controlled by the sysfs setting /sys/kernel/mm/transparent_hugepage/enabled. Possible values: THP creation is controlled by the sysfs setting /sys/kernel/mm/transparent_hugepage/defrag. Possible values: An application that "always" requests THP often can benefit from waiting for an allocation until those huge pages can be assembled.
For more information see the Linux transparent hugepage documentation.
tuned-adm:

A commandline interface for switching between different tuning profiles available in supported Linux distributions. The distribution provided profiles are located in /usr/lib/tuned and the user defined profiles in /etc/tuned. To set a profile, one can issue the command "tuned-adm profile (profile_name)". Below are details about some relevant profiles.

We use "tuned-adm profile throughput-performance" to set this environment variable to "throughput-performance".


Firmware / BIOS / Microcode Settings

Determinism Control:
This BIOS option allows user can set customized determinism slider mode to control performance.
Determinism Enable:
This BIOS option allows for AGESA determinism to control performance.
TDP Control:
This BIOS option is for "Configurable TDP (cTDP)", it allows user can set customized value for TDP. Available settings are:
TDP:
TDP is an acronym for "Thermal Design Power." TDP is the recommended target for power used when designing the cooling capacity for a server. EPYC processors are able to control this target power consumption within certain limits. This capability is referred to as "configurable TDP" or "cTDP." TDP can be used to reduce power consumption for greater efficiency, or in some cases, increase power consumption above the default value to provide additional performance. TDP is controlled using a BIOS option.

The default EPYC TDP value corresponds with the microprocessor's nominal TDP. The default TDP value is set at a good balance between performance and energy efficiency. The EPYC 9654 TDP can be reduced as low as 320W, which will minimize the power consumption for the processor under load, but at the expense of peak performance. Increasing the EPYC 9654 TDP to 400W will maximize peak performance by allowing the CPU to maintain higher dynamic clock speeds, but will make the microprocessor less energy efficient. Note that at maximum TDP, the CPU thermal solution must be capable of dissipating at least 400W or the EPYC 9654 processor might engage in thermal throttling under load.

The available TDP ranges for each EPYC model are in the table below:
ModelMinimum TDPMaximum TDP
EPYC 9654320400
EPYC 9654P320400
EPYC 9554320400
EPYC 9554P320400
EPYC 9534240300
EPYC 9474F320400
EPYC 9374F320400
EPYC 9354240300
EPYC 9354P240300
EPYC 9334200240
EPYC 9224200240
EPYC 9174F320400
EPYC 9124200240
* TDP must remain below the thermal solution design parameters or thermal throttling could be frequently encountered.
DRAM Scrub Time:
DRAM scrubbing is a mechanism for the memory controller to periodically read all memory locations and write back corrected data. The time interval for scrubbing the entire memory can be: Disabled/1 hour/4 hours/8 hours/12 hours/16 hours/24 hours/48 hours. Current default is Auto(24 hours).
NUMA Nodes Per Socket:
Specifies the number of desired NUMA nodes per populated socket in the system: Current default is Auto.
APBDIS:
Application Power Management (APM) allows the processor to provide maximum performance while remaining within the specified power delivery and removal envelope. APM dynamically monitors processor activity and generates an approximation of power consumption. If power consumption exceeds a defined power limit, a P-state limit is applied by APM hardware to reduce power consumption. APM ensures that average power consumption over a thermally significant time period remains at or below the defined power limit. Set APBDIS=1 will disable Data Fabric APM and the SOC P-state will be fixed. Available settings are:
ACPI SRAT L3 Cache As NUMA Domain:
Each L3 Cache will be exposed as a NUMA node when enabling ACPI SRAT L3 Cache as a NUMA node. On a dual processor system, with up to 8 L3 Caches per processor, this setting will expose 16 NUMA domains. Available settings are:
PPT Control:
This BIOS option allows user can set customized value for processor package power limit(PPT). Available settings are:
PPT:
Set customize processor Package Power Limit (PPT) value to be used on all populated processors in the system. Current default value is 0. ***PPT will be used as the ASIC power limit***