# CPU2006 Flag Description
Hygon

## Compilers: x86 Open64 Compiler Suite

## Base Compiler Invocation

### C benchmarks

- - opencc
  - CC, LD

  - Invoke the Open64 C compiler.
    Also used to invoke linker for C programs.

### C++ benchmarks

- - openCC
  - CXX, LD

  - Invoke the Open64 C++ compiler.
    Also used to invoke linker for C++ programs.

### Fortran benchmarks

- - openf95
  - FC, LD

  - Invoke the Open64 Fortran 77, 90 and 95 compilers.
    Also used to invoke linker for Fortran programs and for mixed C / Fortran. openf90 and openf95 are synonymous.

### Benchmarks using both Fortran and C

- - opencc
  - CC

  - Invoke the Open64 C compiler.
    Also used to invoke linker for C programs.

- - openf95
  - FC, LD

  - Invoke the Open64 Fortran 77, 90 and 95 compilers.
    Also used to invoke linker for Fortran programs and for mixed C / Fortran. openf90 and openf95 are synonymous.

## Base Portability Flags

#### 410.bwaves

- - -DSPEC_CPU_LP64
  - PORTABILITY

  - This option is used to indicate that the host system's integers are 32-bits wide, and longs and pointers are 64-bits wide. Not all benchmarks recognize this macro, but the preferred practice for data model selection applies the flags to all benchmarks; this flag description is a placeholder for those benchmarks that do not recognize this macro.

#### 416.gamess

- - -DSPEC_CPU_LP64
  - PORTABILITY

  - This option is used to indicate that the host system's integers are 32-bits wide, and longs and pointers are 64-bits wide. Not all benchmarks recognize this macro, but the preferred practice for data model selection applies the flags to all benchmarks; this flag description is a placeholder for those benchmarks that do not recognize this macro.

#### 433.milc

- - -DSPEC_CPU_LP64
  - PORTABILITY

  - This option is used to indicate that the host system's integers are 32-bits wide, and longs and pointers are 64-bits wide. Not all benchmarks recognize this macro, but the preferred practice for data model selection applies the flags to all benchmarks; this flag description is a placeholder for those benchmarks that do not recognize this macro.

#### 434.zeusmp

- - -DSPEC_CPU_LP64
  - PORTABILITY

  - This option is used to indicate that the host system's integers are 32-bits wide, and longs and pointers are 64-bits wide. Not all benchmarks recognize this macro, but the preferred practice for data model selection applies the flags to all benchmarks; this flag description is a placeholder for those benchmarks that do not recognize this macro.

#### 435.gromacs

- - -DSPEC_CPU_LP64
  - PORTABILITY

  - This option is used to indicate that the host system's integers are 32-bits wide, and longs and pointers are 64-bits wide. Not all benchmarks recognize this macro, but the preferred practice for data model selection applies the flags to all benchmarks; this flag description is a placeholder for those benchmarks that do not recognize this macro.

#### 436.cactusADM

- - -DSPEC_CPU_LP64
  - PORTABILITY

  - This option is used to indicate that the host system's integers are 32-bits wide, and longs and pointers are 64-bits wide. Not all benchmarks recognize this macro, but the preferred practice for data model selection applies the flags to all benchmarks; this flag description is a placeholder for those benchmarks that do not recognize this macro.

- - -fno-second-underscore
  - FPORTABILITY

  - CFP2006:

    If -funderscoring is in effect, and the original Fortran external identifier contained an underscore, -fsecond-underscore appends a second underscore to the one added by -funderscoring. -fno-second-underscore does not append a second underscore. The default is both -funderscoring and -fsecond-underscore, the same defaults as g77 uses. -fno-second-underscore corresponds to the default policies of PGI Fortran and Intel Fortran.

#### 437.leslie3d

- - -DSPEC_CPU_LP64
  - PORTABILITY

  - This option is used to indicate that the host system's integers are 32-bits wide, and longs and pointers are 64-bits wide. Not all benchmarks recognize this macro, but the preferred practice for data model selection applies the flags to all benchmarks; this flag description is a placeholder for those benchmarks that do not recognize this macro.

#### 444.namd

- - -DSPEC_CPU_LP64
  - PORTABILITY

  - This option is used to indicate that the host system's integers are 32-bits wide, and longs and pointers are 64-bits wide. Not all benchmarks recognize this macro, but the preferred practice for data model selection applies the flags to all benchmarks; this flag description is a placeholder for those benchmarks that do not recognize this macro.

#### 447.dealII

- - -DSPEC_CPU_LP64
  - PORTABILITY

  - This option is used to indicate that the host system's integers are 32-bits wide, and longs and pointers are 64-bits wide. Not all benchmarks recognize this macro, but the preferred practice for data model selection applies the flags to all benchmarks; this flag description is a placeholder for those benchmarks that do not recognize this macro.

#### 450.soplex

- - -DSPEC_CPU_LP64
  - PORTABILITY

  - This option is used to indicate that the host system's integers are 32-bits wide, and longs and pointers are 64-bits wide. Not all benchmarks recognize this macro, but the preferred practice for data model selection applies the flags to all benchmarks; this flag description is a placeholder for those benchmarks that do not recognize this macro.

#### 453.povray

- - -DSPEC_CPU_LP64
  - PORTABILITY

  - This option is used to indicate that the host system's integers are 32-bits wide, and longs and pointers are 64-bits wide. Not all benchmarks recognize this macro, but the preferred practice for data model selection applies the flags to all benchmarks; this flag description is a placeholder for those benchmarks that do not recognize this macro.

#### 454.calculix

- - -DSPEC_CPU_LP64
  - PORTABILITY

  - This option is used to indicate that the host system's integers are 32-bits wide, and longs and pointers are 64-bits wide. Not all benchmarks recognize this macro, but the preferred practice for data model selection applies the flags to all benchmarks; this flag description is a placeholder for those benchmarks that do not recognize this macro.

#### 459.GemsFDTD

- - -DSPEC_CPU_LP64
  - PORTABILITY

  - This option is used to indicate that the host system's integers are 32-bits wide, and longs and pointers are 64-bits wide. Not all benchmarks recognize this macro, but the preferred practice for data model selection applies the flags to all benchmarks; this flag description is a placeholder for those benchmarks that do not recognize this macro.

#### 465.tonto

- - -DSPEC_CPU_LP64
  - PORTABILITY

  - This option is used to indicate that the host system's integers are 32-bits wide, and longs and pointers are 64-bits wide. Not all benchmarks recognize this macro, but the preferred practice for data model selection applies the flags to all benchmarks; this flag description is a placeholder for those benchmarks that do not recognize this macro.

#### 470.lbm

- - -DSPEC_CPU_LP64
  - PORTABILITY

  - This option is used to indicate that the host system's integers are 32-bits wide, and longs and pointers are 64-bits wide. Not all benchmarks recognize this macro, but the preferred practice for data model selection applies the flags to all benchmarks; this flag description is a placeholder for those benchmarks that do not recognize this macro.

#### 481.wrf

- - -DSPEC_CPU_LINUX
  - PORTABILITY

  - This macro indicates that the benchmark is being compiled on a Linux system.

- - -DSPEC_CPU_CASE_FLAG
  - PORTABILITY

  - This macro indicates that Fortran functions called from C should have their names lower-cased.

- - -DSPEC_CPU_LP64
  - PORTABILITY

  - This option is used to indicate that the host system's integers are 32-bits wide, and longs and pointers are 64-bits wide. Not all benchmarks recognize this macro, but the preferred practice for data model selection applies the flags to all benchmarks; this flag description is a placeholder for those benchmarks that do not recognize this macro.

- - -fno-second-underscore
  - FPORTABILITY

  - CFP2006:

    If -funderscoring is in effect, and the original Fortran external identifier contained an underscore, -fsecond-underscore appends a second underscore to the one added by -funderscoring. -fno-second-underscore does not append a second underscore. The default is both -funderscoring and -fsecond-underscore, the same defaults as g77 uses. -fno-second-underscore corresponds to the default policies of PGI Fortran and Intel Fortran.

#### 482.sphinx3

- - -DSPEC_CPU_LP64
  - PORTABILITY

  - This option is used to indicate that the host system's integers are 32-bits wide, and longs and pointers are 64-bits wide. Not all benchmarks recognize this macro, but the preferred practice for data model selection applies the flags to all benchmarks; this flag description is a placeholder for those benchmarks that do not recognize this macro.

## Base Optimization Flags

### C benchmarks

- - -Ofast
  - COPTIMIZE

  - Uses a selection of optimizations in order to maximize performance.
    Specifying "-Ofast" is equivalent to -O3 -ipa -OPT:Ofast -fno-math-errno -ffast-math.
    These optimization options are generally safe. Floating-point accuracy may be affected due to the transformation of the computational code. Note the interprocedural analysis option, -ipa, specifies limitations on how libraries and object files (.o files) are built.

  - Includes:
    - -O3
      - -O2
        - -O1
    - -ipa
    - -OPT:Ofast
      - -OPT:ro
      - -OPT:Olimit
      - -OPT:div_split
      - -OPT:alias
    - -fno-math-errno
    - -ffast-math
- - -OPT:malloc_alg=1
  - COPTIMIZE

  - -OPT:malloc_algorithm,malloc_alg=(0|1|2)
    To improve runtime speed the compiler will select an optimal malloc algorithm. To enable the selected algorithm, setup code is included in the C/C++ and Fortran main function.

    - n=0: Default, no changes to the malloc options. No call to mallopt() is made.
    - n=1: M_MMAP_MAX=2 and M_TRIM_THRESHOLD=0x10000000. Call mallopt with the two settings.
    - n=2: M_MMAP_MAX=2 and M_TRIM_THRESHOLD=0x40000000. Call mallopt with the two settings.

    The two parameters, M_MMAP_MAX and M_TRIM_THRESHOLD, are described below.

    Function: int mallopt (int param, int value) When calling mallopt, the param argument specifies the parameter to be set, and value the new value to be set. Possible choices for param, as defined in malloc.h, are:

    - M_MMAP_MAX The maximum number of chunks to allocate with mmap. Setting this to zero disables all use of mmap.
    - M_TRIM_THRESHOLD This is the minimum size (in bytes) of the top-most, releasable chunk that will cause sbrk to be called with a negative argument in order to return memory to the system.
- - -HP:bd=2m:heap=2m
  - COPTIMIZE

  - Instructs the compiler to use 2MB hugepages for bss, data and text segments (i.e. bdt), and/or for heap allocation. Mixed usage of huge and small pages is not supported for bdt, but is supported for heap allocation. When the -HP:bd option is specified, the text segment is not mapped to huge pages while the bss and data segments still get mapped to 2MB hugepages. The limit option specifies a combined limit on the number of hugepages that may be used by the compiled program. If no limit is set, the number of hugepages that can be used by the program is effectively limited by the system configuration.

- - -IPA:plimit=8000
  - COPTIMIZE

  - -IPA:plimit=N: The compiler is instructed to halt inlining within a program once the intermediate representation indicates that the code size of the program has surpassed the limit set by N. The default is "-IPA:plimit=2500".

- - -IPA:small_pu=100
  - COPTIMIZE

  - -IPA:small_pu=N: The compiler is instructed not to restrict a procedure from inlining with a code size smaller than N when invoking the "-IPA:plimit" flag. The default is "-IPA:small_pu=30".

- - -mso
  - COPTIMIZE

  - Instructs the compiler to perform aggressive optimizations that are likely to improve the scalability of an application running on a system with multi-core processors. In particular, these optimizations may target machine resources that are shared among the multiple cores of a processor, e.g. memory bandwidth, shared L3 cache, etc.

- - -march=bdver1
  - COPTIMIZE

  - Compiler will generate instructions and schedule them appropriately for the selected processor type. The default value, auto, means to optimize for the platform on which the compiler is running, as determined by reading /proc/cpuinfo. anyx86 means a generic 32-bit x86 processor without SSE2 support.

- - -mno-fma4
  - COPTIMIZE

  - Enables/disables the use of FMA4 instructions on targets which support them. Specifying "-march=bdver1" enables this option, else the default is "-mno-fma4". Note specifying option "-mno-avx" implies option "-mno-fma4"

- - -mno-xop
  - COPTIMIZE

  - Enables/disables the use of XOP instructions on targets which support them. Specifying ‘-march=bdver1’ enables this option, else the default is ‘-mno-xop’. Note specifying option ‘-mno-avx’ implies option ‘-mno-xop’.

- - -mno-tbm
  - COPTIMIZE

  - Turns on/off support for tail bit manipulation(TBM) instruction set. By default ‘-mtbm’ is turned ON for bdver2 and above architectures.

- - -WB,
  - COPTIMIZE

  - When IPO is used, "-WB," passes the following flag (e.g. -mno-fma4) to the code generator.

- - -Wl,
  - COPTIMIZE

  - -Wl tells the linker to accept the following argument. In the example, it tells the linker to allow multiple definitions.

- - -z,muldefs
  - COPTIMIZE

  - The example, "-Wl, -z,muldefs", tells the linker to allow multiple definitions and use the first one encountered.

- - -L/root/jemalloc/lib64/lib -ljemalloc
  - EXTRA_CLIBS

  - -L<library directory> -ljemalloc ,
    when used as an EXTRA_CXXLIB variable, results in linking with jemalloc library for Linux. This is a library that optimizes calls to new, delete, malloc and free.

### C++ benchmarks

- - -Ofast
  - CXXOPTIMIZE

  - Uses a selection of optimizations in order to maximize performance.
    Specifying "-Ofast" is equivalent to -O3 -ipa -OPT:Ofast -fno-math-errno -ffast-math.
    These optimization options are generally safe. Floating-point accuracy may be affected due to the transformation of the computational code. Note the interprocedural analysis option, -ipa, specifies limitations on how libraries and object files (.o files) are built.

  - Includes:
    - -O3
      - -O2
        - -O1
    - -ipa
    - -OPT:Ofast
      - -OPT:ro
      - -OPT:Olimit
      - -OPT:div_split
      - -OPT:alias
    - -fno-math-errno
    - -ffast-math
- - -static
  - CXXOPTIMIZE

  - -static
    On systems that support dynamic linking, this prevents linking with shared libraries. On other systems, this option has no effect.

- - -CG:load_exe=0
  - CXXOPTIMIZE

  - -CG:load_exe=N : The parameter N must be a non-negative integer which specifies the threshold for the compiler to consider folding a memory load operation directly into its subsequent use in an arithmetic instruction (thereby eliminating the memory load operation). If N=0 this folding optimization is not performed (in other words, the optimization is turned off). If the number of times the result of the memory load is used exceeds the value of N, then the folding optimization is not performed. For example, if N=1 this optimization is performed only when the result of the memory load has only one use. The default value of N varies with target processor and source language.

- - -OPT:malloc_alg=1
  - CXXOPTIMIZE

  - -OPT:malloc_algorithm,malloc_alg=(0|1|2)
    To improve runtime speed the compiler will select an optimal malloc algorithm. To enable the selected algorithm, setup code is included in the C/C++ and Fortran main function.

    - n=0: Default, no changes to the malloc options. No call to mallopt() is made.
    - n=1: M_MMAP_MAX=2 and M_TRIM_THRESHOLD=0x10000000. Call mallopt with the two settings.
    - n=2: M_MMAP_MAX=2 and M_TRIM_THRESHOLD=0x40000000. Call mallopt with the two settings.

    The two parameters, M_MMAP_MAX and M_TRIM_THRESHOLD, are described below.

    Function: int mallopt (int param, int value) When calling mallopt, the param argument specifies the parameter to be set, and value the new value to be set. Possible choices for param, as defined in malloc.h, are:

    - M_MMAP_MAX The maximum number of chunks to allocate with mmap. Setting this to zero disables all use of mmap.
    - M_TRIM_THRESHOLD This is the minimum size (in bytes) of the top-most, releasable chunk that will cause sbrk to be called with a negative argument in order to return memory to the system.
- - -INLINE:aggressive=on
  - CXXOPTIMIZE

  - -INLINE:aggressive=(ON|On|on|OFF|Off|off|0|1): Instructs the compiler to be very aggressive when performing inlining. The default is "-INLINE:aggressive=OFF".

- - -HP:bd=2m:heap=2m
  - CXXOPTIMIZE

  - Instructs the compiler to use 2MB hugepages for bss, data and text segments (i.e. bdt), and/or for heap allocation. Mixed usage of huge and small pages is not supported for bdt, but is supported for heap allocation. When the -HP:bd option is specified, the text segment is not mapped to huge pages while the bss and data segments still get mapped to 2MB hugepages. The limit option specifies a combined limit on the number of hugepages that may be used by the compiled program. If no limit is set, the number of hugepages that can be used by the program is effectively limited by the system configuration.

- - -D__OPEN64_FAST_SET
  - CXXOPTIMIZE

  - Instructs the compiler to use a more efficient STL set map implementation. This allows faster iterations over sets and maps. This option is currently available only on open64 compilers downloaded from AMD.

- - -march=bdver2
  - CXXOPTIMIZE

  - Compiler will generate instructions and schedule them appropriately for the selected processor type. The default value, auto, means to optimize for the platform on which the compiler is running, as determined by reading /proc/cpuinfo. anyx86 means a generic 32-bit x86 processor without SSE2 support.

- - -mno-fma4
  - CXXOPTIMIZE

  - Enables/disables the use of FMA4 instructions on targets which support them. Specifying "-march=bdver1" enables this option, else the default is "-mno-fma4". Note specifying option "-mno-avx" implies option "-mno-fma4"

- - -mno-xop
  - CXXOPTIMIZE

  - Enables/disables the use of XOP instructions on targets which support them. Specifying ‘-march=bdver1’ enables this option, else the default is ‘-mno-xop’. Note specifying option ‘-mno-avx’ implies option ‘-mno-xop’.

- - -mno-tbm
  - CXXOPTIMIZE

  - Turns on/off support for tail bit manipulation(TBM) instruction set. By default ‘-mtbm’ is turned ON for bdver2 and above architectures.

- - -WB,
  - CXXOPTIMIZE

  - When IPO is used, "-WB," passes the following flag (e.g. -mno-fma4) to the code generator.

- - -Wl,
  - CXXOPTIMIZE

  - -Wl tells the linker to accept the following argument. In the example, it tells the linker to allow multiple definitions.

- - -z,muldefs
  - CXXOPTIMIZE

  - The example, "-Wl, -z,muldefs", tells the linker to allow multiple definitions and use the first one encountered.

- - -L/root/jemalloc/lib64/lib -ljemalloc
  - EXTRA_CXXLIBS

  - -L<library directory> -ljemalloc ,
    when used as an EXTRA_CXXLIB variable, results in linking with jemalloc library for Linux. This is a library that optimizes calls to new, delete, malloc and free.

- - -pthread
  - EXTRA_CXXLIBS

  - -pthread
    when used as an EXTRA_CXXLIB variable, results in linking with pthread library for Linux.

- - -ldl
  - EXTRA_CXXLIBS

  - -ldl
    when used as an EXTRA_CXXLIB variable, required by jemalloc library for Linux.

- - -l rt
  - EXTRA_CXXLIBS

  - -ldl
    when used as an EXTRA_CXXLIB variable, required by jemalloc library and results in linking with librt.so library for Linux.

### Fortran benchmarks

- - -Ofast
  - FOPTIMIZE

  - Uses a selection of optimizations in order to maximize performance.
    Specifying "-Ofast" is equivalent to -O3 -ipa -OPT:Ofast -fno-math-errno -ffast-math.
    These optimization options are generally safe. Floating-point accuracy may be affected due to the transformation of the computational code. Note the interprocedural analysis option, -ipa, specifies limitations on how libraries and object files (.o files) are built.

  - Includes:
    - -O3
      - -O2
        - -O1
    - -ipa
    - -OPT:Ofast
      - -OPT:ro
      - -OPT:Olimit
      - -OPT:div_split
      - -OPT:alias
    - -fno-math-errno
    - -ffast-math
- - -LNO:blocking=off
  - FOPTIMIZE

  - This option group commands the compiler loop nest optimizer to perform nested loop analysis and transformations. Note an optimization level of "-O3" or higher must be specified in order to enable the "-LNO:" options. To verify the LNO options that were invoked during compilation use the option "-LIST:all_options=ON".

    -LNO:blocking=(on|off|0|1): Instructs the compiler to perform cache blocking transformation. The default is "-LNO:blocking=ON".

- - -LNO:simd_peel_align=on
  - FOPTIMIZE

  - Instrucuts the compiler to create multiple copies of loops with different alignment assumptions. The compiler adds a runtime test to branch to particular loop when alignment assumptions are met. As necessary, the compiler will peel one or more iterations to improve alignment in alternate loop copies. The default is "-LNO:simd_peel_align=OFF".

- - -OPT:rsqrt=2
  - FOPTIMIZE

  - -OPT:rsqrt=(0|1|2)
    Instructs the compiler to use the reciprocal square root instruction when calculating the square root. This transformation may vary the accuracy slightly.
    0 Restrain from using the reciprocal square root instruction.
    1 Use the reciprocal square root instruction followed by operations that will improve the accuracy of the results.
    2 Use the reciprocal square root instruction without improving the result accuracy.
    Note "-OPT:rsqrt=1" if "-OPT:roundoff=2" or "-OPT:roundoff=3". The default is "-OPT:rsqrt=0".

- - -OPT:unroll_size=256
  - FOPTIMIZE

  - -OPT:unroll_size=N
    Instructs the compiler to limit the number of instructions produced when unrolling inner loops. When N=0 the ceiling is disregarded. Note by specifying "-O3" sets "-OPT:unroll_size=128". The default is "-OPT:unroll_size=40".

- - -HP:bd=2m:heap=2m
  - FOPTIMIZE

  - Instructs the compiler to use 2MB hugepages for bss, data and text segments (i.e. bdt), and/or for heap allocation. Mixed usage of huge and small pages is not supported for bdt, but is supported for heap allocation. When the -HP:bd option is specified, the text segment is not mapped to huge pages while the bss and data segments still get mapped to 2MB hugepages. The limit option specifies a combined limit on the number of hugepages that may be used by the compiled program. If no limit is set, the number of hugepages that can be used by the program is effectively limited by the system configuration.

- - -mso
  - FOPTIMIZE

  - Instructs the compiler to perform aggressive optimizations that are likely to improve the scalability of an application running on a system with multi-core processors. In particular, these optimizations may target machine resources that are shared among the multiple cores of a processor, e.g. memory bandwidth, shared L3 cache, etc.

- - -march=bdver1
  - FOPTIMIZE

  - Compiler will generate instructions and schedule them appropriately for the selected processor type. The default value, auto, means to optimize for the platform on which the compiler is running, as determined by reading /proc/cpuinfo. anyx86 means a generic 32-bit x86 processor without SSE2 support.

- - -mno-fma4
  - FOPTIMIZE

  - Enables/disables the use of FMA4 instructions on targets which support them. Specifying "-march=bdver1" enables this option, else the default is "-mno-fma4". Note specifying option "-mno-avx" implies option "-mno-fma4"

- - -mno-xop
  - FOPTIMIZE

  - Enables/disables the use of XOP instructions on targets which support them. Specifying ‘-march=bdver1’ enables this option, else the default is ‘-mno-xop’. Note specifying option ‘-mno-avx’ implies option ‘-mno-xop’.

- - -mno-tbm
  - FOPTIMIZE

  - Turns on/off support for tail bit manipulation(TBM) instruction set. By default ‘-mtbm’ is turned ON for bdver2 and above architectures.

- - -WB,
  - FOPTIMIZE

  - When IPO is used, "-WB," passes the following flag (e.g. -mno-fma4) to the code generator.

- - -Wl,
  - FOPTIMIZE

  - -Wl tells the linker to accept the following argument. In the example, it tells the linker to allow multiple definitions.

- - -z,muldefs
  - FOPTIMIZE

  - The example, "-Wl, -z,muldefs", tells the linker to allow multiple definitions and use the first one encountered.

- - -L/root/jemalloc/lib64/lib -ljemalloc
  - EXTRA_FLIBS

  - -L<library directory> -ljemalloc ,
    when used as an EXTRA_CXXLIB variable, results in linking with jemalloc library for Linux. This is a library that optimizes calls to new, delete, malloc and free.

### Benchmarks using both Fortran and C

- - -Ofast
  - COPTIMIZE, FOPTIMIZE

  - Uses a selection of optimizations in order to maximize performance.
    Specifying "-Ofast" is equivalent to -O3 -ipa -OPT:Ofast -fno-math-errno -ffast-math.
    These optimization options are generally safe. Floating-point accuracy may be affected due to the transformation of the computational code. Note the interprocedural analysis option, -ipa, specifies limitations on how libraries and object files (.o files) are built.

  - Includes:
    - -O3
      - -O2
        - -O1
    - -ipa
    - -OPT:Ofast
      - -OPT:ro
      - -OPT:Olimit
      - -OPT:div_split
      - -OPT:alias
    - -fno-math-errno
    - -ffast-math
- - -OPT:malloc_alg=1
  - COPTIMIZE

  - -OPT:malloc_algorithm,malloc_alg=(0|1|2)
    To improve runtime speed the compiler will select an optimal malloc algorithm. To enable the selected algorithm, setup code is included in the C/C++ and Fortran main function.

    - n=0: Default, no changes to the malloc options. No call to mallopt() is made.
    - n=1: M_MMAP_MAX=2 and M_TRIM_THRESHOLD=0x10000000. Call mallopt with the two settings.
    - n=2: M_MMAP_MAX=2 and M_TRIM_THRESHOLD=0x40000000. Call mallopt with the two settings.

    The two parameters, M_MMAP_MAX and M_TRIM_THRESHOLD, are described below.

    Function: int mallopt (int param, int value) When calling mallopt, the param argument specifies the parameter to be set, and value the new value to be set. Possible choices for param, as defined in malloc.h, are:

    - M_MMAP_MAX The maximum number of chunks to allocate with mmap. Setting this to zero disables all use of mmap.
    - M_TRIM_THRESHOLD This is the minimum size (in bytes) of the top-most, releasable chunk that will cause sbrk to be called with a negative argument in order to return memory to the system.
- - -HP:bd=2m:heap=2m
  - COPTIMIZE, FOPTIMIZE

  - Instructs the compiler to use 2MB hugepages for bss, data and text segments (i.e. bdt), and/or for heap allocation. Mixed usage of huge and small pages is not supported for bdt, but is supported for heap allocation. When the -HP:bd option is specified, the text segment is not mapped to huge pages while the bss and data segments still get mapped to 2MB hugepages. The limit option specifies a combined limit on the number of hugepages that may be used by the compiled program. If no limit is set, the number of hugepages that can be used by the program is effectively limited by the system configuration.

- - -IPA:plimit=8000
  - COPTIMIZE

  - -IPA:plimit=N: The compiler is instructed to halt inlining within a program once the intermediate representation indicates that the code size of the program has surpassed the limit set by N. The default is "-IPA:plimit=2500".

- - -IPA:small_pu=100
  - COPTIMIZE

  - -IPA:small_pu=N: The compiler is instructed not to restrict a procedure from inlining with a code size smaller than N when invoking the "-IPA:plimit" flag. The default is "-IPA:small_pu=30".

- - -mso
  - COPTIMIZE, FOPTIMIZE

  - Instructs the compiler to perform aggressive optimizations that are likely to improve the scalability of an application running on a system with multi-core processors. In particular, these optimizations may target machine resources that are shared among the multiple cores of a processor, e.g. memory bandwidth, shared L3 cache, etc.

- - -march=bdver1
  - COPTIMIZE, FOPTIMIZE

  - Compiler will generate instructions and schedule them appropriately for the selected processor type. The default value, auto, means to optimize for the platform on which the compiler is running, as determined by reading /proc/cpuinfo. anyx86 means a generic 32-bit x86 processor without SSE2 support.

- - -mno-fma4
  - COPTIMIZE, FOPTIMIZE

  - Enables/disables the use of FMA4 instructions on targets which support them. Specifying "-march=bdver1" enables this option, else the default is "-mno-fma4". Note specifying option "-mno-avx" implies option "-mno-fma4"

- - -mno-xop
  - COPTIMIZE, FOPTIMIZE

  - Enables/disables the use of XOP instructions on targets which support them. Specifying ‘-march=bdver1’ enables this option, else the default is ‘-mno-xop’. Note specifying option ‘-mno-avx’ implies option ‘-mno-xop’.

- - -mno-tbm
  - COPTIMIZE, FOPTIMIZE

  - Turns on/off support for tail bit manipulation(TBM) instruction set. By default ‘-mtbm’ is turned ON for bdver2 and above architectures.

- - -WB,
  - COPTIMIZE, FOPTIMIZE

  - When IPO is used, "-WB," passes the following flag (e.g. -mno-fma4) to the code generator.

- - -Wl,
  - COPTIMIZE, FOPTIMIZE

  - -Wl tells the linker to accept the following argument. In the example, it tells the linker to allow multiple definitions.

- - -z,muldefs
  - COPTIMIZE, FOPTIMIZE

  - The example, "-Wl, -z,muldefs", tells the linker to allow multiple definitions and use the first one encountered.

- - -LNO:blocking=off
  - FOPTIMIZE

  - This option group commands the compiler loop nest optimizer to perform nested loop analysis and transformations. Note an optimization level of "-O3" or higher must be specified in order to enable the "-LNO:" options. To verify the LNO options that were invoked during compilation use the option "-LIST:all_options=ON".

    -LNO:blocking=(on|off|0|1): Instructs the compiler to perform cache blocking transformation. The default is "-LNO:blocking=ON".

- - -LNO:simd_peel_align=on
  - FOPTIMIZE

  - Instrucuts the compiler to create multiple copies of loops with different alignment assumptions. The compiler adds a runtime test to branch to particular loop when alignment assumptions are met. As necessary, the compiler will peel one or more iterations to improve alignment in alternate loop copies. The default is "-LNO:simd_peel_align=OFF".

- - -OPT:rsqrt=2
  - FOPTIMIZE

  - -OPT:rsqrt=(0|1|2)
    Instructs the compiler to use the reciprocal square root instruction when calculating the square root. This transformation may vary the accuracy slightly.
    0 Restrain from using the reciprocal square root instruction.
    1 Use the reciprocal square root instruction followed by operations that will improve the accuracy of the results.
    2 Use the reciprocal square root instruction without improving the result accuracy.
    Note "-OPT:rsqrt=1" if "-OPT:roundoff=2" or "-OPT:roundoff=3". The default is "-OPT:rsqrt=0".

- - -OPT:unroll_size=256
  - FOPTIMIZE

  - -OPT:unroll_size=N
    Instructs the compiler to limit the number of instructions produced when unrolling inner loops. When N=0 the ceiling is disregarded. Note by specifying "-O3" sets "-OPT:unroll_size=128". The default is "-OPT:unroll_size=40".

- - -L/root/jemalloc/lib64/lib -ljemalloc
  - EXTRA_FLIBS

  - -L<library directory> -ljemalloc ,
    when used as an EXTRA_CXXLIB variable, results in linking with jemalloc library for Linux. This is a library that optimizes calls to new, delete, malloc and free.

## Implicitly Included Flags

This section contains descriptions of flags that were included implicitly by other flags, but which do not have a permanent home at SPEC.

- - -O3

  - Perform all the optimizations at the -O2 level as well as many more aggressive optimizations. Examples of such aggressive optimizations are loop nest optimizations and generation of prefetch instructions. Although these more aggressive optimizations can significantly speed up the run time execution of the compiled program, in rare cases they may not be profitable and may instead lead to a slow down. Also, some of these more aggressive optimizations may affect the accuracy of some floating point computations.

    If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective. Level 2 is assumed if no value is specified (i.e. "-O". The default is "-O2".

  - Includes:
    - -O2
      - -O1
- - -O2

  - Perform extensive global optimizations. Examples of such optimizations are control flow optimizations, partial redundancy elimination, and strength reduction. These optimizations can very often reduce the execution time of the compiled program significantly, but they may do so at the expense of increased compilation time. This is the default level of optimization.

    If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective. Level 2 is assumed if no value is specified (i.e. "-O". The default is "-O2".

  - Includes:
    - -O1
- - -O1

  - Perform minimal local optimizations on sections of straight-line code (basic blocks) only. Examples of such optimizations are instruction scheduling and some peephole optimizations. These optimizations do not usually have any noticeable impact on compilation time.

    If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective. Level 2 is assumed if no value is specified (i.e. "-O". The default is "-O2".

- - -ipa

  - Instructs the compiler to invoke inter-procedural analysis. Specifying "-ipa" is equivalent to "-IPA" and "-IPA:" with no suboptions, thus the default settings for the individual IPA suboptions are used.

- - -OPT:Ofast

  - -OPT:Ofast
    Maximizes performance for a given platform using the selected optimizations. "-OPT:Ofast" specifies four optimizations; "-OPT:ro=2", "-OPT:Olimit=0", "-OPT:div_split=ON", and "-OPT:alias=typed". Note the specified optimizations are ordinarily safe but floating point accuracy due to transformations may be diminished.

  - Includes:
    - -OPT:ro
    - -OPT:Olimit
    - -OPT:div_split
    - -OPT:alias
- - -OPT:ro

  - -OPT:roundoff,ro=(0|1|2|3)
    "-OPT:roundoff" specifies acceptable levels of divergence for both accuracy and overflow/underflow behavior of floating-point results relative to the source language rules. The roundoff value is in the range 0-3 with each value described as follows:
    0 Do no transformations which could affect floating-point results. The default for optimization levels "-O0", "-O1", and "-O2".
    1 Allow all transformations which have a limited affect on floating-point results. For roundoff, limited is defined as only the last bit or two of the mantissa is affected. For overflow or underflow, limited is defined as intermediate results of the transformed calculation may overflow or underflow within a factor of two of where the original expression may have overflowed or underflowed. Note that effects may be less limited when compounded by multiple transformations. This is the default when "-O3" is specified.
    2 Specifies transformations with extensive effects on floating-point results. For example, allow associative rearrangement (i.e. even across loop iterations) and the distribution of multiplication over addition or subtraction. Do not specify transformations known to cause: a. cumulative roundoff errors, or b. overflow/underflow of operands in a large range of valid floating-point values. This is the default when specifying "-OPT:Ofast".
    3 Specify any mathematically valid transformation of floating-point expressions. For example, floating point induction variables in loops are permitted (even if known to cause cumulative roundoff errors). Also permitted are fast algorithms for complex absolute value and divide (which will overflow/underflow for operands beyond the square root of the representable extremes).

- - -OPT:Olimit

  - -OPT:Olimit=N
    Controls the size of procedures to be optimized. Procedures above the specified cutoff limit, N, are not optimized. N=0 means "infinite Olimit", which causes all procedures to be optimized with no consideration regarding compilation times. Note if "-OPT:Ofast" is enabled then "-OPT:Olimit=0" or when "-O3" is enabled "-OPT:Olimit=9000". The default is "-OPT:Olimit=6000".

- - -OPT:div_split

  - -OPT:div_split=(on|off|0|1)
    Instruct the compiler to transform x/y into x*(recip(y)). Flags -OPT:Ofast or -OPT:IEEE_arithmetic=3 will enable this optimization. Note this transform generates fairly accurate code. The default is "-OPT:div_split=OFF".

- - -OPT:alias

  - The "-OPT:" option group controls various optimizations. The "-OPT:" options supersede the defaults that are based on the main optimization level.

    -OPT:alias=<model>
    Identify which pointer aliasing model to use. The compiler will make assumptions during compilation when one or more of the following <model> is specified:

    typed
    Assumes that two pointers of different types will not point to the same location in memory (i.e. the code adheres to the ANSI/ISO C standards). Note when specifying "-OPT:Ofast" turns this option ON.

    (restricted|restrict)
    Assumes that distinct pointers are pointing to distinct non-overlapping objects. The default is that this optimization is disabled.

    disjoint
    Assumes that any two pointer expressions are pointing to distinct non-overlapping objects. This default is that this optimization is disabled.

    ^M field_sensitive
    ^M Replaces the alias algorithm with an alternate implementation that tracks fields of individual pointers, i.e. it is field sensitive. This alternate implementation is also designed to be flow-insensitive, scalable and context-sensitive to heap allocations.

    no_f90_pointer_alias
    Assumes that any two different Fortran 90 pointers are pointing to distinct non-overlapping objects. The default is that this optimization is disabled.

- - -fno-math-errno

  - Do not set ERRNO after calling math functions that are executed with a single instruction, e.g. sqrt. A program that relies on IEEE exceptions for math error handling may want to use this flag for speed while maintaining IEEE arithmetic compatibility. Note specifying "-Ofast" implies "-fno-math-errno". The default is "-fmath-errno".

- - -ffast-math

  - "-fast-math" instructs the compiler to relax ANSI/ISO or IEEE rules/specifications for math functions in order to optimize floating-point computations to improve runtime. "-fno-fast-math" instructs the compiler to conform to ANSI and IEEE math rules. This option causes the preprocessor macro __FAST_MATH__ to be defined.
    Note:
    "-Ofast" implies "-ffast-math".
    "-ffast-math" sets options "-fno-math-errno" and "-OPT:IEEE_arithmetic=2".
    "-fno-fast-math" sets options "-fmath-errno" and "-OPT:IEEE arithmetic=1".

## Commands and Options Used to Submit Benchmark Runs

Using numactl to bind processes and memory to cores

For multi-copy runs or single copy runs on systems with multiple sockets, it is advantageous to bind a process to a particular core. Otherwise, the OS may arbitrarily move your process from one core to another. This can effect performance. To help, SPEC allows the use of a "submit" command where users can specify a utility to use to bind processes. We have found the utility 'numactl' to be the best choice.

numactl runs processes with a specific NUMA scheduling or memory placement policy. The policy is set for a command and inherited by all of its children. The numactl flag "--physcpubind" specifies which core(s) to bind the process. "-l" instructs numactl to keep a process memory on the local node while "-m" specifies which node(s) to place a process memory. For full details on using numactl, please refer to your Linux documentation, 'man numactl'

Note that some versions of numactl, particularly the version found on SLES 10, we have found that the utility incorrectly interprets application arguments as it's own. For example, with the command "numactl --physcpubind=0 -l a.out -m a", numactl will interpret a.out's "-m" option as it's own "-m" option. To work around this problem, a user can put the command to be run in a shell script and then run the shell script using numactl. For example: "echo 'a.out -m a' > run.sh ; numactl --physcpubind=0 bash run.sh"

numactl is also used to invoke runspec so that mememory usage is spread evenly among NUMA nodes. This is accomplished as follows: runspec_command="numactl --interleave=all runspec"

## Shell, Environment, and Other Software Settings

Linux Huge Page settings

In order to take full advantage of using x86 Open64's huge page runtime library, your system must be configured to use huge pages. It is safe to run binaries compiled with "-HP" on systems not configured to use huge pages, however, you will not benefit from the performance improvements huge pages offer. To configure your system for huge pages perform the following steps:

- Create a mount point for the huge pages: "mkdir /mnt/hugepages"
- The huge page file system needs to be mounted when the systems reboots. Add the following to a system boot configuration file before any services are started: "mount -t hugetlbfs nodev /mnt/hugepages"
- Set vm/nr_hugepages=N in your /etc/sysctl.conf file where N is the maximum number of pages the system may allocate.
- Reboot to have the changes take effect.

Note that further information about huge pages may be found in your Linux documentation file: /usr/src/linux/Documentation/vm/hugetlbpage.txt

HUGETLB_LIMIT

For the x86 Open64 compiler, the maximum number of huge pages an application is allowed to use can be set at run time via the environment variable HUGETLB_LIMIT. If not set, then the process may use all available huge pages when compiled with "-HP (or -HUGEPAGE)" or a maximum of n pages where the value of n is set via the compile time flag "-HP:limit=n".

Transparent Huge Pages (THP)

THP is an abstraction layer that automates most aspects of creating, managing, and using huge pages. THP is designed to hides much of the complexity in using huge pages from system administrators and developers, as normal huge pages must be assigned at boot time, can be difficult to manage manually, and often require significant changes to code in order to be used effectively.

Set transparent_hugepage boot parameter

In the file /boot/grub/menu.lst, add the boot parameter "transparent_hugepage=never" to the OS you plan to select during boot, to instruct it to disable Transparent Huge Pages (THP). A reboot is required for this setting to take effect.

Set Ubuntu power governor to performance

To produce the best performance on Ubuntu, the system power governor must be set to performance as follows:

- Install cpufrequtils: "sudo apt install cpufrequtils"
- Edit the following system file (create it if it doesn't exist): "sudo vi /etc/default/cpufrequtils"
- Add the following line: GOVERNOR="performance"
- Save and exit your editor.
- Disable the ondemand daemon so that your changes will not be overwritten: "sudo update-rc.d ondemand disable"

ulimit -s <n>

Sets the stack size to n kbytes, or unlimited to allow the stack size to grow without limit.

ulimit -l <n>

Sets the maximum size of memory that may be locked into physical memory.

dirty_ratio

Sets the percentage limit of system memory that can hold dirty cache data until it is written out via pdflush.

swappiness

The swappiness value can range from 1 to 100. A value of 100 will cause the kernel to swap out inactive processes frequently in favor of file system performance, resulting in large disk cache sizes. A value of 1 tells the kernel to only swap processes to disk if absolutely necessary.

zone_reclaim_mode

When zone_reclaim_mode is set to 0, the kernel will allocate memory from a remote node, rather than try to reclaim memory from the local node. A value of 1 will cause the page allocator to reclaim local page caches that are not currently used before allocating remote node memory.

sync, drop_caches

Used in conjunction, the two commands, sync and drop_caches, free disk cache memory for other uses. sync writes dirty pages to disk, while drop_caches reclaims clean disk cache pages.

OMP_NUM_THREADS

Sets the maximum number of OpenMP parallel threads auto-parallelized (-apo) applications may use.

O64_OMP_AFFINITY_MAP

Specifies the thread-CPU relationship when the operating system's affinity mechanism is used to assign OpenMP threads to CPUs.

O64_OMP_SPIN_USER_LOCK

Specifies whether or not to use the user-level spin mechanism for OpenMP locks. If the variable is set to TRUE then user-level spin mechanisms are used. If the variable is set to FALSE then pthread mutexes are used. The default if the variable is not set is the same as FALSE.

powersave -f (on SuSE)

Makes the powersave daemon set the CPUs to the highest supported frequency.

/etc/init.d/cpuspeed stop (on Red Hat)

Disables the cpu frequency scaling program in order to set the CPUs to the highest supported frequency.

LD_LIBRARY_PATH

An environment variable set to include the x86 Open64 and SmartHeap libraries used during compilation of the binaries. This environment variable setting is not needed when building the binaries on the system under test.

kernel/randomize_va_space

This option can be used to select the type of process address space randomization that is used in the system, for architectures that support this feature. 0 - Turn the process address space randomization off. This is the default for architectures that do not support this feature anyways, and kernels that are booted with the "norandmaps" parameter. 1 - Make the addresses of mmap base, stack and VDSO page randomized. This, among other things, implies that shared libraries will be loaded to random addresses. Also for PIE-linked binaries, the location of code start is randomized. This is the default if the CONFIG_COMPAT_BRK option is enabled. 2 - Additionally enable heap randomization. This is the default if CONFIG_COMPAT_BRK is disabled.

O64_OMP_SPIN_COUNT

Specify the number of times the spin loops will spin at user-level before falling back to operating system schedule/reschedule mechanisms. The default value is 20000.

## Firmware / BIOS / Microcode Settings

SMT Mode

This option could be set as "Auto" and "OFF". When set as "Auto", the Simultaneous multithreading function will enable two-thread if capable. When set as "OFF", single-thread will be enabled. Default is Auto.

Determinism Slider

This option could be set as "Performance" or "Power". When set as "Performance", the Performance Determinism is enabled to minimize sample-to-sample variance. When it is set as "Power", the system will switch to Power Deterministic mode for potentially increased (≥0%, sample dependent) performance.

cTDP Control

This option is used for Configurable TDP (cTDP) which is controlled by the PCD “PcdAmdcTDP” to define TDP ceiling. It could be set as "Auto" or "Manual". When it is set as "Manual", a child option "cTDP" could be seen and the value of TDP ceiling could be set in it. Default is Auto.

Flag description origin markings:

| Indicates that the flag description came from the user flags file.
| Indicates that the flag description came from the suite-wide flags file.
| Indicates that the flag description came from a per-benchmark flags file.

For questions about the meanings of these flags, please contact the tester.
For other inquiries, please contact webmaster@spec.org
Copyright 2006-2023 Standard Performance Evaluation Corporation
Tested with SPEC CPU2006 v1.2.
Report generated on Mon Nov 27 14:16:15 2023 by SPEC CPU2006 flags formatter v6417.