Benefit to use Linux Hugepages with Oracle

Benefit to use Linux Hugepages with Oracle

Introduction

The memory is managed in blocks called pages. A page is 4096 bytes and then 1MB of memory is equivalent to 256 pages. The CPUs have an integrated memory management unit which contains a list of these pages.

Hugepages are Linux Kernel features that allows to manage memory other than 4KB page. Hugepages can give substantial benefits in virtual memory management. Without Hugepages, the memory of the SGA is divided into 4KB pages, which have to be managed by the Linux Kernel. Using Hugepages, the pages size is increased to 2MB (configurable to 1GG if supported by the hardware). Thereby reducing the total number of pages to be managed by the kernel and therefore reducing the amount of memory required to hold the page table in memory.

If you run an Oracle Database on Linux Server with more than 16GB physical memory and your System Global Area (SGA) is greater than 8GB, you should configure Hugepages. Oracle promise more performance by doing this. But, because you have a large SGA, it doesn’t automatically mean you will have a problem if you don’t use Hugepages.

However, there is a limitation by Oracle, because Automatic Memory Management (AMM) does not support Hugepages. If you already use AMM and MEMORY_TARGET is set, you have to disable and switch back to Automatic Shared Memory Management (ASMM) That means to set SGA_TARGET and PGA_AGGREGATE_TARGET.

Configuring HugePages

1.       Check the current Hugepage Usage

The default HugePage size is 2MB from Linux 5.x. You can run the following command to determine the current HugePage:

[mbadi_d@MyServer ~]$ grep Huge /proc/meminfo

AnonHugePages:         0 kB

HugePages_Total:   66560

HugePages_Free:    12797

HugePages_Rsvd:        0

HugePages_Surp:        0

Hugepagesize:       2048 kB

2.       Check the current Memory and Database parameter

You should check the physical available memory and be sure that SGA_TARGET and PGA_AGGREGATE_TARGET together should not be more than the available memory.

[oracle@MyServer ~]$   grep MemTotal /proc/meminfo

MemTotal:       263850180 kB

SQL> show parameter sga_target

NAME                          TYPE        VALUE

-------------------------- ----------- -----------------------

sga_target                  big integer  100G

SQL> show parameter pga

NAME                          TYPE        VALUE

--------------------------- ----------- ----------------------

pga_aggregate_limit                  big integer 25710M

pga_aggregate_target                 big integer 12855M

SQL>

Since AMM disabled, MEMORY_TARGET and MEMORY_MAX_TARGET should be set to 0.

SQL> show parameter memory

NAME                      TYPE        VALUE

-------------------- ----------- ----------------

………                   …………             …………

………                   …………             …………

memory_max_target     big integer       0

memory_target         big integer       0

………

………

SQL>

If you’re using an SPFILE, and these parameters are explicitly set to non-zero values, run the following commands to unset them (that is, set them to “0”) in the SPFILE:

SQL> alter system reset memory_target scope=spfile;

SQL> alter system reset memory_max_target scope=spfile;

If these parameters are not explicitly set (that is, set to 0), you do not need to unset them, and the preceding commands will accordingly error out with “ORA-32010: cannot find entry to delete in SPFILE.”

You must check the use_large_pages parameter. This will help you to force later force Oracle to use HugePages:

SQL> show parameter large

NAME                     TYPE        VALUE

---------------------- ----------- -------------------

large_pool_size         big integer 0

use_large_pages         string      TRUE

 

3.       Calculate HugePages

For the calculation of the number of hugepages there is a easy way:

SGA / Hugepagesize = Number Hugepages

Following our example:

104857600 / 2048 = 51200

If you run more than one database on your server, you should include the SGA of all of your instances into the calculation:

( SGA 1. Instance + SGA 2. Instance + … etc. ) / Hugepagesize = Number Hugepages

You can create a file called "hugepages_setting.sh" with the following contents:

#!/bin/bash

#

# hugepages_settings.sh

#

# Linux bash script to compute values for the

# recommended HugePages/HugeTLB configuration

#

# Note: This script does calculation for all shared memory

# segments available when the script is run, no matter it

# is an Oracle RDBMS shared memory segment or not.

# Check for the kernel version

KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'`

# Find out the HugePage size

HPG_SZ=`grep Hugepagesize /proc/meminfo | awk {'print $2'}`

# Start from 1 pages to be on the safe side and guarantee 1 free HugePage

NUM_PG=1

# Cumulative number of pages required to handle the running shared memory segments

for SEG_BYTES in `ipcs -m | awk {'print $5'} | grep "[0-9][0-9]*"`

do

   MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q`

   if [ $MIN_PG -gt 0 ]; then

      NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q`

   fi

done

# Finish with results

case $KERN in

   '2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`;

          echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;;

   '2.6' | '3.8' | '3.10' | '4.1' ) echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;

    *) echo "Unrecognized kernel version $KERN. Exiting." ;;

esac

# End

Make the file executable.

[oracle@MyServer ~]$ chmod u+x hugepages_setting.sh

Make sure all the Oracle services are running as normal on the server, then run the script and make a note of the recommended "vm.nr_hugepages" value.

[oracle@MyServer ~]$ ./hugepages_setting.sh

You should have the below output :

Recommended setting: vm.nr_hugepages = 51200

$

4.       Change Server Configuration

Edit the "/etc/sysctl.conf" file as the "root" user, adding the following entry, adjusted based on your output from the script. You should set the value greater than or equal to the value displayed by the script.

#vi /etc/sysctl.conf

vm.nr_hugepages=51200

Correctly inserted, following result should show up:

#grep vm.nr_hugepages /etc/sysctl.conf

vm.nr_hugepages=51200

Run the following command as the "root" user to reload the parameter.

# sysctl -p

As root user, Add the following entries into the "/etc/security/limits.conf" script or "/etc/security/limits.d/99-grid-oracle-limits.conf" script, where the setting is at least the size of the HugePages allocation in KB (HugePages * Hugepagesize). In this case the value is 51200*2048=104857600.

vi /etc/security/limits.conf

oracle               soft    memlock 104857600

oracle               hard    memlock 104857600

Correctly inserted, following result should show up:

grep oracle /etc/security/limits.conf

...

oracle               soft    memlock 104857600

oracle               hard    memlock 104857600

5. Force Oracle to use HugePages (USE_LARGE_PAGES)

Sizing the number of HugePages correctly is important because prior to 11.2.0.3, if the whole SGA doesn't fit into the available HugePages, the instance will start up without using any. From 11.2.0.3 onward, the SGA can run partly in HugePages and partly not, so the impact of this issue is not so great. Incorrect sizing may not be obvious to spot. Later releases of the database display a "Large Pages Information" section in the alert log during startup.

****************** Large Pages Information *****************

Total Shared Global Region in Large Pages = 602 MB (100%)

Large Pages used by this instance: 301 (602 MB)

Large Pages unused system wide = 5 (10 MB) (alloc incr 4096 KB)

Large Pages configured system wide = 51200 (612 MB)

Large Page size = 2048 KB

***********************************************************

If you are running Oracle 11.2.0.2 or later, you can set the USE_LARGE_PAGES initialization parameter to "ONLY" so the database fails to start if it is not backed by hugepages. 

SQL>ALTER SYSTEM SET use_large_pages=ONLY SCOPE=SPFILE;

SQL>SHUTDOWN IMMEDIATE;

SQL>STARTUP;

  

After configuration, check your database performance.

Just because you have a large SGA, it doesn't automatically mean you will have a problem if you don't use HugePages. It is typically the combination of a large SGA and lots database connections that leads to problems.