Set Hugepages for Oracle 11gR2 database
1. Execute the shell script provided in Note ID: 401749.1 to find out
the ideal setting for the parameter vm.nr_hugepages, set it in /etc/sysctl.conf
and reboot VM/Server.
Script from Note 401749.1
-bash-3.2$ cat huge_pages_calc.sh
#!/bin/bash
#
# hugepages_settings.sh
#
# Linux bash script to compute values for the
# recommended HugePages/HugeTLB configuration
#
# Note: This script does calculation for all shared memory
# segments available when the script is run, no matter it
# is an Oracle RDBMS shared memory segment or not.
#
# This script is provided by Doc ID 401749.1 from My Oracle Support
# http://support.oracle.com
# Welcome text
echo "
This script is provided by Doc ID 401749.1 from My Oracle Support
(http://support.oracle.com) where it is intended to compute values for
the recommended HugePages/HugeTLB configuration for the current shared
memory segments. Before proceeding with the execution please note following:
* For ASM instance, it needs to configure ASMM instead of AMM.
* The 'pga_aggregate_target' is outside the SGA and
you should accommodate this while calculating SGA size.
* In case you changes the DB SGA size,
as the new SGA will not fit in the previous HugePages configuration,
it had better disable the whole HugePages,
start the DB with new SGA size and run the script again.
And make sure that:
* Oracle Database instance(s) are up and running
* Oracle Database 11g Automatic Memory Management (AMM) is not setup
(See Doc ID 749851.1)
* The shared memory segments can be listed by command:
# ipcs -m
Press Enter to proceed..."
read
# Check for the kernel version
KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'`
# Find out the HugePage size
HPG_SZ=`grep Hugepagesize /proc/meminfo | awk '{print $2}'`
if [ -z "$HPG_SZ" ];then
echo "The hugepages may not be supported in the system where the script is being executed."
exit 1
fi
# Initialize the counter
NUM_PG=0
# Cumulative number of pages required to handle the running shared memory segments
for SEG_BYTES in `ipcs -m | cut -c44-300 | awk '{print $1}' | grep "[0-9][0-9]*"`
do
MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q`
if [ $MIN_PG -gt 0 ]; then
NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q`
fi
done
RES_BYTES=`echo "$NUM_PG * $HPG_SZ * 1024" | bc -q`
# An SGA less than 100MB does not make sense
# Bail out if that is the case
if [ $RES_BYTES -lt 100000000 ]; then
echo "***********"
echo "** ERROR **"
echo "***********"
echo "Sorry! There are not enough total of shared memory segments allocated for
HugePages configuration. HugePages can only be used for shared memory segments
that you can list by command:
# ipcs -m
of a size that can match an Oracle Database SGA. Please make sure that:
* Oracle Database instance is up and running
* Oracle Database 11g Automatic Memory Management (AMM) is not configured"
exit 1
fi
# Finish with results
case $KERN in
'2.2') echo "Kernel version $KERN is not supported. Exiting." ;;
'2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`;
echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;;
'2.6') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
'3.8') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
esac
# End
-bash-3.2$ ./huge_pages_calc.sh
This script is provided by Doc ID 401749.1 from My Oracle Support
(http://support.oracle.com) where it is intended to compute values for
the recommended HugePages/HugeTLB configuration for the current shared
memory segments. Before proceeding with the execution please note following:
* For ASM instance, it needs to configure ASMM instead of AMM.
* The 'pga_aggregate_target' is outside the SGA and
you should accommodate this while calculating SGA size.
* In case you changes the DB SGA size,
as the new SGA will not fit in the previous HugePages configuration,
it had better disable the whole HugePages,
start the DB with new SGA size and run the script again.
And make sure that:
* Oracle Database instance(s) are up and running
* Oracle Database 11g Automatic Memory Management (AMM) is not setup
(See Doc ID 749851.1)
* The shared memory segments can be listed by command:
# ipcs -m
Press Enter to proceed...
Recommended setting: vm.nr_hugepages = 3076
2. Set memlock limit at OS level.
a. Modify /etc/security/limits.conf and enter the below values:
oracle soft
memlock
oracle hard
memlock
memlock_value =
(Huge_Pages * Huge_Page_Size)
Huge_Pages (value obtained by script in point 1):
or
sysctl -a | grep -i
vm.nr_hugepages
Huge_Page_Size:
grep Hugepagesize
/proc/meminfo
Note: You can check the current value or memlock
using ulimit -l.
3. Set kernel parameters shmmax and shmall.
kernel.shmmax = (1/2 of RAM) and sga_max_size < kernel.shmmax
So, if
sga_max_size is ½ or RAM, then set shmmax to around 60 or 70% of RAM.
kernel.shmall
= (SGA_MAX_SIZE + PGA_AGGREGATE_TARGET)/PAGE_SIZE
PAGE_SIZE_VALUE:
$ getconf PAGE_SIZE
4. Disable Automatic Memory Management (AMM) if necesary as it is
incompatible with HugePages.
Unset the init parameter memory_target parameter, if set.
5. After the reboot, start up the DB and check the below section in the
alert log:
****************** Large
Pages Information *****************
Total Shared Global Region
in Large Pages = 0 KB (0%)
Large Pages used by this
instance: 0 (0 KB)
Large Pages unused system
wide = 10000 (20 GB) (alloc incr 64 MB)
Large Pages configured
system wide = 10000 (20 GB)
Large Page size = 2048 KB
RECOMMENDATION:
Total Shared Global Region size is 24 GB. For
optimal performance,
prior to the next instance restart increase
the number
of unused Large Pages by atleast 2292 2048 KB
Large Pages (4584 MB)
system wide to get 100% of the Shared
Global Region allocated with Large pages
***********************************************************
Successful message would be as below:
****************** Large
Pages Information *****************
Total Shared Global Region
in Large Pages = 62 GB (100%)
Large Pages used by this
instance: 31745 (62 GB)
Large Pages unused system
wide = 1023 (2046 MB) (alloc incr 128 MB)
Large Pages configured
system wide = 32768 (64 GB)
Large Page size = 2048 KB
***********************************************************
Notes:
1. USE_LARGE_PAGES è
This parameter is new in 11gR2 (11.2.0.2) onwards. It is set to true by
default.
In Oracle Database 11g Release 2 (11.2.0.2), if
there are not enough large pages configured on the system, then regular sized
pages will be used to allocate SGA memory. This can cause the free large pages
to go unused, and the operating system can allocate a huge amount of memory to
create page tables to map SGA into physical pages for the Oracle processes.
This may lead to ORA-04030 errors and severe performance degradation on an
instance.
In Oracle Database 11g Release 2 (11.2.0.3), Oracle
allocates as much of the SGA as it can in large pages, and if it runs out, it
will allocate the rest of the SGA using regular sized pages. This can cause the
instance to create additional shared memory segments for the SGA, but the total
SGA size will be unchanged. In this supported mixed page mode allocation, the
database will exhaust the available large pages before switching to regular
sized pages.
References:
2. 361323.1 - HugePages on
Linux: What It Is... and What It Is Not...
3. 401749.1 - Shell Script to
Calculate Values Recommended Linux HugePages / HugeTLB Configuration
4. 361468.1 - HugePages on
Oracle Linux 64-bit
7. http://www.oracle-base.com/articles/linux/configuring-huge-pages-for-oracle-on-linux-64.php
8. 1392543.1 - Large Pages Information in the Alert Log
9. 1392497.1 - USE_LARGE_PAGES To Enable HugePages In 11.2
10. Bug 9195408 - DB STARTUP DOES NOT CHECK WHETHER HUGEPAGES ARE
ALLOCATED- PROVIDE USE_HUGEPAGES
11.
http://agorbyk.wordpress.com/2012/02/19/oracle-11-2-0-3-and-hugepages-allocation/
12. http://kevinclosson.wordpress.com/category/use_large_pages/
14. 803238.1 - Oracle Not Utilizing Hugepages
Comments