Linux Kernel Tuning for Oracle Database 10g

Linux Kernel Tuning for Oracle 10G
==========================

Note:
Physical memory is 256GB
SHMALL is 240GB in sysctl.conf
SHMMAX is 210GB in sysctl.conf
sga_max_size is 200GB in init.ora
processes is 5000 in init.ora


- Make sure i/o scheduler to deadline (/etc/grub.conf)

# cat /sys/block/sd?/queue/scheduler
noop [deadline] cfq

title Oracle Linux Server (2.6.32-300.25.1.el6uek.x86_64)
        root (hd0,0)
        kernel /vmlinuz-2.6.32-300.25.1.el6uek.x86_64 ro root=LABEL=/ elevator=deadline
        initrd /initramfs-2.6.32-300.25.1.el6uek.x86_64.img

Note. deadline scheduler is default in OL5 UEK2Q5.


- Make sure transparent_hugepage is disabled (/etc/grub.conf)

title Oracle Linux Server (2.6.32-300.25.1.el6uek.x86_64)
        root (hd0,0)
        kernel /vmlinuz-2.6.32-300.25.1.el6uek.x86_64 ro root=LABEL=/ transparent_hugepage=never
        initrd /initramfs-2.6.32-300.25.1.el6uek.x86_64.img

Note: there is no /sys/kernel/mm/transparent_hugepage in OL5 UEK2Q5
https://oss.oracle.com/el5/docs/RELEASE-NOTES-UEK2-QU5-en.html


- Kernel shared memory

kernel.shmall = 240*1024*1024*1024/4096
kernel.shmmax = 210*1024*1024*1024


- Kernel semaphores

check processes parameter in init.ora
SEMMSL = processes + 10
SEMMNS = processes * 128
SEMOPM = processes + 10
SEMMNI = 128

kernel.sem = SEMMSL SEMMNS SEMOPM SEMMNI


- Enable manual HugePages

check sga_max_size parameter in init.ora

X = Hugepagesize (# grep Hugepage /proc/meminfo)
Y = sga_max_size (in KB)
Z = Y / X

vm.nr_hugepages = Z + 10


- Put this settings into /etc/sysctl.conf

net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 262144
net.core.wmem_default = 262144
net.core.rmem_max = 262144
net.core.wmem_max = 262144
net.ipv4.tcp_rmem = 4096 262144 4194304
net.ipv4.tcp_wmem = 4096 262144 262144
kernel.shmall = 62914560
kernel.shmmax = 225485783040
kernel.shmmni = 4096
kernel.sem = 5010 641280 5010 128
vm.nr_hugepages =  102410
vm.swappiness = 0
vm.dirty_background_ratio = 3
vm.dirty_ratio = 15
vm.dirty_expire_centisecs = 500
vm.dirty_writeback_centisecs = 100
vm.min_free_kbytes = 1024


- Set ulimit parameter for oracle (/etc/security/limits.conf)

Note. memlock is sga_max_size in KB.

oracle soft memlock 209715200
oracle hard memlock 209715200
oracle          soft    nofile          63536
oracle          hard    nofile          63536
oracle          soft    nproc           16384
oracle          hard    nproc           16384


- Reboot your server and start your database

- Make sure oracle use hugepages

# grep Huge /proc/meminfo

HugePages_Total:   102410
HugePages_Free:    56268
HugePages_Rsvd:    55235
HugePages_Surp:        0
Hugepagesize:       2048 kB


- Check the HugePages_Rsvd, if it is still 0, try to reduce your sga_max_size parameter in pfile/spfile without change sysctl.conf and limits.conf anymore.
For example, try sga_max_size is 198GB.
Shutdown and start the database, and make sure HugePages_Rsvd is not empty.