Payday loan very quickly they often decide Emergency Payday Loans Emergency Payday Loans not refusing to new one.

Home > Linux > Linux Hugepages

Linux Hugepages

October 24th, 2010

Hugepages是从Linux kernal 2.6后被引入的,其目的是使用更大的memory page size以适应越来越大的系统内存。

在我上大学那会,买一条64M 133Mhz的内存(对,你没看错,64M)价格为500多人民币,而现在4G 1600Mhz的内存的价格也就500多。

计算机硬件的发展速度太快了,所以操作系统的一些配置也要相应的随之改变。

在Linux下,默认的page size大小为4k。显然对于现在的SGA比较大的数据库系统来说,4k的page size有点太小了。

我们来看看两者之间有什么区别

1. Page Table大小

Page Table是用来存放虚拟内存也和物理内存页对应关系的内存结构。因为page size较小,所以相应的改内存结构也会比较大。

而Hugepages的常见page size为2M,是4k size的500倍,所以可以大大减小page table的size。

我们来看两个例子:

这是一个没有配置Hugepage的系统,系统内存128G,pagetable大小大约为4G。

cat /proc/meminfo

MemTotal: 132086880 kB
PageTables: 4059612 kB

这是配置了Hugepage的系统,系统内存96G, PageTable大小仅为78M

MemTotal: 98999880 kB
PageTables: 79916 kB

2. 大大提高了CPU cache中存放的page table所覆盖的内存大小,从而提高了TLB命中率

进程的虚拟内存地址段先连接到page tables然后再连接到物理内存。所以在访问内存时需要先访问page tables得到虚拟内存和物理内存的映射关系,然后再访问物理内存。

CPU cache中有一部分TLB(Translation Lookaside Buffer)用来存放部分page table以提高这种装换的速度。因为page size变大了,所以同样大小的TLB,所覆盖的内存大小也变大了。提高了TBL命中率,也就是提高了地址转换的速度。

3. 使用Hugepages的内存页是不会被交换出去的,永远常驻在内存中,所以也减少了内存也替换的额外开销

下面再说说在数据库服务器上使用Hugepages要注意的几点

1. Hugepages是在分配后就会预留出来的,其大小一定要比服务器上所有实例的SGA总和要大,差一点都不行。

比如说Hugepages设置为90G,oracle SGA为91G,那么oracle在启动的时候就不会使用到这90G的Hugepages。这90G就浪费了。所以在设置Hugepages时要计算SGA的大小,后面会给出一个脚本来计算。

2. 其他进程无法使用Hugepages的内存,所以不要设置太大,稍稍比SGA大一点保证SGA可以使用到hugepages就好了。

3. PGA不会使用Hugepages的内存。所以11g的AMM (Automatic Memory Management,memory_target参数)是不被支持的。而ASMM(Automatic Shared Memory Management, SGA_target参数)是被支持的,这两个不要搞混淆了。

4. 在meminfo中和Hugepage相关的有四项(RHEL5)

HugePages_Total: 43000
HugePages_Free: 29493
HugePages_Rsvd: 23550
Hugepagesize: 2048 kB

HugePages_Total为所分配的页面数目,和Hugepagesize相乘后得到所分配的内存大小。43000*2/1024大约为84GB
HugePages_Free为从来没有被使用过的Hugepages数目。即使oracle sga已经分配了这部分内存,但是如果没有实际写入,那么看到的还是Free的。这是很容易误解的地方
HugePages_Rsvd为已经被分配预留但是还没有使用的page数目。在Oracle刚刚启动时,大部分内存应该都是Reserved并且Free的,随着oracle SGA的使用,Reserved和Free都会不断的降低

HugePages_Free – HugePages_Rsvd 这部分是没有被使用到的内存,如果没有其他的oracle instance,这部分内存也许永远都不会被使用到,也就是被浪费了。在该系统上有11.5GB的内存被浪费了。

Note: RHEL4上的meminfo有所区别,没有HugePages_Rsvd这一项,并且当oracle instance启动时,所分配的内存就从free list上被移除掉了。也就是启动后HugePages_Free就是没有被SGA用到被浪费的内存。

最后说说如何设置HugePages:

1. 首先计算SGA大小已决定你要使用多少HugePages内存页。

你可以手工计算,如果使用了ASMM可以用SGA_Target/Hugepagesize,否则可以将db_cache_size,large_pool_size, shared_pool_size,jave_pool_size, streams_pool_size五个部分加起来除以Hugepagesize。

或者可以先将oracle instance都起起来,然后ipcs -m查看共享内存段大小来计算。oracle在401749.1中也提供了一个脚本来帮助计算,脚本如下:

#!/bin/bash
#
# hugepages_settings.sh
#
# Linux bash script to compute values for the
# recommended HugePages/HugeTLB configuration
#
# Note: This script does calculation for all shared memory
# segments available when the script is run, no matter it
# is an Oracle RDBMS shared memory segment or not.
#
# This script is provided by Doc ID 401749.1 from My Oracle Support
# http://support.oracle.com

# Welcome text
echo "
This script is provided by Doc ID 401749.1 from My Oracle Support
(http://support.oracle.com) where it is intended to compute values for
the recommended HugePages/HugeTLB configuration for the current shared
memory segments. Before proceeding with the execution please make sure
that:
 * Oracle Database instance(s) are up and running
 * Oracle Database 11g Automatic Memory Management (AMM) is not setup
   (See Doc ID 749851.1)
 * The shared memory segments can be listed by command:
     # ipcs -m

Press Enter to proceed..."

read

# Check for the kernel version
KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'`

# Find out the HugePage size
HPG_SZ=`grep Hugepagesize /proc/meminfo | awk '{print $2}'`

# Initialize the counter
NUM_PG=0

# Cumulative number of pages required to handle the running shared memory segments
for SEG_BYTES in `ipcs -m | awk '{print $5}' | grep "[0-9][0-9]*"`
do
   MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q`
   if [ $MIN_PG -gt 0 ]; then
      NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q`
   fi
done

RES_BYTES=`echo "$NUM_PG * $HPG_SZ * 1024" | bc -q`

# An SGA less than 100MB does not make sense
# Bail out if that is the case
if [ $RES_BYTES -lt 100000000 ]; then
   echo "***********"
   echo "** ERROR **"
   echo "***********"
   echo "Sorry! There are not enough total of shared memory segments allocated for
HugePages configuration. HugePages can only be used for shared memory segments
that you can list by command:

   # ipcs -m

of a size that can match an Oracle Database SGA. Please make sure that:
 * Oracle Database instance is up and running
 * Oracle Database 11g Automatic Memory Management (AMM) is not configured"
   exit 1
fi

# Finish with results
case $KERN in
   '2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`;
          echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;;
   '2.6') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
    *) echo "Unrecognized kernel version $KERN. Exiting." ;;
esac

# End

2. 关闭所有oracle实例

3. 用root设定oracle memlock limit,设置一个较大的数值或者unlimited

在/etc/security/limits.conf最后添加
oracle hard memlock unlimited
oracle soft memlock unlimited

4. 分配hugepages内存

在/etc/sysctl.conf中添加

vm.nr_hugepages =

执行sysctl -p使其生效。这时候内存就已经被分配了,可以查看meminfo

grep Huge /proc/meminfo

HugePages_Total为设定的值大小,HugePages_Free应该和HugePages_Total一样大,HugePages_Rsvd为0.

5. 启动Oracle instance

这时候再次查看meminfo

HugePages_Total为设定的值大小不变,HugePages_Free有所降低,HugePages_Rsvd为一个较大的数值(因为刚刚启动时,大部分SGA被分配但是没有被使用到)。

如果Hugepages没有被使用,可能一些memory page被分配为4k大小了,那么需要重启server来设置。

从我们的测试结果看,Hugepages可以提高OLTP系统10%的吞吐量,当然不同的数据库应用结果可能不同,但是总体来说这是一个nice to have的设置。

推荐一些Hugepages相关的文章:

Metalink Note: 361323.1,744769.1, 748637.1
Yong’s blog: http://yong321.freeshell.org/oranotes/HugePages.txt
Kevin Closson’s blog: http://kevinclosson.wordpress.com/2010/09/28/configuring-linux-hugepages-for-oracle-database-is-just-too-difficult-part-i/ http://kevinclosson.wordpress.com/2010/10/21/configuring-linux-hugepages-for-oracle-database-is-just-too-difficult-isn%E2%80%99t-it-part-%E2%80%93-ii/

Eagle Fan Linux ,

  1. August 9th, 2013 at 23:17 | #1

    I precisely wished to thank you so much all over again. I do not know the things I would have achieved in the absence of those advice discussed by you directly on my topic. It previously was a very frustrating setting in my view, however , understanding this skilled fashion you managed it made me to cry over delight. I am just thankful for this advice and in addition believe you find out what an amazing job you have been getting into training people through the use of your website. I know that you’ve never come across all of us.

  2. August 10th, 2013 at 15:32 | #2

    I’m impressed, I have to admit. Definitely hardly ever must i encounter a blog that is both educative and entertaining, and without a doubt, you may have hit the nail around the head. Your notion is outstanding; the catch is one thing not adequate customers are speaking intelligently about. We’re delighted i stumbled across this inside my obtain some thing relating to this.

  3. August 22nd, 2013 at 21:34 | #3

    I would like to get across my admiration for your kindness for those individuals that have the need for assistance with your topic. Your special dedication to getting the solution along appeared to be surprisingly insightful and has continually made professionals much like me to arrive at their goals. Your own invaluable guide implies a lot to me and much more to my colleagues. Regards; from everyone of us.

  4. August 23rd, 2013 at 04:14 | #4

    It’s arduous to seek out knowledgeable folks on this topic, however you sound like you recognize what you’re talking about! Thanks

  5. September 5th, 2013 at 13:13 | #5

    I’d need to verify with you here. Which is not one thing I often do! I take pleasure in studying a submit that will make folks think. Additionally, thanks for permitting me to comment!

  6. September 22nd, 2013 at 16:45 | #6

    I beloved as much as you will obtain conducted right here. The caricature is elegant, your published subject stylish. nevertheless, you management get bought an shakiness over that you want be http://www.rs2joy.com/ offering the following. fed up indubitably come more before again as exactly the similar just about very consistently within scenario you protected this enhance.

  7. October 18th, 2013 at 20:35 | #7

    I rattling pleased to find this website on bing, just what I was searching for : D as well saved to fav.

  8. March 14th, 2014 at 08:35 | #8

    You have verified you’re eligible to produce within this theme. Information that you simply refer to and also the understanding and also perception of these materials evidently reveal that there is a lot of encounter.

  9. May 20th, 2014 at 20:55 | #9

    Je souhaiterai vous dire que c’est sans mentir une
    joie de vous lire

  10. June 8th, 2014 at 00:53 | #10

    Great write-up, I¡¦m regular visitor of one¡¦s blog, maintain up the excellent operate, and It is going to be a regular visitor for a long time.

  11. June 9th, 2014 at 01:45 | #11

    magnificent issues altogether, you just received a logo new reader. What may you recommend about your put up that you simply made a few days ago? Any certain?

  12. June 9th, 2014 at 04:52 | #12

    Hello There. I found your blog using msn. This is a really well written article. I will make sure to bookmark it and come back to read more of your useful information. Thanks for the post. I’ll certainly return.

  13. June 9th, 2014 at 17:24 | #13

    Great tremendous things here. I¡¦m very glad to look your article. Thank you so much and i am taking a look ahead to contact you. Will you please drop me a mail?

  14. June 10th, 2014 at 00:08 | #14

    I’m still learning from you, but I’m trying to reach my goals. I absolutely enjoy reading all that is posted on your blog.Keep the stories coming. I liked it!

  15. June 10th, 2014 at 01:04 | #15

    There is noticeably a lot to realize about this. I consider you made some good points in features also.

  16. June 12th, 2014 at 13:53 | #16

    I have been surfing online more than 3 hours today, yet I never found any interesting article like yours. It is pretty worth enough for me. In my opinion, if all web owners and bloggers made good content as you did, the internet will be much more useful than ever before.

  17. June 12th, 2014 at 14:39 | #17

    Normally I do not learn post on blogs, however I wish to say that this write-up very forced me to take a look at and do so! Your writing style has been amazed me. Thanks, quite great article.

  18. June 12th, 2014 at 18:51 | #18

    I’ve been surfing online more than 3 hours today, but I never discovered any attention-grabbing article like yours. It¡¦s pretty worth enough for me. In my view, if all web owners and bloggers made good content material as you did, the internet can be much more useful than ever before.

  19. June 12th, 2014 at 19:40 | #19

    Great tremendous things here. I¡¦m very glad to look your article. Thank you so much and i am taking a look ahead to contact you. Will you please drop me a mail?

  20. June 12th, 2014 at 22:48 | #20

    Well I sincerely liked studying it. This tip offered by you is very helpful for good planning.

  21. June 12th, 2014 at 23:03 | #21

    I have to get across my admiration for your generosity supporting persons who really want assistance with the area. Your very own dedication to getting the message all over has been extraordinarily functional and has in every case enabled ladies like me to reach their endeavors. Your new helpful publication signifies a great deal to me and far more to my colleagues. With thanks; from each one of us.

  22. June 12th, 2014 at 23:14 | #22

    Thank you so much for giving everyone an extraordinarily special opportunity to read critical reviews from this site. It’s always very enjoyable and jam-packed with amusement for me and my office colleagues to search your web site a minimum of thrice in a week to see the newest tips you have. And definitely, we are at all times amazed with all the wonderful tactics you give. Selected 1 areas in this post are particularly the finest we have all had.

  23. June 13th, 2014 at 00:57 | #23

    I think other site proprietors should take this website as an model, very clean and great user genial style and design, let alone the content. You’re an expert in this topic!

  24. June 13th, 2014 at 01:14 | #24

    I’m still learning from you, but I’m trying to reach my goals. I absolutely enjoy reading all that is posted on your blog.Keep the stories coming. I liked it!

  25. Learn More
    June 16th, 2014 at 08:43 | #25

    Thanks , I have recently been searching for info about this topic for ages and yours is the best I’ve found out till now. But, what concerning the bottom line? Are you sure about the source?

  26. June 16th, 2014 at 09:59 | #26

    Hello.This post was extremely interesting, particularly because I was looking for thoughts on this topic last Thursday.

  27. June 16th, 2014 at 10:44 | #27

    Great write-up, I¡¦m regular visitor of one¡¦s blog, maintain up the excellent operate, and It is going to be a regular visitor for a long time.

  28. June 16th, 2014 at 17:04 | #28

    Hello.This post was extremely interesting, particularly because I was looking for thoughts on this topic last Thursday.

  29. June 23rd, 2014 at 16:58 | #29

    Definitely, what a splendid blog and illuminating posts, I surely will bookmark your site.All the Best!

  30. June 24th, 2014 at 21:48 | #30

    Thanks for another informative blog. The place else could I get that kind of information written in such a perfect means? I’ve a project that I’m simply now working on, and I’ve been on the glance out for such info.

  31. June 28th, 2014 at 19:46 | #31

    magnificent issues altogether, you just received a logo new reader. What may you recommend about your put up that you simply made a few days ago? Any certain?

Comment pages
  1. July 2nd, 2012 at 20:44 | #1