AIX主机内存使用100%分析(dia0消耗大量内存)

2024-02-11ASM / Linux/AIX / Oracle / RAC

主机内存使用100%,需要通过重启主机来释放内存。通过teamviewer远程看看:

1,使用nmon查看内存使用情况

PhysicalPageSpace |pages/secInOut | FileSystemCache

% Used99.5%20.3%| to Paging Space0.00.0 | (numperm)7.3%

% Free0.5%79.7%| to File System3.70.0 | Process75.1%

MB Used31594.2MB1662.7MB | Page Scans0.0| System17.1%

MB Free149.8MB6529.3MB | Page Cycles0.0| Free0.5%

Total(MB) 31744.0MB8192.0MB | Page Steals0.0|——

Page Faults73604.8| Total100.0%

———————————————————— | numclient7.3%

Min/Maxperm3075MB( 10%)18452MB( 58%) <–% of RAM| maxclient 19.4%

Min/Maxfree9601088Total Virtual39.0GB| User79.4%

Min/Maxpgahead28Accessed Virtual29.8GB 76.4%| Pinned18.7%

这里看到剩余的物理内存只有149M,numclient,maxclient,system内存使用率都不算太高,process达到了75%。

2,统计进程内存使用情况

查看ORACLE用户进程的内存使用

htz.pw:/home/grid>ps vx|grep ora|grep -v grep|awk ‘{ print ($7-$10)/1024}’

16.1211

7.75391

6.07031

0.546875

13.0547

22.9883

13.0547

16.5625

70.6914

21.6992

22.6094

16.8867

13.7734

21.1328

21.1172

20.7852

16.8984

18.9531

19.9297

21.8945

22.3398

21.8203

5.42578

2.00781

90.0039

2.56641

0.753906

1.52344

2.40234

0.371094

1.04688

0.316406

22.1914

19.668

20.9805

21.6445

16.8984

25.2656

16.9062

17.0117

23.0078

21.0938

19.3047

查看grid内存使用情况

htz.pw:/home/grid>ps vx|grep asm|grep -v grep|awk ‘{ print ($7-$10)/1024,$1}’

1.73438 3670252

5.74609 4653094

1.21094 4915366

1.05469 4980854

1.02734 5046308

0.898438 5111870

1.89453 5243102

0.988281 5374008

1.4375 5439526

1.48047 5505070

1.51562 5570602

1.95312 5701680

1.01562 3211554

12.5898 3408140

0.628906 3604896

0.976562 3735942

2.33203 3801398

1.43359 3866920

7187.71 3998108

1.41797 4063522

1.63281 4194604

0.722656 4391226

这里可以看到3998108内存使用达到了7187.71M。

3,分析3998108进程内存使用

htz.pw:/home/grid>ps -ef|grep 3998108

grid399810815Oct 24– 4815:55 asm_dia0_+ASM1

grid891338485853160 00:36:12pts/40:00 grep 3998108

htz.pw:/home/grid>svmon -P 3998108

——————————————————————————-

Pid CommandInusePinPgspVirtual 64-bit Mthrd16MB

3998108 oracle19376932600074311922142YNN

PageSizeInusePinPgspVirtual

s4 KB1873069074311857518

m64 KB4039162504039

VsidEsid Type DescriptionPSizeInusePin Pgsp Virtual

a919a925 work text data BSS heapsm655360065536

800cc01f work text data BSS heapsm655360065536

8d230d29 work text data BSS heapsm655360065536

b50c752a work text data BSS heapsm655360065536

80284021 work text data BSS heapsm655360065536

97261715 work text data BSS heapsm655360065536

95241524 work text data BSS heapsm655360065536

这里可以看到进程为ASM实例的dia0进程,跟hm功能相关系,可以手动kill。

htz.pw:/home/grid>ps -ef|grep dia0

oracle655371811Oct 24– 1495:31 ora_dia0_fids1

grid3998108117Oct 24– 4817:32 asm_dia0_+ASM1

grid6488392 110102960 00:54:37pts/40:00 grep dia0

htz.pw:/home/grid>kill -9 3998108

kill后查看内存的使用情况

PhysicalPageSpace |pages/secInOut | FileSystemCache

% Used76.9%20.4%| to Paging Space0.00.0 | (numperm)7.4%

% Free23.1%79.6%| to File System0.00.4 | Process52.4%

MB Used24416.9MB1671.9MB | Page Scans0.0| System17.1%

MB Free7327.1MB6520.1MB | Page Cycles0.0| Free23.1%

Total(MB) 31744.0MB8192.0MB | Page Steals0.0|—–

Page Faults50420.1| Total100.0%

———————————————————— | numclient7.4%

Min/Maxperm3075MB( 10%)18452MB( 58%) <–% of RAM| maxclient 19.4%

Min/Maxfree9601088Total Virtual39.0GB| User56.9%

Min/Maxpgahead28Accessed Virtual22.8GB 58.4%| Pinned18.8%

4,MOS搜索

下面是MOS的一些信息收集

dia0 High Memory Usage (Doc ID 1376981.1)

Bug 13680405 – PGA consumption keeps growing in DIA0 process (Doc ID 13680405.8)

通过查询MOS文档,得到是由于BUG导致。