  | |  | OCFS and Out Of Memory issue | OCFS and Out Of Memory issue 2005-02-03 - By Frank, Ryan
Back Hello all!
I am using AS 3.0 update 3 (Kernel 2.4.21-20.Elsmp) on a pair of Compaq DL-580's with 12gb of RAM. We are running Oracle 10g release 3, in a RAC configuration. I am using SAN storage (Hitachi 9585) via Emulex LP9002 HBA's. Cluster file system is provided by OCFS 1.0.13.
The problem is like this... When I try to create a large file on an OCFS volume I get the following error from the system when the cached memory approaches 4GB.
--BEGIN PASTE-- Feb 3 11:53:54 alvmnrlnx004 kernel: Mem-info: Feb 3 11:53:54 alvmnrlnx004 kernel: Zone:DMA freepages: 2356 min: 0 low: 0 high: 0 Feb 3 11:53:54 alvmnrlnx004 kernel: Zone:Normal freepages: 617 min: 1279 low: 4544 high: 6304 Feb 3 11:53:54 alvmnrlnx004 kernel: Zone:HighMem freepages:1705708 min: 255 low: 45566 high: 68349 Feb 3 11:53:54 alvmnrlnx004 kernel: Free pages: 1708681 (1705708 HighMem) Feb 3 11:53:54 alvmnrlnx004 kernel: ( Active: 259469/674762, inactive_laundry: 101303, inactive_clean: 101243, free: 1708681 ) Feb 3 11:53:54 alvmnrlnx004 kernel: aa:0 ac:0 id:0 il:0 ic:0 fr:2356 Feb 3 11:53:54 alvmnrlnx004 kernel: aa:0 ac:340 id:2 il:74 ic:0 fr:617 Feb 3 11:53:54 alvmnrlnx004 kernel: aa:234326 ac:24803 id:674760 il:101229 ic:101243 fr:1705708 Feb 3 11:53:54 alvmnrlnx004 kernel: 2*4kB 1*8kB 2*16kB 3*32kB 1*64kB 0*128kB 0*256kB 0*512kB 1*1024kB 0*2048kB 2*4096kB = 9424kB) Feb 3 11:53:54 alvmnrlnx004 kernel: 4*4kB 1*8kB 1*16kB 0*32kB 0*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 2472kB) Feb 3 11:53:54 alvmnrlnx004 kernel: 0*4kB 0*8kB 1*16kB 1*32kB 0*64kB 1*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 1665*4096kB = 6822832kB) Feb 3 11:53:54 alvmnrlnx004 kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0 Feb 3 11:53:54 alvmnrlnx004 kernel: 161776 pages of slabcache Feb 3 11:53:54 alvmnrlnx004 kernel: 502 pages of kernel stacks Feb 3 11:53:54 alvmnrlnx004 kernel: 0 lowmem pagetables, 7439 highmem pagetables Feb 3 11:53:54 alvmnrlnx004 kernel: Free swap: 2097096kB Feb 3 11:53:54 alvmnrlnx004 kernel: 3145727 pages of RAM Feb 3 11:53:54 alvmnrlnx004 kernel: 2850807 pages of HIGHMEM Feb 3 11:53:54 alvmnrlnx004 kernel: 125269 reserved pages Feb 3 11:53:54 alvmnrlnx004 kernel: 1015244 pages shared Feb 3 11:53:54 alvmnrlnx004 kernel: 0 pages swap cached Feb 3 11:53:54 alvmnrlnx004 kernel: Out of Memory: Killed process 4104 (oracle). --END PASTE--
If I try the same process on an EXT3 volume (Also on the same storage) it works just great... I found BUG 112891 (https://bugzilla.redhat.com/bugzilla/long_list.cgi?buglist=112891) that seemed to reference the same issue, the bug is now closed as of kernel .9. We are running .20, and still seem to have the problem...
I have tried reformatting the OCFS volumes with different block sizes, adjusting vm.pagecache values (Set to vm.pagecache = 10 20 30), and removing all other kernel parameters set for Oracle in /etc/sysctl.conf. All to no avail... I have opened another case with RedHat, along with a case with Oracle.
I was wondering if anyone here might be able to shed some light on what I am doing wrong...
-- -Ryan Allvac Unix Services Monroe, NC (704) 282-1586
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN"> <HTML> <HEAD> <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"> <META NAME="Generator" CONTENT="MS Exchange Server version 6.0.6556.0"> <TITLE>OCFS and Out Of Memory issue</TITLE> </HEAD> <BODY> <!-- Converted from text/rtf format -->
<P><FONT SIZE=2 FACE="Arial">Hello all!</FONT> </P>
<P><FONT SIZE=2 FACE="Arial">I am using AS 3.0 update 3 (Kernel 2.4.21-20.Elsmp ) on a pair of Compaq DL-580's with 12gb of RAM. We are running Oracle 10g release 3, in a RAC configuration. I am using SAN storage (Hitachi 9585) via Emulex LP9002 HBA's. Cluster file system is provided by OCFS 1.0.13. </FONT></P>
<P><FONT SIZE=2 FACE="Arial">The problem is like this… When I try to create a large file on an OCFS volume I get the following error from the system when the cached memory approaches 4GB. </FONT></P>
<P><FONT SIZE=2 FACE="Arial">--BEGIN PASTE--</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: Mem -info:</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: Zone :DMA freepages: 2356 min: 0 low: 0 high: 0</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: Zone :Normal freepages: 617 min: 1279 low: 4544 high: 6304</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: Zone :HighMem freepages:1705708 min: 255 low: 45566 high: 68349</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: Free pages: 1708681 (1705708 HighMem)</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: ( Active: 259469/674762, inactive_laundry: 101303, inactive_clean: 101243, free: 1708681 )</FONT></P>
<P><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: aa:0 ac:0 id:0 il:0 ic:0 fr:2356</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: aa:0 ac:340 id:2 il:74 ic:0 fr:617</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: aa:234326 ac:24803 id:674760 il:101229 ic:101243 fr:1705708</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: 2*4kB 1 *8kB 2*16kB 3*32kB 1*64kB 0*128kB 0*256kB 0*512kB 1*1024kB 0*2048kB 2*4096kB = 9424kB)</FONT></P>
<P><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: 4*4kB 1 *8kB 1*16kB 0*32kB 0*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 2472kB)</FONT></P>
<P><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: 0*4kB 0 *8kB 1*16kB 1*32kB 0*64kB 1*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 1665*4096kB = 6822832kB)</FONT></P>
<P><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: 161776 pages of slabcache</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: 502 pages of kernel stacks</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: 0 lowmem pagetables, 7439 highmem pagetables</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: Free swap: 2097096kB</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: 3145727 pages of RAM</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: 2850807 pages of HIGHMEM</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: 125269 reserved pages</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: 1015244 pages shared</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: 0 pages swap cached</FONT>
<BR><FONT SIZE=2 FACE="Arial">Feb 3 11:53:54 alvmnrlnx004 kernel: Out of Memory: Killed process 4104 (oracle).</FONT>
<BR><FONT SIZE=2 FACE="Arial">--END PASTE--</FONT> </P>
<P><FONT SIZE=2 FACE="Arial">If I try the same process on an EXT3 volume (Also on the same storage) it works just great… I found BUG 112891 (</FONT><A HREF="https://bugzilla.redhat.com/bugzilla/long_list.cgi?buglist=112891"><U> <FONT COLOR="#0000FF" SIZE=2 FACE="Arial">https://bugzilla.redhat.com/bugzilla /long_list.cgi?buglist=112891</FONT></U></A><FONT SIZE=2 FACE="Arial">) that seemed to reference the same issue, the bug is now closed as of kernel .9.  ; We are running .20, and still seem to have the problem… </FONT></P>
<P><FONT SIZE=2 FACE="Arial">I have tried reformatting the OCFS volumes with different block sizes, adjusting vm.pagecache values (Set to vm.pagecache = 10 20 30), and removing all other kernel parameters set for Oracle in /etc/sysctl .conf. All to no avail… I have opened another case with RedHat, along with a case with Oracle. </FONT></P>
<P><FONT SIZE=2 FACE="Arial">I was wondering if anyone here might be able to shed some light on what I am doing wrong… </FONT> </P>
<P><FONT SIZE=2 FACE="Arial">--</FONT>
<BR><FONT SIZE=2 FACE="Arial">-Ryan</FONT>
<BR><FONT SIZE=2 FACE="Arial">Allvac Unix Services</FONT>
<BR><FONT SIZE=2 FACE="Arial">Monroe, NC</FONT>
<BR><FONT SIZE=2 FACE="Arial">(704) 282-1586</FONT> </P>
</BODY> </HTML> -- Taroon-list mailing list Taroon-list@(protected) http://www.redhat.com/mailman/listinfo/taroon-list
Earn $52 per hosting referral at Lunarpages.
|
|
 |