It is not very known that you can run into serious problems if you run Linux x86-32bit with a big amount of RAM installed, if using RHEL below 5. The official name for the issue is called “Low Memory Starvation”. The best solution is to use x86-64bit to be able to address the whole amount of RAM efficiently. <\/p>\n
However, if that is not feasible, then make sure that you at least run the hugemem kernel if you use RHEL < 5. In RHEL5-32bit, the hugemem kernel is default.\n\nA quick demonstration about what can happen if you don\u00b4t use hugemem kernel is shown here:\n\nWe realized that RMAN backup is taking more than 24 hours. Querying v$session, we find out that \none session is in ACTION \"STARTED\", whereas the other sessions are FINISHED.\n\n\n\n
\nSQL> select program, module,action \r\n from v$session \r\n where username = 'SYS' and program like 'rman%'\r\n\/ \r\n\r\nPROGRAM MODULE ACTION \r\n-------------------------- --------------------------- -------------------\r\nrman@ora-vm1 (TNS V1-V3) backup full datafile 0000078 FINISHED129\r\nrman@ora-vm1 (TNS V1-V3) backup full datafile 0000272 STARTED16 \r\nrman@ora-vm1 (TNS V1-V3) backup full datafile 0000084 FINISHED129\r\nrman@ora-vm1 (TNS V1-V3) rman@ora-vm1 (TNS V1-V3) \r\nrman@ora-vm1 (TNS V1-V3) rman@ora-vm1 (TNS V1-V3) 0000004 FINISHED131\r\nrman@ora-vm1 (TNS V1-V3) backup full datafile 0000092 FINISHED129<\/pre>\n<\/blockquote>\nThen we check the SID,serial# from v$session of this session and also query the UNIX PID from v$process.spid<\/p>\n
\nSQL> select sid,serial# from v$session where event like 'RMAN%';\r\n\r\n SID SERIAL#\r\n---------- ----------\r\n 4343 5837<\/pre>\n<\/blockquote>\nWe activate SQL Tracing for this session to determine its activity: <\/p>\n
\nSQL> select spid from v$process where addr = \r\n (select paddr from v$session where sid = 4343);\r\n\r\nSPID\r\n------------\r\n1681\r\n\r\nSQL> begin dbms_monitor.session_trace_enable(4343,5837,true,true);\r\n 2 end;\r\n 3 \/<\/pre>\n<\/blockquote>\nHowever, no trace file gets created. Then we start tracing system calls with strace:<\/p>\n
\nora-vm1:# strace -fp 1681\r\nattach: ptrace(PTRACE_ATTACH, ...): Operation not permitted<\/pre>\n<\/blockquote>\n“Not permitted”? – Although I am connected as root?<\/p>\n
\nps -ef|grep 1681\r\noracle 1681 1582 0 Aug24 ? 00:00:09 [oracle] <defunct><\/pre>\n<\/blockquote>\nThe linux command “ps” reports the server process as “defunct”. <\/p>\n
\nora-vm1:\/usr\/oracle\/admin\/labo\/udump$ ps -ef|grep 1582\r\noracle 1582 21578 0 Aug24 ? 00:00:02 rman oracle\/product\/10.2.0\/bin\/rman nocatalog\r\noracle 21663 1582 0 Aug24 ? 00:00:01 oraclelabo (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))\r\noracle 21665 1582 0 Aug24 ? 00:00:03 oraclelabo (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))\r\noracle 1681 1582 0 Aug24 ? 00:00:09 [oracle] <defunct>\r\noracle 21691 1582 0 Aug24 ? 00:01:36 oraclelabo (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))\r\noracle 21695 1582 0 Aug24 ? 00:01:41 oraclelabo (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))\r\noracle 21793 1582 0 Aug24 ? 00:01:30 oraclelabo (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))<\/pre>\n<\/blockquote>\nNext, I checked logfile \/var\/log\/messages.1 and realized that the kernel out-of-memory killer (OOM) killed this PID because of low memory starvation.<\/p>\n
\n\/var\/log\/messages.1:\r\nAug 24 22:32:44 ora-vm1 kernel: Out of Memory: Killed process 1681 (oracle).<\/pre>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"It is not very known that you can run into serious problems if you run Linux x86-32bit with a big amount of RAM installed, if using RHEL below 5. The official name for the issue is called “Low Memory Starvation”. The best solution is to use x86-64bit to be able to address the whole amount […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[13,14,15,44,5],"tags":[],"class_list":["post-746","post","type-post","status-publish","format-standard","hentry","category-10g","category-11g","category-9ir2","category-linux-unix-2","category-oracle-database"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.ora-solutions.net\/web\/wp-json\/wp\/v2\/posts\/746","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ora-solutions.net\/web\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ora-solutions.net\/web\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ora-solutions.net\/web\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ora-solutions.net\/web\/wp-json\/wp\/v2\/comments?post=746"}],"version-history":[{"count":13,"href":"https:\/\/www.ora-solutions.net\/web\/wp-json\/wp\/v2\/posts\/746\/revisions"}],"predecessor-version":[{"id":765,"href":"https:\/\/www.ora-solutions.net\/web\/wp-json\/wp\/v2\/posts\/746\/revisions\/765"}],"wp:attachment":[{"href":"https:\/\/www.ora-solutions.net\/web\/wp-json\/wp\/v2\/media?parent=746"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ora-solutions.net\/web\/wp-json\/wp\/v2\/categories?post=746"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ora-solutions.net\/web\/wp-json\/wp\/v2\/tags?post=746"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}