I have uploaded- a step-by-step Installation Guide for Oracle Enterprise Manager Grid Control 11g on RHEL 5.5 x86-32 to the papers section:
I would like to draw your attention to a particularly dangerous security vulnerability, which was recently published by David Litchfield.
How dangerous is the vulnerability?
Any database user, who has “create session” privilege, which means, who can log into the database, can use the security hole to execute any OS command in the ownership of the oracle database owner. This means, that both denial of service as well as access to all data is exposed.
Which versions are affected?
Affected are database versions 10.2.0.4 (incl. 10.2.0.4.3 containing latest security patches as of January 2010) as well as 11g (incl. 11.2.0.1).
What can I do to close this security vulernability?
You can revoke privileges from PUBLIC:
revoke execute on dbms_java from PUBLIC;
revoke execute on dbms_java_test from PUBLIC;
revoke execute on “oracle/aurora/util/Wrapper” from PUBLIC;
grant execute on sys.dbms_jvm_exp_perms to IMP_FULL_DATABASE;
grant execute on sys.dbms_jvm_exp_perms to EXP_FULL_DATABASE;
revoke execute on sys.dbms_jvm_exp_perms from PUBLIC;
If you are using a third party vendor application, you should contact your vendor to check compatibility with revoked privileges or test before implementing in production.
I just came back from the DOAG Conference 2009, the german Oracle user group conference in Nürnberg where I had a speech about RAC PreProduction Testing. I have uploaded the slides and the paper to the papers section. At this time, the presentation is available in german only.
To find out whether the database was initially created as a 32-bit or 64-bit database, you can query:
SQL> select metadata from sys.kopm$ ;
When the output contains: ‘B023‘ , the database was created as 32-bit whereas ‘B047‘ indicates 64-bit.
BTW: A short introduction to converting a database from Linux x86-32 to Linux x86-64 is available in MetaLink Note 341880.1.
During a 11gR1 Clusterware installation for a Single Instance Failover Cluster at a customer site, I have experienced an interesting behaviour, which was caused by an Oracle Bug.
The environment was:
The ASM instances could be started with SQL*Plus without any problems, but if the ASM instances were started by means of clusterware using srvctl (either from root, crs or oracle) the ASM instances would crash at diskgroup mount with:
ORA-07445: exception encountered: core dump [sskgds_find_rtn_hdr()+1171] [SIGBUS] [ADDR:0x2AACD701342C] [PC:0x25799DF] [Non-existent physical address] []
Oracle Support identified this behaviour as Bug 6952915, for which there are patches for Linux x86, x86_64 and Solaris Sparc64.
Oracle has released the October 2009 Patch Set Update (PSU) which contains several interesting news:
More info can be found in Metalink Note 854428.1.
Yesterday, on September 1st, Oracle released the much awaited database version 11gR2 for Linux x86 for both 32bit and 64bit platforms. The software can be downloaded via OTN: http://www.oracle.com/technology/software/products/database/index.html.
It is not very known that you can run into serious problems if you run Linux x86-32bit with a big amount of RAM installed, if using RHEL below 5. The official name for the issue is called “Low Memory Starvation”. The best solution is to use x86-64bit to be able to address the whole amount of RAM efficiently.
However, if that is not feasible, then make sure that you at least run the hugemem kernel if you use RHEL < 5. In RHEL5-32bit, the hugemem kernel is default. A quick demonstration about what can happen if you don´t use hugemem kernel is shown here: We realized that RMAN backup is taking more than 24 hours. Querying v$session, we find out that one session is in ACTION "STARTED", whereas the other sessions are FINISHED.
SQL> select program, module,action
from v$session
where username = 'SYS' and program like 'rman%'
/
PROGRAM MODULE ACTION
-------------------------- --------------------------- -------------------
rman@ora-vm1 (TNS V1-V3) backup full datafile 0000078 FINISHED129
rman@ora-vm1 (TNS V1-V3) backup full datafile 0000272 STARTED16
rman@ora-vm1 (TNS V1-V3) backup full datafile 0000084 FINISHED129
rman@ora-vm1 (TNS V1-V3) rman@ora-vm1 (TNS V1-V3)
rman@ora-vm1 (TNS V1-V3) rman@ora-vm1 (TNS V1-V3) 0000004 FINISHED131
rman@ora-vm1 (TNS V1-V3) backup full datafile 0000092 FINISHED129
Then we check the SID,serial# from v$session of this session and also query the UNIX PID from v$process.spid
SQL> select sid,serial# from v$session where event like 'RMAN%';
SID SERIAL#
---------- ----------
4343 5837
We activate SQL Tracing for this session to determine its activity:
SQL> select spid from v$process where addr = (select paddr from v$session where sid = 4343); SPID ------------ 1681 SQL> begin dbms_monitor.session_trace_enable(4343,5837,true,true); 2 end; 3 /
However, no trace file gets created. Then we start tracing system calls with strace:
ora-vm1:# strace -fp 1681 attach: ptrace(PTRACE_ATTACH, ...): Operation not permitted
“Not permitted”? – Although I am connected as root?
ps -ef|grep 1681 oracle 1681 1582 0 Aug24 ? 00:00:09 [oracle] <defunct>
The linux command “ps” reports the server process as “defunct”.
ora-vm1:/usr/oracle/admin/labo/udump$ ps -ef|grep 1582 oracle 1582 21578 0 Aug24 ? 00:00:02 rman oracle/product/10.2.0/bin/rman nocatalog oracle 21663 1582 0 Aug24 ? 00:00:01 oraclelabo (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) oracle 21665 1582 0 Aug24 ? 00:00:03 oraclelabo (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) oracle 1681 1582 0 Aug24 ? 00:00:09 [oracle] <defunct> oracle 21691 1582 0 Aug24 ? 00:01:36 oraclelabo (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) oracle 21695 1582 0 Aug24 ? 00:01:41 oraclelabo (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) oracle 21793 1582 0 Aug24 ? 00:01:30 oraclelabo (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
Next, I checked logfile /var/log/messages.1 and realized that the kernel out-of-memory killer (OOM) killed this PID because of low memory starvation.
/var/log/messages.1: Aug 24 22:32:44 ora-vm1 kernel: Out of Memory: Killed process 1681 (oracle).
In MetaLink Note 853691.1: “ALERT: Important Upgrade Steps Required for Enterprise Manager Grid Control 10gR5 (10.2.0.5) Upgrades”, Oracle Support recommends to install some patches after the upgrade of OEM to 10.2.0.5.
Especially, patch bundle #8708893 solves around 20 bugs which could interfere with normal operations of Grid Control.
I have recently encountered a problem at a customer site, where the database instances resource utilization was so high that the application did not work anymore. Version was 10.2.0.4 on Linux with Oracle Recommended Generic Patches installed.
Unfortunately, the customer decided to bounce the instance, so that there is no possibility for intensive diagnosis. However, ASH report shows that a dozen sessions either waited on Wait event “cursor: pin S” or were active (ON CPU) without any SQL_ID. An AWR Report showed:
Top 5 Timed Events Avg %Total ~~~~~~~~~~~~~~~~~~ wait Call Event Waits Time (s) (ms) Time Wait Class ------------------------------ ------------ ----------- ------ ------ ---------- cursor: pin S 140,036,615 24,833 0 ###### Other CPU time 19 72.9 log file sync 8,767 8 1 30.2 Commit log file parallel write 9,039 8 1 29.9 System I/O control file parallel write 1,269 5 4 20.2 System I/O
Oracle Support confirmed that this is Bug 6904068 High CPU usage when there are “cursor: pin S” waits. We have filed a backport request for it, as there is no patch for our platform available yet.
