All entries by this author

Advert: Presentation about Minimal Downtime Upgrade at DOAG Conference 2010

By | November 3rd, 2010 | Category: 11g, 11gR2, Oracle Database | No Comments »

Just a short information:  I will be holding a presentation at DOAG Conference 2010 in Nürnberg about “Minimal Downtime Oracle Database 11g Upgrades“. The presentation will be on Tuesday, 16.11.2010 at 10:00. The presentation will be in german. This is a short preview in german:

Nachdem der Premier Support für das Datenbank-Release 10gR2 im Juli 2010 endet und nur das erste Jahr des Extended Supports kostenfrei ist, stehen viele Kunden vor dem Problem des Upgrades von Oracle 10g nach 11gR1 oder 11gR2 bis spätestens Juli 2011. Dieser Vortrag zeigt Möglichkeiten, den Oracle-Upgrade mittels Logical Standby sowie mittels Transportable Tablespaces mit minimaler Downtime durchzuführen. Es wird aufgezeigt, welche Voraussetzungen für die beiden Alternativen jeweils benötigt werden sowie die potentiellen Probleme bei der Durchführung. Der Vortrag enthält zudem einen praktischen Teil, bei dem die beiden Varianten demonstriert werden.

You can find the presentation materials (including online demo) on the presentations page.


Grid Infrastructure 11.2.0.2 – Multicast Patch News

By | October 28th, 2010 | Category: 11gR2, Oracle Database, Real Application Clusters | No Comments »

The addition of multicast with 11.2.0.2 caused a lot of troubles and confusion. See the following articles regarding that:

I just saw that Oracle has published a note on October 25th, which says that there will be a patch 9974223 (not yet public), which changes the multicast address from 230.0.1.0 to 224.0.0.251.

See: How to Validate Network and Name Resolution Setup for the Clusterware and RAC [ID 1054902.1]

I found it very interesting that in our current environment, the test script mcast1 fails for 230.0.1.0, but succeeds for 224.0.0.251. Other than that, the suspected host route for multicast is not even necessary at all.

So, the suggested method from Blog Article http://www.ora-solutions.net/web/2010/09/29/upgradeinstall-of-grid-infrastructure-to-11-2-0-2-is-failing/ is not necessary. It simply turned out that the Java test script is not appropriate for testing, but mcast1.c is.

Update – 10.11.2010

Patch 9974223 is officially available for Linux x86-64.


Grid Control 11.1 – Encountered Issues

By | October 26th, 2010 | Category: Oracle Enterprise Manager | 2 comments

I am currently working with Grid Control 11.1 (plus PSU October 2010) on Linux x86-64 and came across multiple issues:

  • For provisioning, you need 3 specific patches as explained in MOS Note 427577.1.
  • For one of those patches, you need to fix OMS_HOME/bin/PARTool.pm (MOS Note 1145125.1)
  • If you are using HP-UX Itanium Agents, you need to apply agent patch Patch 9756966 (MOS Note 1214754.1)
  • If you are managing 11.2 RAC databases with grid control you need Patch 10015330 (MOS Note 1183783.1)
  • Some of the target databases have database time zone set differently. This results in future timestamp values in MGMT_TARGETS_LOAD_TIMES.LAST_LOAD_TIME and this in turn results in “Status Pending” for targets under specific circumstances. We did not observe this behaviour with 10.2.0.5 agents.

I am sure there will be more to come ….


suboptimal prorated density calculation with DESC index – cardinality feedback feature

By | October 25th, 2010 | Category: 11g, 11gR2, Bugs, MetaLink, Oracle Database | No Comments »

At a customer site, I came across a problem of the Oracle cost based optimizer with selectivity calculation with a composite index with one column sorted descending. The following testcase describes the issue. Currently Oracle development is working on the issue. During testing on 11.2.0.1, I realized that the new cardinality feedback issue takes care of the problem and subsequent executions are using the correct plan.

-- drop table if exists
DROP TABLE ADMDECKER.DUMMY purge;
 
-- create table
CREATE TABLE ADMDECKER.DUMMY (
      a NUMBER NOT NULL,
      B NUMBER NOT NULL,
      C NUMBER NOT NULL,
      D NUMBER NOT NULL,
      E NUMBER ) NOLOGGING;
 
-- fill table
INSERT /*+ APPEND */  INTO ADMDECKER.DUMMY
SELECT  rownum AS a,
        MOD(rownum,100) AS B,
        MOD(rownum,10000) AS C,
        1 AS D,
        234 AS E
        FROM (
               SELECT rownum r
               FROM
                       (SELECT rownum r FROM dual CONNECT BY rownum <= 100) a,
                       (SELECT rownum r FROM dual CONNECT BY rownum <= 100) b,
                       (SELECT rownum r FROM dual CONNECT BY rownum <= 100) c
               WHERE rownum <= 1000000
       );
 
commit;
 
-- create indexes
CREATE INDEX ADMDECKER.DUMMY_IDX1 ON ADMDECKER.DUMMY(C,B) nologging;
CREATE INDEX ADMDECKER.DUMMY_IDX2 ON ADMDECKER.DUMMY(B DESC,a) nologging;
CREATE INDEX ADMDECKER.DUMMY_IDX3 ON ADMDECKER.DUMMY(C,B,D) nologging;
CREATE INDEX ADMDECKER.DUMMY_IDX4 ON ADMDECKER.DUMMY(B,a) nologging;
 
-- gather statistics (including tab stats for virtual hidden column for "B desc" column.
BEGIN DBMS_STATS.GATHER_TABLE_STATS(
      OWNNAME=>'ADMDECKER',
      TABNAME=>'DUMMY',
      METHOD_OPT=>'FOR ALL COLUMNS SIZE 1',
      cascade=>TRUE,
      ESTIMATE_PERCENT=>100);
END;
/
 
SELECT * FROM DBA_TAB_STATISTICS WHERE TABLE_NAME = 'DUMMY';
SELECT * FROM DBA_TAB_COL_STATISTICS WHERE TABLE_NAME = 'DUMMY';
 
-- note min/max/density values for system generated virtual hidden column
SELECT * FROM DBA_TAB_COLS WHERE TABLE_NAME = 'DUMMY';
SELECT * FROM dba_ind_statistics  WHERE TABLE_NAME = 'DUMMY';

I enable 10053 tracing and execute the query. Afterwards I display the execution plan.

SQL&gt; ALTER SESSION SET EVENTS='10053 trace name context forever, level 1';
 
SESSION altered.
 
SELECT /*+ GATHER_PLAN_STATISTICS RUN2 */ COUNT(E) FROM ADMDECKER.DUMMY
WHERE C = 9592 AND B >= 93 AND D=1;
 
  COUNT(E)
----------
         0
 
SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST +COST'));
 
PLAN_TABLE_OUTPUT
---------------------
SQL_ID  f5mkaprk7fzt8, child NUMBER 0
-------------------------------------
SELECT /*+ GATHER_PLAN_STATISTICS RUN2 */ COUNT(E) FROM ADMDECKER.DUMMY
WHERE C = 9592 AND B >= 93 AND D=1
 
Plan hash VALUE: 1903509519
 
------------------------------------------------------------------------------------------------
| Id |Operation                    |Name      |E-ROWS| Cost (%CPU)|A-ROWS|   A-TIME   | Buffers |
------------------------------------------------------------------------------------------------
|   0|SELECT STATEMENT             |          |      |     4 (100)|     1|00:00:00.07 |   23437 |
|   1| SORT AGGREGATE              |          |     1|            |     1|00:00:00.07 |   23437 |
|*  2|  TABLE ACCESS BY INDEX ROWID|DUMMY     |     7|     4   (0)|     0|00:00:00.07 |   23437 |
|*  3|   INDEX RANGE SCAN          |DUMMY_IDX2|     1|     3   (0)| 70000|00:00:00.03 |     197 |
------------------------------------------------------------------------------------------------
 
Predicate Information (IDENTIFIED BY operation id):
---------------------------------------------------
 
   2 - FILTER(("C"=9592 AND "D"=1))
   3 - access("DUMMY"."SYS_NC00006$"<=HEXTORAW('3EA1FF') )
       FILTER(SYS_OP_UNDESCEND("DUMMY"."SYS_NC00006$")>=93)

The complete 10053 can be viewed here: 10053 Trace of bad plan.

It can be seen that the execution plan contains the following statement and uses a very, very low density.

  Using prorated density: 0.000002 of col #6 as selectvity of out-of-range/non-existent value pred

So, instead of using Index DUMMY_IDX3, which contains all three predicate columns, especially more selective equality predicate “C = 9592”, it uses quite unselective index DUMMY_IDX2, only because density is calculated wrongly.

If the statement is executed a second time, the right plan is used. Cardinality feedback feature kicks in here. Please note the higher costs and higher cardinality for index range scan at row source operation id 3.

CONNECT ADmdecker
Enter password:
Connected.
ALTER SESSION SET EVENTS='10053 trace name context forever, level 1';
 
SESSION altered.
 
SELECT /*+ GATHER_PLAN_STATISTICS RUN2 */ COUNT(E) FROM ADMDECKER.DUMMY
WHERE C = 9592 AND B >= 93 AND D=1;
 
  COUNT(E)
----------
         0
 
SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST +COST'));
 
PLAN_TABLE_OUTPUT
----------------------
SQL_ID  f5mkaprk7fzt8, child NUMBER 1
-------------------------------------
SELECT /*+ GATHER_PLAN_STATISTICS RUN2 */ COUNT(E) FROM MDECKER.DUMMY
WHERE C = 9592 AND B >= 93 AND D=1
 
Plan hash VALUE: 1171978535
 
-------------------------------------------------------------------------------------------------
| Id |Operation                    |Name       |E-ROWS| Cost (%CPU)|A-ROWS |   A-TIME   |Buffers |
-------------------------------------------------------------------------------------------------
|   0|SELECT STATEMENT             |           |      |    11 (100)|     1 |00:00:00.01 |      3 |
|   1| SORT AGGREGATE              |           |    1 |            |     1 |00:00:00.01 |      3 |
|   2|  TABLE ACCESS BY INDEX ROWID|DUMMY      |    7 |    11   (0)|     0 |00:00:00.01 |      3 |
|*  3|   INDEX RANGE SCAN          |DUMMY_IDX3 |    7 |     3   (0)|     0 |00:00:00.01 |      3 |
-------------------------------------------------------------------------------------------------
 
Predicate Information (IDENTIFIED BY operation id):
---------------------------------------------------
 
   3 - access("C"=9592 AND "B">=93 AND "D"=1)
       FILTER("D"=1)
 
Note
-----
 - cardinality feedback used FOR this statement

The complete 10053 of the new plan (using cardinality feedback feature) can be viewed here: 10053 Trace of good plan using cardinality feedback. More information about this little documented new feature can be found at Dion Cho´s blog and from this Oracle paper.

Update: 06.10.2011

Oracle Support has provided Patch 11072246, which fixes a bug in DBMS_STATS on how min/max values are set for hidden desc columns. After installing the patch, the bug is gone and the execution plan is optimal.


Oracle Index Internals Seminar with Richard Foote

By | October 22nd, 2010 | Category: Oracle Database | 2 comments

I just got back home from an outstanding two-day Seminar of Index Guru Richard Foote. He was coming all the way from Canberra, Australia for this seminar in Munich and next week´s in Prague. I can highly recommend this seminar to any Oracle professional interested in Performance Analysis in general and indexes in particular. He has the rare gift of explaining quite complex operations very simply, sometimes even entertaining. He finished the ~ 700 slides exactly as planned at 5 p.m. the second day, on the minute. The seminar was very well worth the money. Although he starts from the basics in regards of indexes, in my opinion the seminar is quite advanced. What is very special about the seminar is that it is unique and there is no other training material or documentation around, which has such a strong focus on index internals.

I have learned quite a lot about index internals (and even some David Bowie albums) during the 2 days and understood almost everything. However, I am still puzzled about the meaning of some Ozzie phrases like “diggysquat”, “doggledigook”, “dadada” or “presto” (although this one sounds like italian…).

If you have a chance to attend to a seminar of Richard or even a presentation on one of the major conferences – don´t miss it. BTW, Richard also blogs at http://richardfoote.wordpress.com. You can see the seminars agenda at http://richardfoote.wordpress.com/oracle-index-internals-seminar/ and you can get a taste of it in http://richardfoote.files.wordpress.com/2007/12/index-internals-rebuilding-the-truth-ii.pdf

I might even check out amazon.com for one of those Bowie albums now…


Oracle 11g Certified Masters …

By | October 22nd, 2010 | Category: 11g, Certification, Oracle Database | No Comments »

I was quite surprised when I received a mail from Oracle that my OCM 11g profile is now online. It turned out that I am one of only 6 people with an 11g OCM profile! Although there might be some others who succeeded in certification and didn´t publish a profile, I am still quite surprised that there are so few. Maybe the run will begin as soon as the full exam is publicly available as currently only upgrades are possible.

http://education.oracle.com/education/otn/
http://education.oracle.com/education/otn/ocm-11g.html


opatch auto problem with Grid Infrastructure PSUs

By | September 29th, 2010 | Category: 11gR2, Bugs, MetaLink, Oracle Database | 1 Comment »

With Grid Infrastructure, Oracle customers are encouraged to seperate privileges between Grid Infrastructure and RDBMS owner. Therefore we are using grid11 as GI owner and rdbms11 as RDBMS owner. However, due to a bug in opatch (which is currently being worked on by Oracle) “opatch auto” functionality does not work as documented in the PSU readme. Workaround is to apply the patch manually. The procedure is similar to the CRS bundles we know from previous releases.

In the following, I have documented the procedure to apply the PSU patch manually.

mkdir GI_PSU
cd GI_PSU
unzip –d . ../GI_PSU_112012_p9655006_112010_Linux-x86-64.zip
 
[grid11@node1 GI_PSU]$ /opt/oracle/grid11/11.2.0.1/grid/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -oh /opt/oracle/grid11/11.2.0.1/grid -phBaseDir ./9655006
...
Prereq "checkConflictAgainstOHWithDetail" passed.
 
as root:
 
[root@node1 grid]# cd /stage/GI_PSU
[root@node1 GI_PSU]# /opt/oracle/grid11/11.2.0.1/grid/OPatch/opatch  version
Invoking OPatch 11.2.0.1.3
 
OPatch Version: 11.2.0.1.3
 
OPatch succeeded.
 
[root@node1 GI_PSU]# /opt/oracle/grid11/11.2.0.1/grid/OPatch/opatch auto . -oh /opt/oracle/grid11/11.2.0.1/grid
Executing /usr/bin/perl /opt/oracle/grid11/11.2.0.1/grid/OPatch/crs/patch112.pl -patchdir . -patchn . -oh /opt/oracle/grid11/11.2.0.1/grid -paramfile /opt/oracle/grid11/11.2.0.1/grid/crs/install/crsconfig_params
2010-09-22 10:27:24: Parsing the host name
2010-09-22 10:27:24: Checking for super user privileges
2010-09-22 10:27:24: User has super user privileges
Using configuration parameter file: /opt/oracle/grid11/11.2.0.1/grid/crs/install/crsconfig_params
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'node1'
CRS-2673: Attempting to stop 'ora.crsd' on 'node1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'node1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'node1'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'node1'
CRS-2673: Attempting to stop 'ora.CRS.dg' on 'node1'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'node1'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.node1.vip' on 'node1'
CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.scan2.vip' on 'node1'
CRS-2677: Stop of 'ora.node1.vip' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.node1.vip' on 'node2'
CRS-2677: Stop of 'ora.scan2.vip' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.scan2.vip' on 'node2'
CRS-2677: Stop of 'ora.registry.acfs' on 'node1' succeeded
CRS-2676: Start of 'ora.node1.vip' on 'node2' succeeded
CRS-2676: Start of 'ora.scan2.vip' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN2.lsnr' on 'node2'
CRS-2676: Start of 'ora.LISTENER_SCAN2.lsnr' on 'node2' succeeded
CRS-2677: Stop of 'ora.CRS.dg' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'node1'
CRS-2677: Stop of 'ora.asm' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'node1'
CRS-2673: Attempting to stop 'ora.eons' on 'node1'
CRS-2677: Stop of 'ora.ons' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'node1'
CRS-2677: Stop of 'ora.net1.network' on 'node1' succeeded
CRS-2677: Stop of 'ora.eons' on 'node1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'node1' has completed
CRS-2677: Stop of 'ora.crsd' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'node1'
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'node1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'node1'
CRS-2673: Attempting to stop 'ora.evmd' on 'node1'
CRS-2673: Attempting to stop 'ora.asm' on 'node1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'node1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'node1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'node1' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'node1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'node1' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'node1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'node1' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'node1' succeeded
CRS-2677: Stop of 'ora.asm' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'node1'
CRS-2677: Stop of 'ora.cssd' on 'node1' succeeded
CRS-2673: Attempting to stop 'ora.diskmon' on 'node1'
CRS-2673: Attempting to stop 'ora.gipcd' on 'node1'
CRS-2677: Stop of 'ora.diskmon' on 'node1' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'node1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'node1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
Successfully unlock /opt/oracle/grid11/11.2.0.1/grid
Invoking OPatch 11.2.0.1.3
 
Oracle Interim Patch Installer version 11.2.0.1.3
Copyright (c) 2010, Oracle Corporation.  All rights reserved.
 
UTIL session
 
Oracle Home       : /opt/oracle/grid11/11.2.0.1/grid
Central Inventory : /opt/oracle/oraInventory
   from           : /etc/oraInst.loc
OPatch version    : 11.2.0.1.3
OUI version       : 11.2.0.1.0
OUI location      : /opt/oracle/grid11/11.2.0.1/grid/oui
Log file location : /opt/oracle/grid11/11.2.0.1/grid/cfgtoollogs/opatch/opatch2010-09-22_10-28-11AM.log
 
Patch history file: /opt/oracle/grid11/11.2.0.1/grid/cfgtoollogs/opatch/opatch_history.txt
 
Invoking utility "napply"
Checking conflict among patches...
Checking if Oracle Home has components required by patches...
Checking conflicts against Oracle Home...
OPatch continues with these patches:   9654983
 
Do you want to proceed? [y|n]
y
User Responded with: Y
 
Running prerequisite checks...
Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:
 
You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]:  Y
 
You selected -local option, hence OPatch will patch the local system only.
 
Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/opt/oracle/grid11/11.2.0.1/grid')
 
Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files affected by the patch 'NApply' for restore. This might take a while...
Execution of 'sh /stage/GI_PSU/9654983/custom/scripts/pre -apply 9654983 ':
 
Return Code = 0
 
Applying patch 9654983...
 
ApplySession applying interim patch '9654983' to OH '/opt/oracle/grid11/11.2.0.1/grid'
ApplySession: Optional component(s) [ oracle.oraolap, 11.2.0.1.0 ] , [ oracle.precomp.common, 11.2.0.1.0 ]  not present in the Oracle Home or a higher version is found.
Backing up files affected by the patch '9654983' for rollback. This might take a while...
 
Patching component oracle.rdbms.rsf, 11.2.0.1.0...
Updating archive file "/opt/oracle/grid11/11.2.0.1/grid/lib/libpls11.a"  with "lib/libpls11.a/phd.o"
...
ApplySession adding interim patch '9654983' to inventory
 
Verifying the update...
Inventory check OK: Patch ID 9654983 is registered in Oracle Home inventory with proper meta-data.
Files check OK: Files from Patch ID 9654983 are present in Oracle Home.
Running make for target client_sharedlib
Running make for target irman
Running make for target ikfod
Running make for target irenamedg
Running make for target idgmgrl
Running make for target imkpatch
Running make for target client_sharedlib
Running make for target ioracle
Running make for target iwrap
Running make for target itnslsnr
Execution of 'sh /stage/GI_PSU/9654983/custom/scripts/post -apply 9654983 ':
 
Return Code = 0
 
The local system has been patched and can be restarted.
 
UtilSession: N-Apply done.
 
OPatch succeeded.
Invoking OPatch 11.2.0.1.3
 
Oracle Interim Patch Installer version 11.2.0.1.3
Copyright (c) 2010, Oracle Corporation.  All rights reserved.
 
UTIL session
 
Oracle Home       : /opt/oracle/grid11/11.2.0.1/grid
Central Inventory : /opt/oracle/oraInventory
   from           : /etc/oraInst.loc
OPatch version    : 11.2.0.1.3
OUI version       : 11.2.0.1.0
OUI location      : /opt/oracle/grid11/11.2.0.1/grid/oui
Log file location : /opt/oracle/grid11/11.2.0.1/grid/cfgtoollogs/opatch/opatch2010-09-22_10-32-47AM.log
 
Patch history file: /opt/oracle/grid11/11.2.0.1/grid/cfgtoollogs/opatch/opatch_history.txt
 
Invoking utility "napply"
Checking conflict among patches...
Checking if Oracle Home has components required by patches...
Checking conflicts against Oracle Home...
OPatch continues with these patches:   9655006
 
Do you want to proceed? [y|n]
y
User Responded with: Y
 
Running prerequisite checks...
Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:
 
You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]:  Y
 
You selected -local option, hence OPatch will patch the local system only.
 
Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/opt/oracle/grid11/11.2.0.1/grid')
 
Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files affected by the patch 'NApply' for restore. This might take a while...
 
Applying patch 9655006...
 
ApplySession applying interim patch '9655006' to OH '/opt/oracle/grid11/11.2.0.1/grid'
Backing up files affected by the patch '9655006' for rollback. This might take a while...
 
Patching component oracle.crs, 11.2.0.1.0...
Copying file to "/opt/oracle/grid11/11.2.0.1/grid/bin/appagent.bin"
...
ApplySession adding interim patch '9655006' to inventory
 
Verifying the update...
Inventory check OK: Patch ID 9655006 is registered in Oracle Home inventory with proper meta-data.
Files check OK: Files from Patch ID 9655006 are present in Oracle Home.
Running make for target install_srvm
Running make for target install
 
The local system has been patched and can be restarted.
 
UtilSession: N-Apply done.
 
OPatch succeeded.
CRS-4123: Oracle High Availability Services has been started.
 
[root@node2 ~]# cd /stage/GI_PSU
[root@node2 GI_PSU]# /opt/oracle/grid11/11.2.0.1/grid/OPatch/opatch auto . -oh /opt/oracle/grid11/11.2.0.1/grid
Executing /usr/bin/perl /opt/oracle/grid11/11.2.0.1/grid/OPatch/crs/patch112.pl -patchdir . -patchn . -oh /opt/oracle/grid11/11.2.0.1/grid -paramfile /opt/oracle/grid11/11.2.0.1/grid/crs/install/crsconfig_params
2010-09-22 10:39:36: Parsing the host name
2010-09-22 10:39:36: Checking for super user privileges
2010-09-22 10:39:36: User has super user privileges
Using configuration parameter file: /opt/oracle/grid11/11.2.0.1/grid/crs/install/crsconfig_params
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'node2'
CRS-2673: Attempting to stop 'ora.crsd' on 'node2'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'node2'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'node2'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'node2'
CRS-2673: Attempting to stop 'ora.CRS.dg' on 'node2'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'node2'
CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'node2' succeeded
CRS-2673: Attempting to stop 'ora.scan2.vip' on 'node2'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'node2' succeeded
CRS-2673: Attempting to stop 'ora.node2.vip' on 'node2'
CRS-2677: Stop of 'ora.scan2.vip' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.scan2.vip' on 'node1'
CRS-2677: Stop of 'ora.node2.vip' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.node2.vip' on 'node1'
CRS-2677: Stop of 'ora.registry.acfs' on 'node2' succeeded
CRS-2676: Start of 'ora.scan2.vip' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN2.lsnr' on 'node1'
CRS-2676: Start of 'ora.node2.vip' on 'node1' succeeded
CRS-2676: Start of 'ora.LISTENER_SCAN2.lsnr' on 'node1' succeeded
CRS-2677: Stop of 'ora.CRS.dg' on 'node2' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'node2'
CRS-2677: Stop of 'ora.asm' on 'node2' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'node2'
CRS-2673: Attempting to stop 'ora.eons' on 'node2'
CRS-2677: Stop of 'ora.ons' on 'node2' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'node2'
CRS-2677: Stop of 'ora.net1.network' on 'node2' succeeded
CRS-2677: Stop of 'ora.eons' on 'node2' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'node2' has completed
CRS-2677: Stop of 'ora.crsd' on 'node2' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'node2'
CRS-2673: Attempting to stop 'ora.ctssd' on 'node2'
CRS-2673: Attempting to stop 'ora.evmd' on 'node2'
CRS-2673: Attempting to stop 'ora.asm' on 'node2'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'node2'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'node2'
CRS-2677: Stop of 'ora.cssdmonitor' on 'node2' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'node2' succeeded
CRS-2677: Stop of 'ora.evmd' on 'node2' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'node2' succeeded
CRS-2677: Stop of 'ora.asm' on 'node2' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'node2' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'node2'
CRS-2677: Stop of 'ora.cssd' on 'node2' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'node2'
CRS-2673: Attempting to stop 'ora.diskmon' on 'node2'
CRS-2677: Stop of 'ora.gpnpd' on 'node2' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'node2'
CRS-2677: Stop of 'ora.gipcd' on 'node2' succeeded
CRS-2677: Stop of 'ora.diskmon' on 'node2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'node2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
Successfully unlock /opt/oracle/grid11/11.2.0.1/grid
Invoking OPatch 11.2.0.1.3
 
Oracle Interim Patch Installer version 11.2.0.1.3
Copyright (c) 2010, Oracle Corporation.  All rights reserved.
 
UTIL session
 
Oracle Home       : /opt/oracle/grid11/11.2.0.1/grid
Central Inventory : /opt/oracle/oraInventory
   from           : /etc/oraInst.loc
OPatch version    : 11.2.0.1.3
OUI version       : 11.2.0.1.0
OUI location      : /opt/oracle/grid11/11.2.0.1/grid/oui
Log file location : /opt/oracle/grid11/11.2.0.1/grid/cfgtoollogs/opatch/opatch2010-09-22_10-40-25AM.log
 
Patch history file: /opt/oracle/grid11/11.2.0.1/grid/cfgtoollogs/opatch/opatch_history.txt
 
Invoking utility "napply"
Checking conflict among patches...
Checking if Oracle Home has components required by patches...
Checking conflicts against Oracle Home...
OPatch continues with these patches:   9654983
 
Do you want to proceed? [y|n]
y
User Responded with: Y
 
Running prerequisite checks...
Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:
 
You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]:  Y
 
You selected -local option, hence OPatch will patch the local system only.
 
Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/opt/oracle/grid11/11.2.0.1/grid')
 
Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files affected by the patch 'NApply' for restore. This might take a while...
Execution of 'sh /stage/GI_PSU/9654983/custom/scripts/pre -apply 9654983 ':
 
Return Code = 0
 
Applying patch 9654983...
 
ApplySession applying interim patch '9654983' to OH '/opt/oracle/grid11/11.2.0.1/grid'
ApplySession: Optional component(s) [ oracle.oraolap, 11.2.0.1.0 ] , [ oracle.precomp.common, 11.2.0.1.0 ]  not present in the Oracle Home or a higher version is found.
Backing up files affected by the patch '9654983' for rollback. This might take a while...
 
Patching component oracle.rdbms.rsf, 11.2.0.1.0...
Updating archive file "/opt/oracle/grid11/11.2.0.1/grid/lib/libpls11.a"  with "lib/libpls11.a/phd.o"
...
ApplySession adding interim patch '9654983' to inventory
 
Verifying the update...
Inventory check OK: Patch ID 9654983 is registered in Oracle Home inventory with proper meta-data.
Files check OK: Files from Patch ID 9654983 are present in Oracle Home.
Running make for target client_sharedlib
Running make for target irman
Running make for target ikfod
Running make for target irenamedg
Running make for target idgmgrl
Running make for target imkpatch
Running make for target client_sharedlib
Running make for target ioracle
Running make for target iwrap
Running make for target itnslsnr
Execution of 'sh /stage/GI_PSU/9654983/custom/scripts/post -apply 9654983 ':
 
Return Code = 0
 
The local system has been patched and can be restarted.
 
UtilSession: N-Apply done.
 
OPatch succeeded.
Invoking OPatch 11.2.0.1.3
 
Oracle Interim Patch Installer version 11.2.0.1.3
Copyright (c) 2010, Oracle Corporation.  All rights reserved.
 
UTIL session
 
Oracle Home       : /opt/oracle/grid11/11.2.0.1/grid
Central Inventory : /opt/oracle/oraInventory
   from           : /etc/oraInst.loc
OPatch version    : 11.2.0.1.3
OUI version       : 11.2.0.1.0
OUI location      : /opt/oracle/grid11/11.2.0.1/grid/oui
Log file location : /opt/oracle/grid11/11.2.0.1/grid/cfgtoollogs/opatch/opatch2010-09-22_10-42-48AM.log
 
Patch history file: /opt/oracle/grid11/11.2.0.1/grid/cfgtoollogs/opatch/opatch_history.txt
 
Invoking utility "napply"
Checking conflict among patches...
Checking if Oracle Home has components required by patches...
Checking conflicts against Oracle Home...
OPatch continues with these patches:   9655006
 
Do you want to proceed? [y|n]
y
User Responded with: Y
 
Running prerequisite checks...
Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:
 
You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]:  Y
 
You selected -local option, hence OPatch will patch the local system only.
 
Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/opt/oracle/grid11/11.2.0.1/grid')
 
Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files affected by the patch 'NApply' for restore. This might take a while...
 
Applying patch 9655006...
 
ApplySession applying interim patch '9655006' to OH '/opt/oracle/grid11/11.2.0.1/grid'
Backing up files affected by the patch '9655006' for rollback. This might take a while...
 
Patching component oracle.crs, 11.2.0.1.0...
Copying file to "/opt/oracle/grid11/11.2.0.1/grid/bin/appagent.bin"
...
ApplySession adding interim patch '9655006' to inventory
 
Verifying the update...
Inventory check OK: Patch ID 9655006 is registered in Oracle Home inventory with proper meta-data.
Files check OK: Files from Patch ID 9655006 are present in Oracle Home.
Running make for target install_srvm
Running make for target install
 
The local system has been patched and can be restarted.
 
UtilSession: N-Apply done.
 
OPatch succeeded.
CRS-4123: Oracle High Availability Services has been started.
 
RDBMS Part:
 
/u01/app/oracle/product/11.2.0.1/db/OPatch/opatch prereq CheckComponents -ph 9655006/custom/server/9655006 -oh $ORACLE_HOME
 
[rdbms11@node1 GI_PSU]$ /u01/app/oracle/product/11.2.0.1/db/OPatch/opatch prereq CheckApplicable -ph 9655006/custom/server/9655006 -oh $ORACLE_HOME
Invoking OPatch 11.2.0.1.3
 
Oracle Interim Patch Installer version 11.2.0.1.3
Copyright (c) 2010, Oracle Corporation.  All rights reserved.
 
PREREQ session
 
Oracle Home       : /u01/app/oracle/product/11.2.0.1/db
Central Inventory : /opt/oracle/oraInventory
   from           : /etc/oraInst.loc
OPatch version    : 11.2.0.1.3
OUI version       : 11.2.0.1.0
OUI location      : /u01/app/oracle/product/11.2.0.1/db/oui
Log file location : /u01/app/oracle/product/11.2.0.1/db/cfgtoollogs/opatch/opatch2010-09-22_11-10-28AM.log
 
Patch history file: /u01/app/oracle/product/11.2.0.1/db/cfgtoollogs/opatch/opatch_history.txt
 
Invoking prereq "checkapplicable"
 
Prereq "checkApplicable" for patch 9655006 passed.
 
OPatch succeeded.
[rdbms11@node1 GI_PSU]$ /u01/app/oracle/product/11.2.0.1/db/OPatch/opatch prereq CheckConflictAgainstOH -ph 9655006/custom/server/9655006 -oh $ORACLE_HOME
Invoking OPatch 11.2.0.1.3
 
Oracle Interim Patch Installer version 11.2.0.1.3
Copyright (c) 2010, Oracle Corporation.  All rights reserved.
 
PREREQ session
 
Oracle Home       : /u01/app/oracle/product/11.2.0.1/db
Central Inventory : /opt/oracle/oraInventory
   from           : /etc/oraInst.loc
OPatch version    : 11.2.0.1.3
OUI version       : 11.2.0.1.0
OUI location      : /u01/app/oracle/product/11.2.0.1/db/oui
Log file location : /u01/app/oracle/product/11.2.0.1/db/cfgtoollogs/opatch/opatch2010-09-22_11-10-45AM.log
 
Patch history file: /u01/app/oracle/product/11.2.0.1/db/cfgtoollogs/opatch/opatch_history.txt
 
Invoking prereq "checkconflictagainstoh"
 
Prereq "checkConflictAgainstOH" passed.
 
OPatch succeeded.
 
/u01/app/oracle/product/11.2.0.1/db/OPatch/opatch prereq CheckComponents -ph 9654983 -oh $ORACLE_HOME
 
/u01/app/oracle/product/11.2.0.1/db/OPatch/opatch prereq CheckApplicable -ph 9654983 -oh $ORACLE_HOME
 
/u01/app/oracle/product/11.2.0.1/db/OPatch/opatch prereq CheckConflictAgainstOH -ph 9654983 -oh $ORACLE_HOME
 
[rdbms11@node1 GI_PSU]$ ./9655006/custom/server/9655006/custom/scripts/prepatch.sh -dbhome /u01/app/oracle/product/11.2.0.1/db
./9655006/custom/server/9655006/custom/scripts/prepatch.sh completed successfully.
 
[rdbms11@node1 GI_PSU]$ /u01/app/oracle/product/11.2.0.1/db/OPatch/opatch napply ./9655006/custom/server/9655006 -local -oh $ORACLE_HOME
 
[rdbms11@node1 GI_PSU]$ cd 9654983/
[rdbms11@node1 9654983]$ /u01/app/oracle/product/11.2.0.1/db/OPatch/opatch apply -local -oh $ORACLE_HOME
 
cd ..
[rdbms11@node1 GI_PSU]$ ./9655006/custom/server/9655006/custom/scripts/postpatch.sh -dbhome /u01/app/oracle/product/11.2.0.1/db
Reading /u01/app/oracle/product/11.2.0.1/db/install/params.ora..
Reading /u01/app/oracle/product/11.2.0.1/db/install/params.ora..
Parsing file /u01/app/oracle/product/11.2.0.1/db/bin/racgwrap
Parsing file /u01/app/oracle/product/11.2.0.1/db/bin/srvctl
Parsing file /u01/app/oracle/product/11.2.0.1/db/bin/srvconfig
Parsing file /u01/app/oracle/product/11.2.0.1/db/bin/cluvfy
Verifying file /u01/app/oracle/product/11.2.0.1/db/bin/racgwrap
Verifying file /u01/app/oracle/product/11.2.0.1/db/bin/srvctl
Verifying file /u01/app/oracle/product/11.2.0.1/db/bin/srvconfig
Verifying file /u01/app/oracle/product/11.2.0.1/db/bin/cluvfy
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/bin/racgwrap
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/bin/srvctl
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/bin/srvconfig
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/bin/cluvfy
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/bin/racgmain
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/bin/racgeut
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/bin/diskmon.bin
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/bin/lsnodes
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/bin/osdbagrp
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/bin/rawutl
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/srvm/admin/ractrans
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/srvm/admin/getcrshome
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/bin/gnsd
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/bin/crsdiag.pl
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/lib/libhasgen11.so
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/lib/libclsra11.so
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/lib/libdbcfg11.so
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/lib/libocr11.so
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/lib/libocrb11.so
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/lib/libocrutl11.so
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/lib/libuini11.so
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/lib/librdjni11.so
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/lib/libgns11.so
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/lib/libgnsjni11.so
Reapplying file permissions on /u01/app/oracle/product/11.2.0.1/db/lib/libagfw11.so
 
The same steps have to be repeated for Node2.
 
[grid11@node1 ~]$ /opt/oracle/grid11/11.2.0.1/grid/OPatch/opatch lsinventory
Invoking OPatch 11.2.0.1.3
 
Oracle Interim Patch Installer version 11.2.0.1.3
Copyright (c) 2010, Oracle Corporation.  All rights reserved.
 
Oracle Home       : /opt/oracle/grid11/11.2.0.1/grid
Central Inventory : /opt/oracle/oraInventory
   from           : /etc/oraInst.loc
OPatch version    : 11.2.0.1.3
OUI version       : 11.2.0.1.0
OUI location      : /opt/oracle/grid11/11.2.0.1/grid/oui
Log file location : /opt/oracle/grid11/11.2.0.1/grid/cfgtoollogs/opatch/opatch2010-09-22_11-20-41AM.log
 
Patch history file: /opt/oracle/grid11/11.2.0.1/grid/cfgtoollogs/opatch/opatch_history.txt
 
Lsinventory Output file location : /opt/oracle/grid11/11.2.0.1/grid/cfgtoollogs/opatch/lsinv/lsinventory2010-09-22_11-20-41AM.txt
 
--------------------------------------------------------------------------------
Installed Top-level Products (1):
 
Oracle Grid Infrastructure                                           11.2.0.1.0
There are 1 products installed in this Oracle Home.
 
Interim patches (2) :
 
Patch  9655006      : applied on Wed Sep 22 10:35:11 CEST 2010
Unique Patch ID:  12651761
   Created on 6 Jul 2010, 12:00:17 hrs PST8PDT
   Bugs fixed:
     9655006, 9778840, 9343627, 9783609, 9262748, 9262722
 
Patch  9654983      : applied on Wed Sep 22 10:32:30 CEST 2010
Unique Patch ID:  12651761
   Created on 18 Jun 2010, 00:16:02 hrs PST8PDT
   Bugs fixed:
     9068088, 9363384, 8865718, 8898852, 8801119, 9054253, 8725286, 8974548
     9093300, 8909984, 8755082, 8780372, 8664189, 8769569, 7519406, 8822531
     7705591, 8650719, 9637033, 8639114, 8723477, 8729793, 8919682, 8856478
     9001453, 8733749, 8565708, 8735201, 8684517, 8870559, 8773383, 8981059
     8812705, 9488887, 8813366, 9242411, 8822832, 8897784, 8760714, 8775569
     8671349, 8898589, 9714832, 8642202, 9011088, 9170608, 9369797, 9165206
     8834636, 8891037, 8431487, 8570322, 8685253, 8872096, 8718952, 8799099
     9032717, 9399090, 9546223, 9713537, 8588519, 8783738, 8834425, 9454385
     8856497, 8890026, 8721315, 8818175, 8674263, 9145541, 8720447, 9272086
     9467635, 9010222, 9197917, 8991997, 8661168, 8803762, 8769239, 9654983
     8706590, 8778277, 8815639, 9027691, 9454036, 9454037, 9454038, 9255542
     8761974, 9275072, 8496830, 8702892, 8818983, 8475069, 8875671, 9328668
     8798317, 8891929, 8774868, 8820324, 8544696, 8702535, 8268775, 9036013
     9363145, 8933870, 8405205, 9467727, 8822365, 9676419, 8761260, 8790767
     8795418, 8913269, 8717461, 8607693, 8861700, 8330783, 8780281, 8780711
     8784929, 9341448, 9015983, 9119194, 8828328, 8665189, 8717031, 8832205
     9676420, 8633358, 9321701, 9655013, 8796511, 9167285, 8782971, 8756598
     8703064, 9066116, 9007102, 9461782, 9352237, 8505803, 8753903, 9216806
     8918433, 9057443, 8790561, 8733225, 9067282, 8928276, 9210925, 8837736
 
Rac system comprising of multiple nodes
  Local node = node1
  Remote node = node2
 
--------------------------------------------------------------------------------
 
OPatch succeeded.
 
[grid11@node2 9655006]$ /opt/oracle/grid11/11.2.0.1/grid/OPatch/opatch lsinventory
Invoking OPatch 11.2.0.1.3
 
Oracle Interim Patch Installer version 11.2.0.1.3
Copyright (c) 2010, Oracle Corporation.  All rights reserved.
 
Oracle Home       : /opt/oracle/grid11/11.2.0.1/grid
Central Inventory : /opt/oracle/oraInventory
   from           : /etc/oraInst.loc
OPatch version    : 11.2.0.1.3
OUI version       : 11.2.0.1.0
OUI location      : /opt/oracle/grid11/11.2.0.1/grid/oui
Log file location : /opt/oracle/grid11/11.2.0.1/grid/cfgtoollogs/opatch/opatch2010-09-22_11-21-17AM.log
 
Patch history file: /opt/oracle/grid11/11.2.0.1/grid/cfgtoollogs/opatch/opatch_history.txt
 
Lsinventory Output file location : /opt/oracle/grid11/11.2.0.1/grid/cfgtoollogs/opatch/lsinv/lsinventory2010-09-22_11-21-17AM.txt
 
--------------------------------------------------------------------------------
Installed Top-level Products (1):
 
Oracle Grid Infrastructure                                           11.2.0.1.0
There are 1 products installed in this Oracle Home.
 
Interim patches (2) :
 
Patch  9655006      : applied on Wed Sep 22 10:43:57 CEST 2010
Unique Patch ID:  12651761
   Created on 6 Jul 2010, 12:00:17 hrs PST8PDT
   Bugs fixed:
     9655006, 9778840, 9343627, 9783609, 9262748, 9262722
 
Patch  9654983      : applied on Wed Sep 22 10:42:30 CEST 2010
Unique Patch ID:  12651761
   Created on 18 Jun 2010, 00:16:02 hrs PST8PDT
   Bugs fixed:
     9068088, 9363384, 8865718, 8898852, 8801119, 9054253, 8725286, 8974548
     9093300, 8909984, 8755082, 8780372, 8664189, 8769569, 7519406, 8822531
     7705591, 8650719, 9637033, 8639114, 8723477, 8729793, 8919682, 8856478
     9001453, 8733749, 8565708, 8735201, 8684517, 8870559, 8773383, 8981059
     8812705, 9488887, 8813366, 9242411, 8822832, 8897784, 8760714, 8775569
     8671349, 8898589, 9714832, 8642202, 9011088, 9170608, 9369797, 9165206
     8834636, 8891037, 8431487, 8570322, 8685253, 8872096, 8718952, 8799099
     9032717, 9399090, 9546223, 9713537, 8588519, 8783738, 8834425, 9454385
     8856497, 8890026, 8721315, 8818175, 8674263, 9145541, 8720447, 9272086
     9467635, 9010222, 9197917, 8991997, 8661168, 8803762, 8769239, 9654983
     8706590, 8778277, 8815639, 9027691, 9454036, 9454037, 9454038, 9255542
     8761974, 9275072, 8496830, 8702892, 8818983, 8475069, 8875671, 9328668
     8798317, 8891929, 8774868, 8820324, 8544696, 8702535, 8268775, 9036013
     9363145, 8933870, 8405205, 9467727, 8822365, 9676419, 8761260, 8790767
     8795418, 8913269, 8717461, 8607693, 8861700, 8330783, 8780281, 8780711
     8784929, 9341448, 9015983, 9119194, 8828328, 8665189, 8717031, 8832205
     9676420, 8633358, 9321701, 9655013, 8796511, 9167285, 8782971, 8756598
     8703064, 9066116, 9007102, 9461782, 9352237, 8505803, 8753903, 9216806
     8918433, 9057443, 8790561, 8733225, 9067282, 8928276, 9210925, 8837736
 
Rac system comprising of multiple nodes
  Local node = node2
  Remote node = node1
 
--------------------------------------------------------------------------------
 
OPatch succeeded.
 
[rdbms11@node1 GI_PSU]$  /u01/app/oracle/product/11.2.0.1/db/OPatch/opatch lsinventory
Invoking OPatch 11.2.0.1.3
 
Oracle Interim Patch Installer version 11.2.0.1.3
Copyright (c) 2010, Oracle Corporation.  All rights reserved.
 
Oracle Home       : /u01/app/oracle/product/11.2.0.1/db
Central Inventory : /opt/oracle/oraInventory
   from           : /etc/oraInst.loc
OPatch version    : 11.2.0.1.3
OUI version       : 11.2.0.1.0
OUI location      : /u01/app/oracle/product/11.2.0.1/db/oui
Log file location : /u01/app/oracle/product/11.2.0.1/db/cfgtoollogs/opatch/opatch2010-09-22_11-21-36AM.log
 
Patch history file: /u01/app/oracle/product/11.2.0.1/db/cfgtoollogs/opatch/opatch_history.txt
 
Lsinventory Output file location : /u01/app/oracle/product/11.2.0.1/db/cfgtoollogs/opatch/lsinv/lsinventory2010-09-22_11-21-36AM.txt
 
--------------------------------------------------------------------------------
Installed Top-level Products (1):
 
Oracle Database 11g                                                  11.2.0.1.0
There are 1 products installed in this Oracle Home.
 
Interim patches (2) :
 
Patch  9654983      : applied on Wed Sep 22 11:15:52 CEST 2010
Unique Patch ID:  12651761
   Created on 18 Jun 2010, 00:16:02 hrs PST8PDT
   Bugs fixed:
     9068088, 9363384, 8865718, 8898852, 8801119, 9054253, 8725286, 8974548
     9093300, 8909984, 8755082, 8780372, 8664189, 8769569, 7519406, 8822531
     7705591, 8650719, 9637033, 8639114, 8723477, 8729793, 8919682, 8856478
     9001453, 8733749, 8565708, 8735201, 8684517, 8870559, 8773383, 8981059
     8812705, 9488887, 8813366, 9242411, 8822832, 8897784, 8760714, 8775569
     8671349, 8898589, 9714832, 8642202, 9011088, 9170608, 9369797, 9165206
     8834636, 8891037, 8431487, 8570322, 8685253, 8872096, 8718952, 8799099
     9032717, 9399090, 9546223, 9713537, 8588519, 8783738, 8834425, 9454385
     8856497, 8890026, 8721315, 8818175, 8674263, 9145541, 8720447, 9272086
     9467635, 9010222, 9197917, 8991997, 8661168, 8803762, 8769239, 9654983
     8706590, 8778277, 8815639, 9027691, 9454036, 9454037, 9454038, 9255542
     8761974, 9275072, 8496830, 8702892, 8818983, 8475069, 8875671, 9328668
     8798317, 8891929, 8774868, 8820324, 8544696, 8702535, 8268775, 9036013
     9363145, 8933870, 8405205, 9467727, 8822365, 9676419, 8761260, 8790767
     8795418, 8913269, 8717461, 8607693, 8861700, 8330783, 8780281, 8780711
     8784929, 9341448, 9015983, 9119194, 8828328, 8665189, 8717031, 8832205
     9676420, 8633358, 9321701, 9655013, 8796511, 9167285, 8782971, 8756598
     8703064, 9066116, 9007102, 9461782, 9352237, 8505803, 8753903, 9216806
     8918433, 9057443, 8790561, 8733225, 9067282, 8928276, 9210925, 8837736
 
Patch  9655006      : applied on Wed Sep 22 11:14:30 CEST 2010
Unique Patch ID:  12651761
   Created on 6 Jul 2010, 12:00:17 hrs PST8PDT
   Bugs fixed:
     9655006, 9778840, 9343627, 9783609, 9262748
 
Rac system comprising of multiple nodes
  Local node = node1
  Remote node = node2
 
--------------------------------------------------------------------------------
 
OPatch succeeded.
 
[rdbms11@node2 GI_PSU]$ /u01/app/oracle/product/11.2.0.1/db/OPatch/opatch lsinventory
Invoking OPatch 11.2.0.1.3
 
Oracle Interim Patch Installer version 11.2.0.1.3
Copyright (c) 2010, Oracle Corporation.  All rights reserved.
 
Oracle Home       : /u01/app/oracle/product/11.2.0.1/db
Central Inventory : /opt/oracle/oraInventory
   from           : /etc/oraInst.loc
OPatch version    : 11.2.0.1.3
OUI version       : 11.2.0.1.0
OUI location      : /u01/app/oracle/product/11.2.0.1/db/oui
Log file location : /u01/app/oracle/product/11.2.0.1/db/cfgtoollogs/opatch/opatch2010-09-22_11-21-57AM.log
 
Patch history file: /u01/app/oracle/product/11.2.0.1/db/cfgtoollogs/opatch/opatch_history.txt
 
Lsinventory Output file location : /u01/app/oracle/product/11.2.0.1/db/cfgtoollogs/opatch/lsinv/lsinventory2010-09-22_11-21-57AM.txt
 
--------------------------------------------------------------------------------
Installed Top-level Products (1):
 
Oracle Database 11g                                                  11.2.0.1.0
There are 1 products installed in this Oracle Home.
 
Interim patches (2) :
 
Patch  9654983      : applied on Wed Sep 22 11:19:38 CEST 2010
Unique Patch ID:  12651761
   Created on 18 Jun 2010, 00:16:02 hrs PST8PDT
   Bugs fixed:
     9068088, 9363384, 8865718, 8898852, 8801119, 9054253, 8725286, 8974548
     9093300, 8909984, 8755082, 8780372, 8664189, 8769569, 7519406, 8822531
     7705591, 8650719, 9637033, 8639114, 8723477, 8729793, 8919682, 8856478
     9001453, 8733749, 8565708, 8735201, 8684517, 8870559, 8773383, 8981059
     8812705, 9488887, 8813366, 9242411, 8822832, 8897784, 8760714, 8775569
     8671349, 8898589, 9714832, 8642202, 9011088, 9170608, 9369797, 9165206
     8834636, 8891037, 8431487, 8570322, 8685253, 8872096, 8718952, 8799099
     9032717, 9399090, 9546223, 9713537, 8588519, 8783738, 8834425, 9454385
     8856497, 8890026, 8721315, 8818175, 8674263, 9145541, 8720447, 9272086
     9467635, 9010222, 9197917, 8991997, 8661168, 8803762, 8769239, 9654983
     8706590, 8778277, 8815639, 9027691, 9454036, 9454037, 9454038, 9255542
     8761974, 9275072, 8496830, 8702892, 8818983, 8475069, 8875671, 9328668
     8798317, 8891929, 8774868, 8820324, 8544696, 8702535, 8268775, 9036013
     9363145, 8933870, 8405205, 9467727, 8822365, 9676419, 8761260, 8790767
     8795418, 8913269, 8717461, 8607693, 8861700, 8330783, 8780281, 8780711
     8784929, 9341448, 9015983, 9119194, 8828328, 8665189, 8717031, 8832205
     9676420, 8633358, 9321701, 9655013, 8796511, 9167285, 8782971, 8756598
     8703064, 9066116, 9007102, 9461782, 9352237, 8505803, 8753903, 9216806
     8918433, 9057443, 8790561, 8733225, 9067282, 8928276, 9210925, 8837736
 
Patch  9655006      : applied on Wed Sep 22 11:19:03 CEST 2010
Unique Patch ID:  12651761
   Created on 6 Jul 2010, 12:00:17 hrs PST8PDT
   Bugs fixed:
     9655006, 9778840, 9343627, 9783609, 9262748
 
Rac system comprising of multiple nodes
  Local node = node2
  Remote node = node1
 
--------------------------------------------------------------------------------
 
OPatch succeeded.

Upgrade/Install of Grid Infrastructure to 11.2.0.2 is failing

By | September 29th, 2010 | Category: 11gR2, Linux, Unix | 13 comments

I have come across a quite difficult issue when trying to install/upgrade to Grid Infrastructure 11.2.0.2 on Linux. We have performed the following trials:

  • Installation of 11.1.0.6 to have Cluster Files (OCR/Votedisk) outside of ASM, then upgrade to 11.2.0.1. After that, we installed all the latest patches (PSU 11.2.0.1.2). Firstly we considered running RDBMS 11.1.0.7 in a 11.2 Grid Infrastructure cluster. However, there are quite a few issues when running this mixed setup. For most of them, patches or workarounds do exist. (see Note 948456.1) However, during failover testing, we realized that the remote database instance does not survive the failure of the public lan of the local node. This is fixed in 11.2.0.2 which motivated us to upgrade.
  • We tried to upgrade our GI 11.2.0.1.2 installation to 11.2.0.2. Everything went fine until it failed when running rootupgrade.sh on the last (second) node. Here, the second node could not join the cluster. Important to know is that 11.2.0.2 brings the new feature HAIP (High available IP), which allows to have multiple private interconnect interfaces (each with a different private interconnect network ip) to be used for failover/load balancing by Grid Infrastructure.  For this feature, Oracle uses Multicast on network “230.0.1.0”. This is stated in the updated version of the README for Oracle Database 11.2: http://download.oracle.com/docs/cd/E11882_01/readmes.112/e17129/toc.htm#CHDIEHCH
ACFS-9309: ADVM/ACFS installation correctness verified.
Failed to start Oracle Clusterware stack
Failed to start Cluster Synchorinisation Service in clustered mode at
/opt/oracle/grid11/11.2.0.2/grid/crs/install/crsconfig_lib.pm line 1016.
/opt/oracle/grid11/11.2.0.2/grid/perl/bin/perl -I/opt/oracle/grid11/11.2.0.2/grid/perl/lib -
I/opt/oracle/grid11/11.2.0.2/grid/crs/install
/opt/oracle/grid11/11.2.0.2/grid/crs/install/rootcrs.pl execution failed
  • The next try was to remove everything and directly install 11.2.0.1 with cluster files stored inside of ASM. This worked fine and we then proceeded in patching with latest PSU 11.2.0.1.2. Then, we gave it another try to upgrade to 11.2.0.2. The same problem occurred again at rootupgrade.sh on second node.
  • Then we decided to remove everything again and try a direct installation of 11.2.0.2 without any upgrade. This also failed at root.sh on last node.

With the hints of several colleagues from OTN forum (http://forums.oracle.com/forums/messageview.jspa?messageID=5393319&stqc=true) we realized that it might be related to multicast setup. We verified that multicast works with the script from MetaLink Note 413783.1. However this script does not take into account that we want multicasting via private interconnect interface instead of public LAN.

I realized that it might be necessary to add a network route to our system to direct multicast to the private interconnect network:

e.g.: /sbin/route add -net 224.0.0.0 netmask 240.0.0.0 dev bond1

After doing that, the multicast test script from MOS succeeded.

starting receiver:

[grid11@Node2 ]$ java MultiCastTestReceive

starting sender:

[grid11@Node1 ]$ java MultiCastTestSend
Sent 10 bytes to /230.0.1.0

message received by receiver:

Received data from: /10.128.128.1:13139 with length: 10

Please note that in this case 10.128.128.1 is private interconnect ip. Although it is currently just a strong suspicion, I am quite positive that this was the problem and we will consider retrying the upgrade if time allows and will report the results.

Unfortunately, Oracle support was of no help at all with this problem.


DOAG Presentation on Oracle Data Guard 11g (R1/R2) What´s new? (german)

By | July 14th, 2010 | Category: 11g, 11gR2, Data Guard, Oracle Database, Oracle Enterprise Manager | No Comments »

I have uploaded the material of the german DOAG presentation on Oracle Data Guard 11gr2 Whats New to the “presentation” section. It also contains 6 recorded demos using Enterprise Manager Grid Control.


Oracle Certified Master 11g

By | June 15th, 2010 | Category: 11g, Certification, Oracle Database | No Comments »

Today, I received a mail from Oracle to inform me that I passed Oracle Certified Master 11g Upgrade Exam. After several weeks of after-work preparations, this is a very satisfying result. If you are OCM 10g and also interested in upgrading, you can find exam schedule and list of objectives here. Good luck.

Schedule: http://education.oracle.co.uk/html/oracle/28US/SCHED_SP_OCM11.htm

Objectives: http://education.oracle.com/pls/web_prod-plq-dad/db_pages.getpage?page_id=41&p_org_id=&lang=&p_exam_id=11gOCMU


« Older articles
Newer articles »