Archive for December 2008

LMON: terminating instance due to error 481 after 10.2.0.4 Upgrade

I have recently come across this nasty bug, which occurred after upgrading the RAC cluster to 10.2.0.4. Out of nowhere the LMON process has terminated one cluster instance. After some research I found a matching bug from September 2008:

Bug 6500033 / MetaLink Note 6500033.8: LMON crash the instance with ORA-481 due to DRM sync timeout

alert_MDDB1.log:

Sun Oct 19 17:47:12 2008
LMON: terminating instance due to error 481

MDDB1_lmon_12345.trc:

-----------------------
sent synca inc 4 lvl 1308 (4,0/38/0)
kjfcdrmrfg: SYNC TIMEOUT (105441, 104540, 900), step 31
Submitting asynchronized dump request [28]
KJC Communication Dump:
state 0x5 flags 0x0 mode 0x0 inst 0 inc 4
nrcv 3 nsp 3 nrcvbuf 1000

It is fixed in RAC Recommended Bundle #2 (7573282), available for Linux x86-32/64, Solaris and HP-UX.



Add Dependency of Database Instance on ASM to OCR

Today at a presentation of Grégory Guillou and Alex Gorbachev from The Pythian Group at UKOUG 2008, I learned that the dependency of RAC database instances on ASM is not set up when adding the database instances to OCR via srvctl add instance.

I was wondering in the past, why the database instances would sometimes not start up correctly when booting and fail at loading the spfile from ASM.

So, if you did not use DBCA to create and register your RAC database with OCR but srvctl, as shown here:

$ srvctl add database -d BOSTON –o /opt/oracle/product/10g_db_rac
$ srvctl add instance -d BOSTON -i BOSTON1 -n boston_host1
$ srvctl add instance -d BOSTON -i BOSTON2 -n boston_host2

then you have to create this dependency manually in order to determine the appropriate startup order:

$ srvctl modify instance –d BOSTON –i BOSTON1 –s +ASM1
$ srvctl modify instance –d BOSTON –i BOSTON2 –s +ASM2