Comments on: Your experience with RAC Dynamic Remastering (DRM) in 10gR2? http://www.ora-solutions.net/web/2009/05/12/your-experience-with-rac-dynamic-remastering-drm-in-10gr2/ Indepented Oracle consultant Tue, 07 Sep 2010 18:53:34 +0200 http://wordpress.org/?v=abc hourly 1 By: lefterhs http://www.ora-solutions.net/web/2009/05/12/your-experience-with-rac-dynamic-remastering-drm-in-10gr2/comment-page-1/#comment-109 lefterhs Thu, 10 Jun 2010 12:05:05 +0000 http://www.ora-solutions.net/web/?p=662#comment-109 We had the same issue on a 2-node 10.2.0.4 RAC (HP-UX). 2 error 481 instance crashes and GRD freezes due to DRM operations. The biggest one recorded, lasted 48 minutes. In our case, the object being remastered was not always the same and not relevant to its size. For instance, the 48 minute freeze was caused by a DRM on a 800mb table, and the database contains tables in the scale of Terabytes (12 TB total db size). Since, the local Oracle branch and Metalink had not suggested any patches, we decided to disable DRM and, since then, these issues have not reappeared. We had the same issue on a 2-node 10.2.0.4 RAC (HP-UX).
2 error 481 instance crashes and GRD freezes due to DRM operations.
The biggest one recorded, lasted 48 minutes.
In our case, the object being remastered was not always the same and not relevant to its size.
For instance, the 48 minute freeze was caused by a DRM on a 800mb table, and the database contains tables in the scale of Terabytes (12 TB total db size).
Since, the local Oracle branch and Metalink had not suggested any patches, we decided to disable DRM and, since then,
these issues have not reappeared.

]]> By: admin http://www.ora-solutions.net/web/2009/05/12/your-experience-with-rac-dynamic-remastering-drm-in-10gr2/comment-page-1/#comment-55 admin Sun, 06 Sep 2009 16:43:27 +0000 http://www.ora-solutions.net/web/?p=662#comment-55 The two interim patches provided by Oracle Support (8541032 and 8625153). They replaced the problematic patch 8516675 and fixed the DRM problems as well. Best regards, Martin The two interim patches provided by Oracle Support (8541032 and 8625153). They replaced the problematic patch 8516675 and fixed the DRM problems as well.

Best regards,
Martin

]]> By: Benjamin http://www.ora-solutions.net/web/2009/05/12/your-experience-with-rac-dynamic-remastering-drm-in-10gr2/comment-page-1/#comment-54 Benjamin Sat, 05 Sep 2009 09:22:29 +0000 http://www.ora-solutions.net/web/?p=662#comment-54 Hi Martin, I've a 2-node Solaris 10/Oracle 10.2.0.4 RAC setup and have run into the same issues with my 1 instance restarting after a shutdown abort about twice within an hour. I have also applied the patch 8516675 which does not appear to have fixed the issue. Most of the references to the error are listed against previous versions 9.2.0.x and 10.1. Would you share some feedback regarding what progress you have made with the 2 new patches you've been provided with? I am currently opening an SR for this though, but a headstart would be good. Regards, Benjamin Hi Martin,

I’ve a 2-node Solaris 10/Oracle 10.2.0.4 RAC setup and have run into the same issues with my 1 instance restarting after a shutdown abort about twice within an hour. I have also applied the patch 8516675 which does not appear to have fixed the issue. Most of the references to the error are listed against previous versions 9.2.0.x and 10.1.

Would you share some feedback regarding what progress you have made with the 2 new patches you’ve been provided with?

I am currently opening an SR for this though, but a headstart would be good.

Regards,

Benjamin

]]> By: Martin http://www.ora-solutions.net/web/2009/05/12/your-experience-with-rac-dynamic-remastering-drm-in-10gr2/comment-page-1/#comment-25 Martin Sat, 27 Jun 2009 16:21:48 +0000 http://www.ora-solutions.net/web/?p=662#comment-25 Unfortunately, patch 8516675 resulted in instance crashes so we had to deinstall it again. Now, Oracle has provided two new patches, which we are currently testing. Unfortunately, patch 8516675 resulted in instance crashes so we had to deinstall it again. Now, Oracle has provided two new patches, which we are currently testing.

]]> By: Martin http://www.ora-solutions.net/web/2009/05/12/your-experience-with-rac-dynamic-remastering-drm-in-10gr2/comment-page-1/#comment-23 Martin Sat, 13 Jun 2009 16:42:00 +0000 http://www.ora-solutions.net/web/?p=662#comment-23 Oracle Support has requested stacktraces of lms processes during the period of performance degradation. We decided to enable OSWatcher to get systemwide linux data and procwatcher to get lms process stacktraces. We created a Grid Control User Defined Metric to check whether the symptoms of a DRM performance problem is taking place. Then we triggered the lms stacktraces with a Grid Control Response Action script of the UDM. Oracle Support has also requested global hanganalyze and system state dumps but we decided not to collect system state dumps because of the big additional performance impact. The oswatcher data showed that during the drm period, the lms processes had very high CPU resource utilization. In the meantime Oracle Support has confirmed that we are hitting 6960699. We have received patch 8516675 which includes the bugfix and have installed it. Now, we are waiting to see whether this indeed fixes the issue. Oracle Support has requested stacktraces of lms processes during the period of performance degradation. We decided to enable OSWatcher to get systemwide linux data and procwatcher to get lms process stacktraces. We created a Grid Control User Defined Metric to check whether the symptoms of a DRM performance problem is taking place. Then we triggered the lms stacktraces with a Grid Control Response Action script of the UDM.

Oracle Support has also requested global hanganalyze and system state dumps but we decided not to collect system state dumps because of the big additional performance impact.

The oswatcher data showed that during the drm period, the lms processes had very high CPU resource utilization.

In the meantime Oracle Support has confirmed that we are hitting 6960699. We have received patch 8516675 which includes the bugfix and have installed it. Now, we are waiting to see whether this indeed fixes the issue.

]]>