11gR2 RAC: add Enterprise Manager Database Control to crs resources for automatic restart

September 18th, 2011 | Posted in 11gR2, Blog, enterprise manager, RAC | 1 Comment

Introduction:

Oracle Restart is a component added to the 11gR2 in order to improve the high availability of different database resources.

Database instances, listeners are resources that can be automatically restarted after a hardware or software failure or whenever the server host restarts.

These components can be easily added to the Oracle Restart using the srvctl add command.

The procedure of adding the Enterprise Manager database control as a resource is different. In this article, I am going to show you how to add this component to the Oracle Restart.

By adding Enterprise Manager database control to the Oracle Restart, the resource will be automatically restarted with the whole cluster resource pack.

Test Environment: The following procedure has been tested on RAC 6 nodes 11.2.0.2.2 version under Redhat Enterprise Linux.

Remember, the database version must be from 11gR2.

The procedure:

Step1: With your gird user, create 2 scripts under $GRID_HOME/crs/script (GRID_HOME or CRS_HOME), in my case: /opt/11.2.0/grid/crs/script/.

See the content of those 2 scripts below;


-bash-3.2$ cat crs_dbconsole.sh

#!/bin/bash

#

# dbcagent.sh - script to start and stop the dbconsole 11gR2 agent

#

# description: Oracle 11gR2 database console agent

ORACLE_BASE=/opt/app/oracle

ORACLE_HOME=/opt/app/oracle/product/11.2.0/db_1

LIBRARY_PATH=$ORACLE_HOME/lib

ORACLE_SID=db01

ORACLE_UNQNAME=racdb

PATH=$ORACLE_HOME/bin:$PATH

export ORACLE_BASE

export ORACLE_HOME

export LD_LIBRARY_PATH

export ORACLE_SID

export ORACLE_UNQNAME

export PATH

agent_start () {

sudo -u oracle  /opt/11.2.0/grid/crs/script/agentstartup.sh start

}

agent_stop () {

sudo -u oracle /opt/11.2.0/grid/crs/script/agentstartup.sh stop

}

agent_check () {

  sudo -u oracle /opt/11.2.0/grid/crs/script/agentstartup.sh check

}

case "$1" in

  start)

        agent_start

        ;;

  stop)

        agent_stop

        ;;

  check)

    agent_check

        ;;

  clean)

    agent_clean $(ps -C emagent -o pid=)

    ;;

  *)

        echo $"Usage: `basename $0` {start|stop|status|clean}"

        exit 1

esac

-bash-3.2$

-bash-3.2$ cat agentstartup.sh

#!/bin/bash

#

# dbcagent.sh - script to start and stop the dbconsole 11gR2 agent

#

# description: Oracle 11gR2 database console agent

ORACLE_BASE=/opt/app/oracle

ORACLE_HOME=/opt/app/oracle/product/11.2.0/db_1

LIBRARY_PATH=$ORACLE_HOME/lib

ORACLE_SID=db01

ORACLE_UNQNAME=racdb

export ORACLE_BASE

export ORACLE_HOME

export LD_LIBRARY_PATH

export ORACLE_SID

export ORACLE_UNQNAME

agent_start () {

 $ORACLE_HOME/bin/emctl start dbconsole

}

agent_stop () {

  $ORACLE_HOME/bin/emctl stop dbconsole

}

agent_check () {

  $ORACLE_HOME/bin/emctl status dbconsole

}

case "$1" in

  start)

          agent_start

        ;;

  stop)

        agent_stop

        ;;

  check)

    agent_check

        ;;

  clean)

    agent_clean $(ps -C emagent -o pid=)

    ;;

  *)

        echo $"Usage: `basename $0` {start|stop|status|clean}"

        exit 1

esac

-bash-3.2$

Step2: Give the appropriate privileged to the 2 scripts created in the step1;


[root@racdbdb01 script]# cd /opt/11.2.0/grid/crs/script

[root@racdbdb01 script]# chown grid:oinstall /opt/11.2.0/grid/crs/script/*

[root@racdbdb01 script]#  chmod 755 *

[root@racdbdb01 script]# ls -rtl

total 8

-rwxr-xr-x 1 grid oinstall 951 Sep 16 12:19 crs_dbconsole.sh

-rwxr-xr-x 1 grid oinstall 838 Sep 16 12:20 agentstartup.sh

[root@racdbdb01 script]#

Step3:

As a root user, edit the /etc/sudoers file; do it for all nodes of your cluster:

First, change the file permission from 440 to 640. Then, edit the sudoer file like described below;

Finally, do not forget to back the permissions to 440 on this sudoer file.


[root@racdbdb01 script]# chmod 640 /etc/sudoers

[root@racdbdb01 script]# vi /etc/sudoers

add at the end of the file;

grid racdbdb01 =(oracle) NOPASSWD: /opt/11.2.0/grid/crs/script/agentstartup.sh start

grid racdbdb01 =(oracle) NOPASSWD: /opt/11.2.0/grid/crs/script/agentstartup.sh stop

grid racdbdb01 =(oracle) NOPASSWD: /opt/11.2.0/grid/crs/script/agentstartup.sh check

and comment out:

#Defaults    requiretty

[root@racdbdb01 script]# chmod 440 /etc/sudoers

[root@racdbdb01 script]#

step4: As oracle user, change the permissions of the following files. Do it for all nodes in your cluster.


-bash-3.2$ su - oracle

Password:

-bash-3.2$ chmod 664 /opt/app/oracle/product/11.2.0/db_1/sysman/config/emd.properties

-bash-3.2$ chmod 664 /opt/app/oracle/product/11.2.0/db_1/racdbdb01_ddb/sysman/config/emd.properties

-bash-3.2$ chmod 755 /opt/app/oracle/product/11.2.0/db_1/bin/emctl

-bash-3.2$

-bash-3.2$

step5: Copy agentstartup.sh and crs_dbconsole.sh files to all remaining nodes of the cluster.

Do not forget to replace the SID values by the SID of each instance of your RAC:


-bash-3.2$ scp agentstartup.sh root@racdbdb06:/opt/11.2.0/grid/crs/script/agentstartup.sh

root@racdbdb06's password:

agentstartup.sh                                  100%  838     0.8KB/s   00:00

-bash-3.2$

-bash-3.2$ scp crs_dbconsole.sh root@racdbdb03:/opt/11.2.0/grid/crs/script/crs_dbconsole.sh

root@racdbdb03's password:

crs_dbconsole.sh                                 100%  951     0.9KB/s   00:00

-bash-3.2$

Step6: Add a new resource “crs_dbconsole” to the Oracle Restart.

From only node 1 and as grid user:


-bash-3.2$ crsctl add resource crs_dbconsole -type local_resource -attr " ACTION_SCRIPT=/opt/11.2.0/grid/crs/script/crs_dbconsole.sh,

DESCRIPTION=Local DB Console Ressource for DB Console Agent, DEGREE=1, ENABLED=1, AUTO_START=restore, START_TIMEOUT=120, UPTIME_THRESHOLD=1h,

CHECK_INTERVAL=60, STOP_TIMEOUT=120, SCRIPT_TIMEOUT=120, RESTART_ATTEMPTS=3, OFFLINE_CHECK_INTERVAL=60, START_DEPENDENCIES=, STOP_DEPENDENCIES=,"

Step7: test by starting and stopping the new added resource;


-bash-3.2$ crsctl start res crs_dbconsole

CRS-2672: Attempting to start 'crs_dbconsole' on 'racdbdb03'

CRS-2672: Attempting to start 'crs_dbconsole' on 'racdbdb05'

CRS-2672: Attempting to start 'crs_dbconsole' on 'racdbdb01'

CRS-2672: Attempting to start 'crs_dbconsole' on 'racdbdb06'

CRS-2672: Attempting to start 'crs_dbconsole' on 'racdbdb02'

CRS-2672: Attempting to start 'crs_dbconsole' on 'racdbdb04'

CRS-2676: Start of 'crs_dbconsole' on 'racdbdb05' succeeded

CRS-2676: Start of 'crs_dbconsole' on 'racdbdb02' succeeded

CRS-2676: Start of 'crs_dbconsole' on 'racdbdb01' succeeded

CRS-2676: Start of 'crs_dbconsole' on 'racdbdb06' succeeded

CRS-2676: Start of 'crs_dbconsole' on 'racdbdb04' succeeded

CRS-2676: Start of 'crs_dbconsole' on 'racdbdb03' succeeded

-bash-3.2$

-bash-3.2$ crsctl stop res crs_dbconsole

CRS-2673: Attempting to stop 'crs_dbconsole' on 'racdbdb04'

CRS-2673: Attempting to stop 'crs_dbconsole' on 'racdbdb05'

CRS-2673: Attempting to stop 'crs_dbconsole' on 'racdbdb01'

CRS-2673: Attempting to stop 'crs_dbconsole' on 'racdbdb03'

CRS-2673: Attempting to stop 'crs_dbconsole' on 'racdbdb06'

CRS-2673: Attempting to stop 'crs_dbconsole' on 'racdbdb02'

CRS-2677: Stop of 'crs_dbconsole' on 'racdbdb05' succeeded

CRS-2677: Stop of 'crs_dbconsole' on 'racdbdb04' succeeded

CRS-2677: Stop of 'crs_dbconsole' on 'racdbdb03' succeeded

CRS-2677: Stop of 'crs_dbconsole' on 'racdbdb02' succeeded

CRS-2677: Stop of 'crs_dbconsole' on 'racdbdb06' succeeded

CRS-2677: Stop of 'crs_dbconsole' on 'racdbdb01' succeeded

Let ‘s reboot one of the servers for more testing, Reboot for example the node 2 of the cluster:


-bash-3.2$ ssh root@racdbdb02

root@racdbdb02's password:

Last login: Fri Sep 16 12:56:07 2011 from racdbdb01.ddomain.com

[root@racdbdb02 ~]#

[root@racdbdb02 ~]# init 6

[root@racdbdb02 ~]#

Wait for some minutes and then check the resource:


-bash-3.2$ crsctl status res crs_dbconsole

NAME=crs_dbconsole

TYPE=local_resource

TARGET=ONLINE              , ONLINE              , ONLINE              , ONLINE              , ONLINE              , ONLINE

STATE=ONLINE on racdbdb01, ONLINE on racdbdb02, ONLINE on racdbdb03, ONLINE on racdbdb04, ONLINE on racdbdb05, ONLINE on racdbdb06

-bash-3.2$

-bash-3.2$ su - oracle

Password:

-bash-3.2$ emctl status dbconsole

Oracle Enterprise Manager 11g Database Control Release 11.2.0.2.0

Copyright (c) 1996, 2010 Oracle Corporation.  All rights reserved.

https://racdbdb01.ddomain.com:1158/em/console/aboutApplication

EM Daemon is running.

------------------------------------------------------------------

Logs are generated in directory /opt/app/oracle/product/11.2.0/db_1/racdbdb02_ddb/sysman/log

-bash-3.2$

Conclusion:

Enterprise Manager database control daemon will automatically restart after a hardware or software failure or whenever the server host restarts.

Scridb filter


One Comment to “11gR2 RAC: add Enterprise Manager Database Control to crs resources for automatic restart”

  1. [...] 11gR2 RAC: add Enterprise Manager Database Control to crs resources for automatic restart [...]


Leave a Comment





Subscribe


Polls

which oracle topic interests you most?

View Results

Loading ... Loading ...


Oracle Class Tweets


Recent Posts


Recent Comments

  • Ravi: Hi, As above, I want to add a new column in to my production database (11g) that has millions of records but...
  • DACCorp: Thanks bro, it worked! XD
  • accutane: Hello there, just became alert to your blog through Google, and found that it is truly informative. I am...
  • Ayman Mohamed: Thanks for your nice article, it is very helpful
  • Moon: Thanks man you solved my problem. i was facing this error: ORA-19625: error identifying file while rman...
  • James: “To solve the issue, After fixing the /etc/hosts file, origin of this issue. I have deleted HAS, using...
  • Darrell Hanning: Awesome information, and very well presented! Stopped thinking I had screwed up in my migration, and...
  • Osama mustafa: Thanks for sharing, you need to confrim with Oracle Support about modify Hidden Parameter as you know...
  • Jaspreet: Brilliant explanation.
  • Yousuf: Very Nice… Just wanted to check you have done all patching at Node 1 only.. is there any thing need to...
  • Yousuf: Very Nice.. Thanks for sharing.. Once question.. You have executed all commands on Node 1 only.. Is there any...
  • Emir: Thanks… Great article
  • borse firmate: Thank you for another informative blog. The place else may just I am getting that kind of information...
  • leandro: why this parameter is systemwide? could you read from a asm instance from one failure group and from the...
  • Mohammad: paul, we create pfile from target database to source database and later we change database name, and...
  • Muhammad Ikram: Thanks Brother for sharing pearls of knowledge. May ALLAH reward you for this both here and...
  • rgrover: Thanks for the POST. I recently encounter similar issue. Your POST helped.
  • gopalredy: really its very use full to dbas
  • Vivian: This is awesome! Thank you so much!
  • henry zhong: CDB=DB, and PDB=SCHEMA but in a sub dictionary?