Drop and recreate ASM diskgroup in Exadata(vote disk & ocr move)

By parwez on August 10, 2020
LinkedIn0

There was a requirement to clone one database from x5 exadata machine to x7 exadata. The database version was 11.2.0.3 and the clone was failing on target Exadata x7 machine , saying that the database compatibility in asm is 11.2.0.4.0 and the compatibility must be 11.2.0.3.0 .

Our current(source db) ASM compatibility was as below :

SQL> select COMPATIBILITY,DATABASE_COMPATIBILITY from  V$ASM_DISKGROUP where NAME='RECOC1' ;

 COMPATIBILITY                                     DATABASE_COMPATIBILITY                                         
 
 12.1.0.1.0                                              11.2.0.4.0

Since the target database server was new empty server and there was no data in the data diskgroup , changing the compatibility was uncomplicated .

I will go through the high level steps , how to change the asm db compatibility:

  1. MOVE all the OCR and votedisk from DATAC1 to RECOC1.
  2. Drop the existing diskgroup.
  3. Create diskgroup with new database compatibility .

Keep a backup of ASM parameter file, which will be used in later stage to open up the asm instances.

create an asm pfile from spfile:
sqlplus / as sysasm

SQL> create pfile='/home/oracle/pfileASM.ora' from spfile;
[oracle@dm11clu1adm03 ~]$ pwd
/home/oracle
[oracle@dm11clu1adm03 ~]$ cat pfileASM.ora
+ASM1._oracle_base='/u01/app/oracle'#ORACLE_BASE set from in memory value +ASM2.__oracle_base='/u01/app/oracle'#ORACLE_BASE set from in memory value +ASM3.__oracle_base='/u01/app/oracle'#ORACLE_BASE set from in memory value +ASM1.asm_diskgroups='RECOC1'#Manual Mount +ASM2.asm_diskgroups='RECOC1'#Manual Mount +ASM3.asm_diskgroups='RECOC1'#Manual Mount .asm_diskstring='o//DATAC1','o//RECOC1_*'
*.asm_power_limit=1
*.audit_sys_operations=TRUE
*.audit_syslog_level='local0.info'
+ASM1.cluster_interconnects='192.168.10.13:192.168.10.14'
+ASM2.cluster_interconnects='192.168.10.15:192.168.10.16'
+ASM3.cluster_interconnects='192.168.10.17:192.168.10.18'
*.large_pool_size=12M
*.memory_target=0
*.pga_aggregate_target=419430400
*.processes=1024
*.remote_login_passwordfile='EXCLUSIVE'
*.sga_target=2147483648create an asm pfile from spfile: sqlplus / as sysasm
  1. Move all ocr and votedisk to another disk group(+RECO)

Since , we have our existing ocr and votedisk in DATA disk group , we need to move them to RECO and tjen only we will be able to drop the DATA disk group 

Existing location ocr and votedisk

[root@exaclu1adm01 ~]# ocrcheck
 Status of Oracle Cluster Registry is as follows :
          Version                  :          4
          Total space (kbytes)     :     409568
          Used space (kbytes)      :       1800
          Available space (kbytes) :     407768
          ID                       : 1734651260
          Device/File Name         :    +RECOC1
                                     Device/File integrity check succeeded
          Device/File Name         :    +DATAC1



[root@exaclu1adm01 ~]# $ORACLE_HOME/bin/crsctl query css votedisk
 STATE    File Universal Id                File Name Disk group
 --  -----    -----------------                --------- ---------
 ONLINE   2fb29d6a40034fe7bf28effeee7fac16 (o/192.xxx.xx.9;192.xxx.xx.10/DATAC1_CD_07_exaceladm02) [DATAC1]
 ONLINE   82622cd7b4144fafbff1fd85cdc03155 (o/192.xxx.xx.7;192.xxx.xx.8/DATAC1_CD_07_exaceladm01)  [DATAC1]
 ONLINE   37cc037645a64ff2bf9fcb34f7628719 (o/192.xxx.xx.11;192.xxx.xx.12/DATAC1_CD_07_exaceladm03) [DATAC1]
 Located 3 voting disk(s).

Move them to RECO

$ORACLE_HOME/bin/ocrconfig -add +RECOC1
$ORACLE_HOME/bin/ocrconfig -delete +DATAC1


$ORACLE_HOME/bin/crsctl replace votedisk +RECOC1

 Successful addition of voting disk fbbc70e5e1624fd0bf9f3892ea2d2d6e.
 Successful addition of voting disk 48ed29726f124f4bbf216ff5a752b02e.
 Successful addition of voting disk 89d9dd61e18a4fc7bfc47d50554fa068.
 Successful deletion of voting disk c3979acea4174ff9bf6503a0726f597c.
 Successful deletion of voting disk 4c79c88becba4fafbf4e0d6f1a1e6a3b.
 Successful deletion of voting disk 15bd9f505f534f6bbfdee6d4bbd0c7e8.
 Successfully replaced voting disk group with +RECOC1.
 CRS-4266: Voting file(s) successfully replaced

Now check the new location of ocr and votedisk

[root@exaclu1adm01 ~]# ocrcheck
 Status of Oracle Cluster Registry is as follows :
          Version                  :          4
          Total space (kbytes)     :     409568
          Used space (kbytes)      :       1800
          Available space (kbytes) :     407768
          ID                       : 1734651260
          Device/File Name         :    +RECOC1
                                     Device/File integrity check succeeded

[root@exaclu1adm01 ~]# $ORACLE_HOME/bin/crsctl query css votedisk
 STATE    File Universal Id                File Name Disk group
 --  -----    -----------------                --------- ---------
 ONLINE   19189c9e09754fafbf596b773016bb82 (o/192.xxx.xx.11;192.xxx.xx.12/RECOC1_CD_02_exaceladm03) [RECOC1]
 ONLINE   cd5a100a7d6c4ff2bfb8a7652438d7a5 (o/192.xxx.xx.7;192.xxx.xx.8/RECOC1_CD_02_exaceladm01) [RECOC1]
 ONLINE   03a169951ef84f00bfa980f188c0dcd2 (o/192.xxx.xx.9;192.xxx.xx.10/RECOC1_CD_02_exaceladm02) [RECOC1]
 Located 3 voting disk(s).

Now since we have moved the ocr and votedisk , we can easlity drop and recreate the DATA disk group

Keep the information handy about the size and failgroup of the disk group

ASMCMD> lsdg
 State    Type    Rebal  Sector  Block       AU  Total_MB   Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
 MOUNTED  NORMAL  N         512   4096  4194304  11059200  10283940           307200         4988370              0             Y  DATAC1/
 MOUNTED  NORMAL  N         512   4096  4194304   1032192   1029684            28672          500506              0             N  RECOC1/


SQL> select a.NAME,b.NAME from  V$ASM_DISKGROUP a,  V$ASM_DISK b where a.GROUP_NUMBER=b.GROUP_NUMBER order by  a.name ;
 NAME                           NAME
 
 DATAC1                         DATAC1_CD_07_exaCELADM02
 DATAC1                         DATAC1_CD_11_exaCELADM02
 DATAC1                         DATAC1_CD_03_exaCELADM02
 DATAC1                         DATAC1_CD_06_exaCELADM02
 DATAC1                         DATAC1_CD_04_exaCELADM02
 DATAC1                         DATAC1_CD_00_exaCELADM02
 DATAC1                         DATAC1_CD_08_exaCELADM02
 DATAC1                         DATAC1_CD_02_exaCELADM02
 DATAC1                         DATAC1_CD_05_exaCELADM02
 DATAC1                         DATAC1_CD_01_exaCELADM02
 DATAC1                         DATAC1_CD_09_exaCELADM02

Drop the DATA diskgrup 

SQL>  alter diskgroup DATAC1 dismount force ;
 Diskgroup altered.

SQL> DROP DISKGROUP DATAC1 INCLUDING CONTENTS;
DROP DISKGROUP DATAC1 INCLUDING CONTENTS
 *
 ERROR at line 1:
 ORA-15039: diskgroup not dropped
 ORA-15001: diskgroup "DATAC1" does not exist or is not mounted
 
The asm instance was running in node 2 and 3 we have shut the other two instance and then tried to drop the DG.

SQL> DROP DISKGROUP DATAC1 force including contents;
Diskgroup dropped.

Create the diskgroup with new database compatibility 

SQL> create diskgroup DATAC1 normal redundancy
 disk 'o//DATAC1exacel*' attribute 'au_size' = '4194304'
 , 'sector_size' = '512'
 , 'compatible.asm' = '12.1.0.2.0'
 , 'compatible.rdbms' = '11.2.0.3.0'
 , 'cell.smart_scan_capable' = 'TRUE';  

 Diskgroup created.

Since our diskgroup has been created with desired diskgroup compatibility, we need to move the OCR and votedisk back to the DATA disk group , because we need to perform the same steps in RECO diskgoruo as well , where the ocr and votedisk are currently residing .

[root@exaclu1adm01 bin]# ocrcheck
 Status of Oracle Cluster Registry is as follows :
          Version                  :          4
          Total space (kbytes)     :     409568
          Used space (kbytes)      :       1800
          Available space (kbytes) :     407768
          ID                       : 1734651260
          Device/File Name         :    +RECOC1

[root@exaclu1adm01 bin]# $ORACLE_HOME/bin/ocrconfig -add +DATAC1
PROT-30: The Oracle Cluster Registry location to be added is not usable
PROC-50: The Oracle Cluster Registry location to be added is inaccessible on nodes exaclu1adm02, exaclu1adm03.

It was failing because the asm instance on node 2 and 3 was down .

So , we need to create the spfile from the pfile backup which we had taken during the inital steps and try to start the start the asm instance using the same spfile on all the nodes 

SQL> create spfile='spfile=+DATAC5/exaclu5/ASMPARAMETERFILE/registry.253.965664453' from pfile='/home/oracle/pfileASM.ora';

Create password file for asm
 orapwd file='+DATAC1/orapwasm' asm=y
 [root@exaclu1adm03 ~]# srvctl  config asm
 ASM home: 
 Password file: +DATAC1/orapwasm
 ASM listener: LISTENER

and try to start the instance 
sqlplus / as sysasm
SQL > startup

Now try to move the ocr and votedisk from RECO to DATA once again

[root@exaclu1adm01 ~]# $ORACLE_HOME/bin/ocrconfig -add +DATAC1
 [root@exaclu1adm01 ~]# ocrcheck
 Status of Oracle Cluster Registry is as follows :
          Version                  :          4
          Total space (kbytes)     :     409568
          Used space (kbytes)      :       1800
          Available space (kbytes) :     407768
          ID                       : 1734651260
          Device/File Name         :    +RECOC1
                                     Device/File integrity check succeeded
          Device/File Name         :    +DATAC1
                                     Device/File integrity check succeeded
                                 Device/File not configured                                 Device/File not configured                                 Device/File not configured      Cluster registry integrity check succeeded      Logical corruption check succeeded

[root@exaclu1adm01 ~]# $ORACLE_HOME/bin/ocrconfig -delete +RECOC1
[root@exaclu1adm01 ~]# $ORACLE_HOME/bin/crsctl query css votedisk
 STATE    File Universal Id                File Name Disk group
 --  -----    -----------------                --------- ---------
 ONLINE   19189c9e09754fafbf596b773016bb82 (o/192.xxx.xx.11;192.xxx.xx.12/RECOC1_CD_02_exaceladm03) [RECOC1]
 ONLINE   cd5a100a7d6c4ff2bfb8a7652438d7a5 (o/192.xxx.xx.7;192.xxx.xx.8/RECOC1_CD_02_exaceladm01) [RECOC1]
 ONLINE   03a169951ef84f00bfa980f188c0dcd2 (o/192.xxx.xx.9;192.xxx.xx.10/RECOC1_CD_02_exaceladm02) [RECOC1]
 Located 3 voting disk(s). 

[root@exaclu1adm01 ~]# $ORACLE_HOME/bin/crsctl replace votedisk +DATAC1
 Successful addition of voting disk 2fb29d6a40034fe7bf28effeee7fac16.
 Successful addition of voting disk 82622cd7b4144fafbff1fd85cdc03155.
 Successful addition of voting disk 37cc037645a64ff2bf9fcb34f7628719.
 Successful deletion of voting disk 19189c9e09754fafbf596b773016bb82.
 Successful deletion of voting disk cd5a100a7d6c4ff2bfb8a7652438d7a5.
 Successful deletion of voting disk 03a169951ef84f00bfa980f188c0dcd2.
 Successfully replaced voting disk group with +DATAC1.
 CRS-4266: Voting file(s) successfully replaced


 [root@exaclu1adm01 ~]# $ORACLE_HOME/bin/crsctl query css votedisk
 STATE    File Universal Id                File Name Disk group
 --  -----    -----------------                --------- ---------
 ONLINE   2fb29d6a40034fe7bf28effeee7fac16 (o/192.xxx.xx.9;192.xxx.xx.10/DATAC1_CD_07_exaceladm02) [DATAC1]
 ONLINE   82622cd7b4144fafbff1fd85cdc03155 (o/192.xxx.xx.7;192.xxx.xx.8/DATAC1_CD_07_exaceladm01) [DATAC1]
 ONLINE   37cc037645a64ff2bf9fcb34f7628719 (o/192.xxx.xx.11;192.xxx.xx.12/DATAC1_CD_07_exaceladm03) [DATAC1]
 Located 3 voting disk(s).

Now, check the compatibility

SQL> select COMPATIBILITY,DATABASE_COMPATIBILITY from  V$ASM_DISKGROUP where NAME='RECOC1' ;
  COMPATIBILITY                                 DATABASE_COMPATIBILITY                                 
 
   12.1.0.2.0                                            11.2.0.3.0

Perform the same steps for RECO diskgroup also and drop and recreate RECO with desired compatibility .

Once done , you will be able to clone the database with expected compatibility .

 859 total views,  1 views today

LinkedIn0

Leave a Reply

Your email address will not be published. Required fields are marked *