CRS-1136 when upgrading Clusterware to 12.1.0.2


12c_1

Recently I have tried to upgrade the Clusterware from 12.1.0.1 to 12.1.0.2 on a two node Linux cluster. After starting the rootupgrade.sh I got the following error.

CRS-1136: Rejecting the rolling upgrade mode change because the cluster is being patched.
CRS-4000: Command Start failed, or completed with errors.
2014/08/07 12:48:41 CLSRSC-511: failed to set Oracle Clusterware and ASM to rolling migration mode
Died at /u01/app/12.1.0.2/grid/crs/install/crsupgrade.pm line 3435.
The command '/u01/app/12.1.0.2/grid/perl/bin/perl -I/u01/app/12.1.0.2/grid/perl/lib -I/u01/app/12.1.0.2/grid/crs/install /u01/app/12.1.0.2/grid/crs/install/rootcrs.pl  -upgrade' execution failed

The error is raised because in the past the OCR file has been restored from a older Backup. Beginning with 12c Oracle also stores the patchlevel of a cluster in the OCR File. This can be checked if you look into the OCR or trying to stop the rolling patch process of the cluster.

[root@server1 #]# crsctl stop rollingpatch 
CRS-1170: Rejecting rolling patch mode change because the patch has not been applied yet. 
The software patch level [0] on this node is not the same as the expected patch level [1650217826].

Content of the OCR file

[SYSTEM.version.hostnames.server1.patchlevel]
UB4 (10) : 0
SECURITY : {USER_PERMISSION : PROCR_ALL_ACCESS, GROUP_PERMISSION : PROCR_READ, OTHER_PERMISSION : PROCR_READ, USER_NAME : root, GROUP_NAME : root}

The solution (which has worked for me) is to update the patchlevel of the cluster. To do this execute clscfg -patch on all Nodes in the cluster and stop afterwards the rollingpatch.

For example:

[root@server2 #]# clscfg -patch 
clscfg: -patch mode specified
clscfg: EXISTING configuration version 5 detected.
clscfg: version 5 is 12c Release 1.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
[root@server2 #]# crsctl stop rollingpatch
CRS-1161: The cluster was successfully patched to patch level [1650217826].

If all went OK, you should be able to query the correct patchlevel on all nodes in the cluster.

[root@server1 #]# crsctl query crs softwarepatch
Oracle Clusterware patch level on node server1 is [1650217826].

Now you can start the rootupgrade.sh on the first node and proceed with your upgrade of your cluster.

2014/08/07 13:23:00 CLSRSC-4015: Performing install or upgrade action for Oracle Trace File Analyzer (TFA) Collector.
2014/08/07 13:23:00 CLSRSC-4012: Shutting down Oracle Trace File Analyzer (TFA) Collector.
2014/08/07 13:23:08 CLSRSC-4013: Successfully shut down Oracle Trace File Analyzer (TFA) Collector.
2014/08/07 13:23:19 CLSRSC-4003: Successfully patched Oracle Trace File Analyzer (TFA) Collector.
2014/08/07 13:23:21 CLSRSC-464: Starting retrieval of the cluster configuration data
2014/08/07 13:23:29 CLSRSC-465: Retrieval of the cluster configuration data has successfully completed.
2014/08/07 13:23:29 CLSRSC-363: User ignored prerequisites during installation
2014/08/07 13:23:36 CLSRSC-468: Setting Oracle Clusterware and ASM to rolling migration mode
2014/08/07 13:23:36 CLSRSC-482: Running command: '/u01/app/12.1.0/grid/bin/crsctl start rollingupgrade 12.1.0.2.0'
CRS-1131: The cluster was successfully set to rolling upgrade mode.
2014/08/07 13:23:41 CLSRSC-482: Running command: '/u01/app/12.1.0.2/grid/bin/asmca -silent -upgradeNodeASM -nonRolling false -oldCRSHome /u01/app/12.1.0/grid -oldCRSVersion 12.1.0.1.0 -nodeNumber 1 -firstNode true -startRolling false'
ASM-Konfiguration wurde erfolgreich in dem lokalen Knoten upgegradet.
2014/08/07 13:23:45 CLSRSC-469: Successfully set Oracle Clusterware and ASM to rolling migration mode
2014/08/07 13:23:45 CLSRSC-466: Starting shutdown of the current Oracle Grid Infrastructure stack
2014/08/07 13:25:40 CLSRSC-467: Shutdown of the current Oracle Grid Infrastructure stack has successfully completed.
OLR initialization - successful
2014/08/07 13:29:36 CLSRSC-329: Replacing Clusterware entries in file 'oracle-ohasd.conf'
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
2014/08/07 13:35:43 CLSRSC-472: Attempting to export the OCR
2014/08/07 13:35:43 CLSRSC-482: Running command: 'ocrconfig -upgrade oracle oinstall'
2014/08/07 13:35:49 CLSRSC-473: Successfully exported the OCR
2014/08/07 13:35:55 CLSRSC-486: 
 At this stage of upgrade, the OCR has changed.
 Any attempt to downgrade the cluster after this point will require a complete cluster outage to restore the OCR.
2014/08/07 13:35:55 CLSRSC-541: 
 To downgrade the cluster: 
 1. All nodes that have been upgraded must be downgraded.
2014/08/07 13:35:55 CLSRSC-542: 
 2. Before downgrading the last node, the Grid Infrastructure stack on all other cluster nodes must be down.
2014/08/07 13:35:55 CLSRSC-543: 
 3. The downgrade command must be run on the node server1 with the '-lastnode' option to restore global configuration data.
2014/08/07 13:36:26 CLSRSC-343: Successfully started Oracle Clusterware stack
clscfg: EXISTING configuration version 5 detected.
clscfg: version 5 is 12c Release 1.
Successfully taken the backup of node specific configuration in OCR. 
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
2014/08/07 13:36:36 CLSRSC-474: Initiating upgrade of resource types
2014/08/07 13:37:13 CLSRSC-482: Running command: 'upgrade model  -s 12.1.0.1.0 -d 12.1.0.2.0 -p first'
2014/08/07 13:37:13 CLSRSC-475: Upgrade of resource types successfully initiated.
2014/08/07 13:37:17 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

As always comments are welcome.

Advertisements

Ein Gedanke zu “CRS-1136 when upgrading Clusterware to 12.1.0.2

Kommentar verfassen

Trage deine Daten unten ein oder klicke ein Icon um dich einzuloggen:

WordPress.com-Logo

Du kommentierst mit Deinem WordPress.com-Konto. Abmelden / Ändern )

Twitter-Bild

Du kommentierst mit Deinem Twitter-Konto. Abmelden / Ändern )

Facebook-Foto

Du kommentierst mit Deinem Facebook-Konto. Abmelden / Ändern )

Google+ Foto

Du kommentierst mit Deinem Google+-Konto. Abmelden / Ändern )

Verbinde mit %s