Oracle's AutoUpgrade Utility

The AutoUpgrade utility helps to upgrade one or many Oracle databases with a single command and single configuration file.

In this blog, explaining how to upgrade Oracle databases from 12.2 to 19.3 release using "AutoUpgrade" utility.

Benefits of AutoUpgrade:

• The AutoUpgrade helps to upgrade one or many databases without human intervention, all with one command and a single configuration file.
• The AutoUpgrade utility is designed to automate the upgrade process - It identifies issues before upgrade. It performs pre and post upgrade actions. It performs database upgrades and starts the upgraded Oracle databases.
• It enables to schedule upgrades for future points in time
• It enables to set, change or remove initialization parameters as desired
• It enables you to call the custom scripts during the upgrade to configure databases.

Target Versions Supported:

The AutoUpgrade utility can be used to upgrade Oracle databases to below listed target versions:

• Oracle Database 21c (21.3 and newer)
• Oracle Database 19c (19.3 and newer)
• Oracle Database 18c (18.5 and newer)
• Oracle Database 12c Release 2 (12.2 + DBJAN2019RU and newer)

AutoUpgrade Modes: 

The steps that AutoUpgrade runs to upgrade the database depends on the processing mode that you select.

• Analyze Mode: Setup, Prechecks.
• Fixups Mode: Setup, Prechecks, and Prefixups.
• Deploy Mode: Setup, guaranteed restore point (GRP), Preupgrade, Prechecks, Prefixups, Drain, DB (database) Upgrade, Postchecks, and Postupgrade. You can run your own scripts before the upgrade (Preupgrade stage) or after the upgrade (Postupgrade stage), or both before and after the upgrade.
• Upgrade Mode: Setup, DB (database) Upgrade, Postchecks, and Postfixups.

Autoupgrade - Steps

        1. Download and install the Oracle 19.3 software binaries.

I have already installed Oracle 19.3 software. 

2. Verify Java version. 

The AutoUpgrade Utility requires Java 8. 

[oracle@localhost ~]$ $ORACLE_HOME/jdk/bin/java -version

java version "1.8.0_201"

Java(TM) SE Runtime Environment (build 1.8.0_201-b09)

Note - Oracle_home is the Target Oracle Database home. 

          3. Download the latest AutoUpgrade version

• The autoupgrade.jar file exists by default, in Oracle Database 19c (19.3) and later targets Oracle homes. Oracle strongly recommends downloading the latest AutoUpgrade version from My Oracle Support Document 2485457.1.

• The autoupgrade.jar file exists in the $ORACLE_HOME/rdbms/admin directory. 

• For both Oracle Database 12c Release 2 (12.2) and Oracle Database 18c (18.5) target homes, you must download the AutoUpgrade kit from My Oracle Support Document 2485457.1.

• Run below command to verify AutoUpgrade version:

[oracle@localhost admin]$ cd $ORACLE_HOME/rdbms/admin

[oracle@localhost admin]$ $ORACLE_HOME/jdk/bin/java -jar autoupgrade.jar -version

Note - Oracle_home is the Target Oracle Database home. 

• Download the latest autoupgrade.jar file from My Oracle Support Document 2485457.1. Replace “autoupgrade.jar” which is already present in $ORACLE_HOME/rdbms/admin directory, with latest downloaded file.

• Now run below command to verify AutoUpgrade version. 

It should be latest version as below:

[oracle@localhost admin]$ $ORACLE_HOME/jdk/bin/java -jar autoupgrade.jar -version

        4. Create the Configuration file as below. AutoUpgrade configuration files contain all the information required to perform Oracle Database upgrades.

        5. Run Autoupgrade in Analyze mode to start analysis.

$ORACLE_HOME/jdk/bin/java -jar $ORACLE_HOME/rdbms/admin/autoupgrade.jar -config /u01/app/oracle/admin/autoupgrade/config.txt -mode analyze

        6. The main commands to monitor the Autoupgrade jobs, from Autoupgrade console are as below:

lsj – Lists the jobs by status

status – Lists the status of a particular job or all jobs

tasks – Displays the tasks that are running.

        7. Review the analysis Summary report.

            8. Once Autopgrade in Analyze mode completed successfully, run Autoupgrade in Deploy mode to start the database upgrade process as below:

$ORACLE_HOME/jdk/bin/java -jar $ORACLE_HOME/rdbms/admin/autoupgrade.jar -config /u01/app/oracle/admin/autoupgrade/config.txt -mode deploy

            9. Review the AutoUpgrade log files for any errors

    10. Databases upgraded from 12.2 to 19.3 successfully, using the Autoupgrade tool.
            

         Connect to databases and verify.  

         The Autoupgrade tool updates the new target Oracle home in “/etc/oratab” for all upgraded databases.

                    Database 1:

                           Database 2:

Migrating database to AWS using AWS DMS

There are many tools to migrate databases to AWS, depending on the type of database migration. For example, for migrating Oracle databases to AWS RDS Oracle, the native tools like data pump export/import or Golden gate also works more effectively.

In this blog, explaining how to migrate data to AWS with the help of AWS DMS service.

Architecture Diagram

AWS DMS, also known as AWS Database Migration Service is AWS cloud service that helps to migrate data, from on-premises databases to Amazon RDS or databases running on Amazon EC2 to RDS, as well as from one Amazon RDS database to another Amazon RDS database.

AWS DMS migrates data, tables and primary keys to the target database. All other database elements are not migrated.

AWS Schema Conversion Tool (SCT) is another AWS cloud service, it helps to migrate schema and code from source database to target database. In case of heterogeneous database migrations, AWS Schema Conversion Tool (SCT) helps to convert schema and code from source database to target database.

Please click on the below link, for my article posted in LinkedIn, explaining the Key benefits of AWS DMS, Limitations of AWS DMS and how to migrate databases to AWS using AWS DMS. 

https://www.linkedin.com/posts/vadirajhonwad_datamigration-databasemigration-awsmigration-activity-6854527909235855360-pYos

Oracle 19c Grid Infrastructure(GI) Rolling upgrade

I tested the Oracle Grid Infrastructure (GI) rolling upgrade from 12.2 to 19.8, a few days back.

It's two node RAC cluster and the OS version is RHEL 7.6.

Followed the below steps to perform Rolling upgrade of Oracle Grid Infrastructure(GI) 

from 12.2 to 19.8:

1. Ensure that the latest PSU is applied to 12.2 Grid Infrastructure (GI) home.

2. Review/Configure Operating system for 19c upgrade:

Ensure that server is having the supported operating system version, kernel release and all the required operating system packages, for 19c upgrade.

The below Oracle document can be referred for more details.

https://docs.oracle.com/en/database/oracle/oracle-database/19/cwlin/index.html

3. You need root user access to run “rootupgrade.sh” script during upgrade

4. Create directory structure for Oracle 19c GI home:

mkdir -p /u01/app/19.3.0/grid

chown -R grid:oinstall /u01/app/19.3.0

chmod -R 775 /u01/app/19.3.0

5. Download the Oracle 19c (19.3) Grid Infrastructure software, for Linux x86-64 to a directory (Example – “/oracle_software/19.3.0”)

6. Extract downloaded 19c GI software in above step to 19c GI home directory created in above step # 4

cd /oracle_software/19.3.0/

unzip LINUX.X64_193000_grid_home.zip -d /u01/app/19.3.0/grid

7. Download and extract the latest Oracle 19c GI RU patch to a working directory (“/oracle_software/19.3.0/31305339”).

I used the current available latest RU patch

"Patch 31305339: GI Release Update 19.8.0.0.200714" patch.

8. Use Cluster Verification Utility (CVU) to perform pre-checks for upgrade

cd /u01/app/19.3.0/grid

./runcluvfy.sh stage -pre crsinst -upgrade -rolling -src_crshome /u01/app/12.2.0.1/grid -dest_crshome /u01/app/19.3.0/grid -dest_version 19.0.0.0.0 -fixup -verbose

Review the Cluster Verification Utility (CVU) output and fix the errors/failures.

9. Start the 19.8 Grid Infrastructure rolling upgrade as below:

The patch # 31305339 "GI Release Update 19.8" will be applied to the 19c GI home during upgrade.

Note - Ensure to set the environment, so that you can open the Graphical User Interface (GUI) for Oracle GI 19c installer.

$ unset ORACLE_BASE

$ unset ORACLE_HOME

$ unset ORACLE_SID

cd /u01/app/19.3.0/grid

./gridSetup.sh -applyPSU /oracle_software/19.3.0/31305339

Oracle Grid Infrastructure 19c Installer will start. There are 9 steps in the Installer.

Step 1 - Select Configuration option:

Choose “Upgrade Oracle Grid Infrastructure” and then click “Next”

Step 2 - Select all Cluster nodes and then click “Next”.

Step 3 - Specify Management options:

Update the details to perform registration with Enterprise Manager or uncheck and click “Next”.

Step 4 - Specify installation Location:

Review/Update Oracle Base and GI home path and then click “Next”.

Step 5 - Root script execution configuration:

            Uncheck “Automatically run configuration scripts” and then click “Next”.

Step 6 - Perform prerequisites check:

Review and fix if there are any errors. Click “Next” once there are no errors in the prerequisites check page.

Step 7 - Summary:

Review Summary and click “Submit”.

Step 8 - Install Product:

Installer will proceed with the upgrade.

Run “rootupgrade.sh” script, as root user, when installer prompts.

Run on local node first, after successful completion you can run on all other nodes in parallel.

Once the “rootupgrade.sh” script execution is completed on all nodes, come back to the Installer window and then click “OK” on the Installer UI window.

Step 9 - Finish

The rolling upgrade of Oracle Grid Infrastructure 19.8 completed successfully.

Completed all 9 steps of upgrade in GI 19c installer.

Click “Close”.

10. Verify Grid infrastructure version using below commands, post upgrade:

$ <GI Home>/bin/crsctl query crs softwareversion -all

Oracle Clusterware version on node [Node1] is [19.0.0.0.0]

Oracle Clusterware version on node [Node2] is [19.0.0.0.0]

$ <GI Home>/bin/crsctl query crs activeversion

Oracle Clusterware active version on the cluster is [19.0.0.0.0]

11. Verify all GI services are online using below commands, post upgrade:

<GI Home>/bin/crsctl check crs

<GI Home>/bin/crsctl stat res -t

How to access the Geo-Blocked websites using AWS

Hello Friends, hope you are doing well.

I just wanted to share my findings/observations, on how to access geographically blocked websites using AWS.

Hope this may be helpful, when you are trying to access your personal information back home, while travelling abroad. Also, if you want to watch your favorite programs while travelling, for example Indian application “Voot”, which works only in India.

VPNs are also good option for accessing the content that is blocked in your own country. There are lots of VPNs out there to choose from, for watching streaming media and services from another country.

AWS is another option to access the Geo-blocked websites.

The below listed high level steps can be followed to access Geo-blocked websites using AWS.

·       Login to AWS console

·       Change Region in AWS to specific region, from where you are trying access website.

Example: “Asia Pacific (Mumbai)” to access Indian application “Voot”

·       Open EC2 service page and create an EC2 instance with Windows OS

·       Connect to this EC2 instance using “Remote Desktop Connection” from your laptop/PC

·    Open IE (Internet Explorer) in EC2 instance and then you can browse the website that you couldn’t access from your laptop/PC

Hope this may be helpful, if you are trying to access Geo-blocked websites.

"ORA-15040: diskgroup is incomplete” while Mounting or Dropping disk group

I faced the below error while trying to mount the disk group. Got the same error while trying to drop this disk group.

ORA-15032: not all alterations performed
ORA-15017: diskgroup "DATA" cannot be mounted
ORA-15040: diskgroup is incomplete

Just to give the background, this is two node RAC cluster and the servers had been reimaged to upgrade the OS. I have built Oracle cluster after server re-imaging.

Then I started ASMCA to create the new ASM disk groups and noticed that the disk group “DATA” was already present in “Dismounted” state as shown in below screenshot.

Now I was suspecting that this issue could be due to some residue remaining in some disks after server re-imaging. It could be possible, if somehow few disks have been missed to format after re-imaging the servers.

Executed below query to find the disks which was causing the issue.

SQL> select GROUP_NUMBER, name, disk_number, MOUNT_STATUS, HEADER_STATUS, MODE_STATUS, state, os_mb, total_mb, path from v$asm_disk where HEADER_STATUS not in ('CANDIDATE', 'PROVISIONED', 'FORMER') order by GROUP_NUMBER,name;

The output of above query showed three disks with:

“HEADER_STATUS” as “MEMBER”,

“GROUP_NUMBER” as “0”

It means that there are three disks associated with the disk group “DATA” which is in dismounted status as shown in screenshot.

Formatted these three disks using “dd” command. Re-opened ASMCA and the disk group “DATA” got disappeared. Then created the disk group “DATA”.

Hope this blog will be helpful if you come across this scenario where servers have been re-imaged for cluster build and missed to format some or all disks in the cluster after server re-imaging.

Rolling upgrade of OS in Oracle Cluster

Hi Friends, hope you are doing well.

Recently I did rolling upgrade of OS from RHEL 6.4 to RHEL 7.4 in Oracle RAC 11gR2 (Two Nodes) environment. I wanted to share the approach that I followed considering to minimize the downtime as possible.

Please note that for OS upgrade from RHEL 6.4 to 7.4, servers has to be reimaged.

Oracle supports the rolling upgrade of the operating system in a cluster when both versions of the operating system are certified with the version of Oracle Database you are running.

Mixed operating system versions are only supported for the duration of an upgrade (i.e. within 24 hours). The cluster should never be operated with mixed operating systems for an extended period.

Followed the below steps for this OS upgrade. This is the two node Oracle RAC (11gR2) environment.

1.       Upgrade OS to RHL7.4 in Node2:

a.       Relocate all DB services from Node2 to Node1.

Example:

# srvctl relocate service -d DB_Name -s Service_Name -i DB_Instance_2 -t DB_Instance_1

b.      Remove the Node2 from cluster. Before removing the node2, please take the backup from node2 if anything needed.

c.       Reimage the server Node2 with RHEL 7.4 OS

d.      Add Node2 back to Oracle Cluster

2.       Upgrade OS to RHL7.4 in Node1:

a.       Relocate all DB services from Node1 to Node2

Example:

# srvctl relocate service -d DB_Name -s Service_Name -i DB_Instance_1 -t DB_Instance_2

b.      Remove Node1 from Cluster

c.       Reimage the server Node1 with RHEL 7.4 OS

d.      Add Node1 back to Oracle Cluster

References:

The below mentioned Oracle document provides detailed steps for adding and deleting cluster nodes on Linux.

https://docs.oracle.com/cd/E11882_01/rac.112/e41959/adddelclusterware.htm#CWADD1167

The below mentioned Oracle document provides more details on having mixed operating systems versions during upgrade.

https://support.oracle.com/epmos/faces/DocumentDisplay?id=1072552.1

Database Migration and Upgrade from Oracle 11g to 12c.

Hello Friends, hope you are doing well.

I recently did a database migration to new hardware/servers and then upgraded from 11.2.0.4 to 12.1.0.2.

For this migration and upgrade activity, followed the below approach to minimize the downtime as possible.

This migration and upgrade activity can be broadly split into two phases.

1.       Migrate the database to new hardware/servers. (e.g. OS version RHEL 6.x to 7.x)

2.       Upgrade the database from 11g to 12c

1.       Migrate the database to new servers

·         Setup two node Oracle 12c cluster in new servers

The below Oracle document can be referred to know how to setup Oracle 12c Cluster in new servers:

https://docs.oracle.com/en/database/oracle/oracle-database/12.2/cwlin/index.html

·         Install RDBMS software 11.2.0.4 in new servers.

This 11g Oracle home is needed to setup physical standby database first for migrating the 11g database from old server to this new server.

·         Setup a Physical standby database 11.2.0.4 in new servers so that both the databases in old and new servers are in synch with Data guard replication

The below Oracle document can be referred to know how to create Oracle 11g physical standby database and how to perform database switchover (Database role transitions):

https://docs.oracle.com/cd/E11882_01/server.112/e41134/title.htm

·         During the change window, activate physical standby database so that it will become primary database (Read and Write mode).

·         Now the 11g database is up and running in new servers

2.       Upgrade database from 11g to 12c in new servers

·         Install Oracle 12c RDBMS software in new servers.

·         Now upgrade this 11.2.0.4 database to 12c, in new servers (This 11g database has been migrated to this new servers as explained in above steps).

     By default upgraded database will be Non-CDB.

The below Oracle document can be referred to know how to upgrade the database to Oracle 12c:

https://docs.oracle.com/en/database/oracle/oracle-database/12.2/upgrd/index.html

References:

·    Oracle 12c Grid Infrastructure Installation and Upgrade Guide

https://docs.oracle.com/en/database/oracle/oracle-database/12.2/cwlin/index.html

·    Oracle 12c Database Upgrade Guide, for upgrading database to 12c

https://docs.oracle.com/en/database/oracle/oracle-database/12.2/upgrd/index.html

·      Oracle 11gR2 - Data Guard Concepts and Administration

https://docs.oracle.com/cd/E11882_01/server.112/e41134/title.htm