October 13, 2022

Live Snowflake Marketplace Data In Oracle

Access the Live Snowflake Marketplace Data in Oracle Database



What is Snowflake Data Marketplace?


Snowflake Data Marketplace connects the Providers of data with Consumers. The Snowflake Secure data sharing enables organizations to buy and sell the secure, governed data in Snowflake Data Marketplace.

The benefits of Snowflake Data Marketplace:

  • No data movement - Consumers are not required to copy the data, data is stored in provider accounts.
  • Ready to query data - Consumers can immediately start querying data without
  • transformation.
  • Discover a variety of data - Consumers can search for a variety of data -Finance, Marketing, Health and Life Sciences, Sports, Travel, Lookup tables, etc.
  • Live access - Receive real-time automatic updates.
  • Reduce cost - Eliminates the costs and delays associated with the traditional ETL processes of data loading and transformation

Steps to access the Live Snowflake Marketplace Data in the Oracle Database



Architecture Diagram:




Here are the steps to access the Snowflake Marketplace data in Oracle 19c database running on Linux server.

1.    Login to Snowflake website and search for the required data sources in Snowflake Data Marketplace

2.    Get the data from Snowflake Marketplace, this will just create a pointer object in the Snowflake                 account, so that data can be accessed directly from the provider account. 
      For this demo, accessing the zip code data from Marketplace and named this share as                  
        “MP_ZIPCODE” in my Snowflake account

3.    Create a view in your Snowflake account database as below with the required columns/format joining the required tables from marketplace share that you got in previous step #2.

CREATE OR REPLACE VIEW VW_ZIPCODE AS
SELECT ZIP::VARCHAR(5) ZIP, USPS_ZIP_PREF_CITY::VARCHAR(30)
CITY_NAME,USPS_ZIP_PREF_STATE::VARCHAR(30) STATE_NAME
FROM "MP_ZIPCODE"."ZIPCODE_CROSSWALK"."CBSA_ZIP";

4.    Connect to Oracle database server

5.    Install Linux ODBC Driver Manager: unixODBC

  • Execute the following command to verify whether unixODBC is installed.

[root@localhost etc]# which odbcinst
/usr/bin/which: no odbcinst in (/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin)

[root@localhost etc]# which isql
/usr/bin/which: no isql in (/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin)

  • If unixODBC is not installed, as root execute the following commands:

# yum install unixODBC.x86_64

Installed:
unixODBC.x86_64 0:2.3.1-14.0.1.el7
Complete!

  • Verify the unixODBC installation:

[root@localhost etc]# which odbcinst
/bin/odbcinst

[root@localhost etc]# which isql
/bin/isql

[root@localhost ~]# odbcinst -j
unixODBC 2.3.1
DRIVERS............: /etc/odbcinst.ini
SYSTEM DATA SOURCES: /etc/odbc.ini
FILE DATA SOURCES..: /etc/ODBCDataSources
USER DATA SOURCES..: /root/.odbc.ini
SQLULEN Size.......: 8
SQLLEN Size........: 8
SQLSETPOSIROW Size.: 8

6.    Download the latest ODBC driver for Linux from:


7.     Install the ODBC Driver for Linux using the RPM package, downloaded at step #6

# yum install snowflake-odbc-2.25.5.x86_64.rpm

Installed:
snowflake-odbc.x86_64 0:2.25.5-1
Complete!

Note - The installation directory is “/usr/lib64/snowflake/odbc”

8. Configure the ODBC Driver

Update “/etc/odbc.ini” file as below:

[root@localhost etc]# cat odbc.ini
[ODBC Data Sources]
snowflakedb=SnowflakeDSIIDriver
[snowflakedb]
Driver = /usr/lib64/snowflake/odbc/lib/libSnowflake.so
Description =
Server = <Full Snowflake URL>
role = <Snowflake Role>
database = DEMODB
schema = SCHEMA1
warehouse = test_wh
Locale=en-US
PORT=443
SSL=on

9. Test the ODBC Driver

# isql -v snowflakedb <Snowflake account username> <account password>
+---------------------------------------+
| Connected! |
| |
| sql-statement |
| help [tablename] |
| quit |
| |
+---------------------------------------+
SQL> select zip, city_name,state_name from VW_ZIPCODE where zip='10001';
+------+-------------------------------+-------------------------------+
| ZIP | CITY_NAME | STATE_NAME |
+------+-------------------------------+-------------------------------+
| 10001| NEW YORK | NY |
+------+-------------------------------+-------------------------------+
SQLRowCount returns 1
1 rows fetched


10.    Oracle Database Configuration

10.1.    Create “initsnowflakedb.ora” in “$ORACLE_HOME/hs/admin” as below:

$ cd $ORACLE_HOME/hs/admin

$ cat initsnowflakedb.ora
# This is a sample agent init file that contains the HS parameters that are
# needed for the Database Gateway for ODBC
#
# HS init parameters
#
HS_FDS_CONNECT_INFO = "SNOWFLAKEDB"
HS_FDS_SHAREABLE_NAME = /usr/lib64/snowflake/odbc/lib/libSnowflake.so
HS_LANGUAGE=AMERICAN_AMERICA.WE8MSWIN1252
# HS_FDS_TRACE_LEVEL = Debug
# HS_FDS_TRACE_FILE_NAME = /tmp/hstrace.log
#
# ODBC specific environment variables
#
set ODBCINI=/etc/odbc.ini
#
# Environment variables required for the non-Oracle system
#

10.2. Update listener.ora in $ORACLE_HOME/network/admin as below:

[oracle@localhost admin]$ cat listener.ora

LISTENER =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = localhost)(PORT = 1521))
(ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1521))
)
)

SID_LIST_LISTENER=
(SID_LIST=
(SID_DESC=
(SID_NAME=snowflakedb)
(ORACLE_HOME=/u01/app/oracle/product/19.3.0/dbhome_1)
(ENVS="LD_LIBRARY_PATH=/usr/lib64:/usr/lib64/snowflake/odbc/lib:/u01/app/oracle/p
roduct/19.3.0/dbhome_1/hs/lib:/u01/app/oracle/product/19.3.0/dbhome_1/lib")
(PROGRAM=dg4odbc)
)
)



10.3. Add entry to “tnsnames.ora” in $ORACLE_HOME/network/admin as below:

snowflakedb=
(DESCRIPTION=
(ADDRESS=(PROTOCOL=TCP)(HOST=localhost)(PORT=1521))
(CONNECT_DATA= (SID=snowflakedb))
(HS=OK)
)

10.4. Start/Restart the listener

$ lsnrctl stop

$ lsnrctl start


10.5. Test connection

[oracle@localhost admin]$ tnsping snowflakedb

TNS Ping Utility for Linux: Version 19.0.0.0.0 - Production on 28-SEP-2022 23:31:12
Copyright (c) 1997, 2019, Oracle. All rights reserved.
Used parameter files:
/u01/app/oracle/product/19.3.0/dbhome_1/network/admin/sqlnet.ora
Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION=
(ADDRESS=(PROTOCOL=TCP)(HOST=localhost)(PORT=1521)) (CONNECT_DATA=
(SID=snowflakedb)) (HS=OK))
OK (10 msec)


11. Connect to Oracle database and create the database link as below:

SQL> CREATE PUBLIC DATABASE LINK snowflakedb connect to
"<Snowflake account username>" identified by “<account password>”
using 'SNOWFLAKEDB';

12. Now access the Snowflake Marketplace live data in Oracle database

You can now execute the queries in SQL*Plus as below:

SQL> col zip format a10
SQL> col CITY_NAME format a30
SQL> col STATE_NAME format a30

SQL> select zip, city_name,state_name from VW_ZIPCODE@snowflakedb where zip='10001';

ZIP CITY_NAME STATE_NAME
---------- ------------------------------ ------------------------------
10001 NEW YORK NY


Click Here to watch my YouTube Video, that demonstrates each of the above steps in detail.


March 11, 2022

Oracle's AutoUpgrade Utility

The AutoUpgrade utility helps to upgrade one or many Oracle databases with a single command and single configuration file.

In this blog, explaining how to upgrade Oracle databases from 12.2 to 19.3 release using "AutoUpgrade" utility.

Benefits of AutoUpgrade:


  • The AutoUpgrade helps to upgrade one or many databases without human intervention, all with one command and a single configuration file.
  • The AutoUpgrade utility is designed to automate the upgrade process - It identifies issues before upgrade. It performs pre and post upgrade actions. It performs database upgrades and starts the upgraded Oracle databases.
  • It enables to schedule upgrades for future points in time
  • It enables to set, change or remove initialization parameters as desired
  • It enables you to call the custom scripts during the upgrade to configure databases.

Target Versions Supported:

The AutoUpgrade utility can be used to upgrade Oracle databases to below listed target versions:

  • Oracle Database 21c (21.3 and newer)
  • Oracle Database 19c (19.3 and newer)
  • Oracle Database 18c (18.5 and newer)
  • Oracle Database 12c Release 2 (12.2 + DBJAN2019RU and newer)


AutoUpgrade Modes: 

The steps that AutoUpgrade runs to upgrade the database depends on the processing mode that you select.

  • Analyze Mode: Setup, Prechecks.
  • Fixups Mode: Setup, Prechecks, and Prefixups.
  • Deploy Mode: Setup, guaranteed restore point (GRP), Preupgrade, Prechecks, Prefixups, Drain, DB (database) Upgrade, Postchecks, and Postupgrade. You can run your own scripts before the upgrade (Preupgrade stage) or after the upgrade (Postupgrade stage), or both before and after the upgrade.
  • Upgrade Mode: Setup, DB (database) Upgrade, Postchecks, and Postfixups.


Autoupgrade - Steps


        1. Download and install the Oracle 19.3 software binaries.


I have already installed Oracle 19.3 software. 


2. Verify Java version. 


The AutoUpgrade Utility requires Java 8. 


[oracle@localhost ~]$ $ORACLE_HOME/jdk/bin/java -version


java version "1.8.0_201"

Java(TM) SE Runtime Environment (build 1.8.0_201-b09)


Note - Oracle_home is the Target Oracle Database home. 


          3. Download the latest AutoUpgrade version

  • The autoupgrade.jar file exists by default, in Oracle Database 19c (19.3) and later targets Oracle homes. Oracle strongly recommends downloading the latest AutoUpgrade version from My Oracle Support Document 2485457.1.
  • The autoupgrade.jar file exists in the $ORACLE_HOME/rdbms/admin directory. 
  • For both Oracle Database 12c Release 2 (12.2) and Oracle Database 18c (18.5) target homes, you must download the AutoUpgrade kit from My Oracle Support Document 2485457.1.
  • Run below command to verify AutoUpgrade version:

[oracle@localhost admin]$ cd $ORACLE_HOME/rdbms/admin


[oracle@localhost admin]$ $ORACLE_HOME/jdk/bin/java -jar autoupgrade.jar -version


Note - Oracle_home is the Target Oracle Database home. 



  • Download the latest autoupgrade.jar file from My Oracle Support Document 2485457.1. Replace “autoupgrade.jar” which is already present in $ORACLE_HOME/rdbms/admin directory, with latest downloaded file.
  • Now run below command to verify AutoUpgrade version. 

It should be latest version as below:


[oracle@localhost admin]$ $ORACLE_HOME/jdk/bin/java -jar autoupgrade.jar -version



        4. Create the Configuration file as below. AutoUpgrade configuration files contain all the information required to perform Oracle Database upgrades.



        5. Run Autoupgrade in Analyze mode to start analysis.


$ORACLE_HOME/jdk/bin/java -jar $ORACLE_HOME/rdbms/admin/autoupgrade.jar -config /u01/app/oracle/admin/autoupgrade/config.txt -mode analyze



        6. The main commands to monitor the Autoupgrade jobs, from Autoupgrade console are as below:


lsj – Lists the jobs by status

status – Lists the status of a particular job or all jobs

tasks – Displays the tasks that are running.


        7. Review the analysis Summary report.


            8. Once Autopgrade in Analyze mode completed successfully, run Autoupgrade in Deploy mode to start the database upgrade process as below:


$ORACLE_HOME/jdk/bin/java -jar $ORACLE_HOME/rdbms/admin/autoupgrade.jar -config /u01/app/oracle/admin/autoupgrade/config.txt -mode deploy




            9. Review the AutoUpgrade log files for any errors


    10. Databases upgraded from 12.2 to 19.3 successfully, using the Autoupgrade tool.
            

         Connect to databases and verify.  


         The Autoupgrade tool updates the new target Oracle home in “/etc/oratab” for all upgraded databases.


                    Database 1:


                           Database 2:




October 17, 2021

Migrating database to AWS using AWS DMS

There are many tools to migrate databases to AWS, depending on the type of database migration. For example, for migrating Oracle databases to AWS RDS Oracle, the native tools like data pump export/import or Golden gate also works more effectively.

In this blog, explaining how to migrate data to AWS with the help of AWS DMS service.


Architecture Diagram


AWS DMS, also known as AWS Database Migration Service is AWS cloud service that helps to migrate data, from on-premises databases to Amazon RDS or databases running on Amazon EC2 to RDS, as well as from one Amazon RDS database to another Amazon RDS database.

AWS DMS migrates data, tables and primary keys to the target database. All other database elements are not migrated.

AWS Schema Conversion Tool (SCT) is another AWS cloud service, it helps to migrate schema and code from source database to target database. In case of heterogeneous database migrations, AWS Schema Conversion Tool (SCT) helps to convert schema and code from source database to target database.


Please click on the below link, for my article posted in LinkedIn, explaining the Key benefits of AWS DMS, Limitations of AWS DMS and how to migrate databases to AWS using AWS DMS. 

https://www.linkedin.com/posts/vadirajhonwad_datamigration-databasemigration-awsmigration-activity-6854527909235855360-pYos


September 6, 2020

Oracle 19c Grid Infrastructure(GI) Rolling upgrade

I tested the Oracle Grid Infrastructure (GI) rolling upgrade from 12.2 to 19.8a few days back.

It's two node RAC cluster and the OS version is RHEL 7.6.


Followed the below steps to perform Rolling upgrade of Oracle Grid Infrastructure(GI) 

from 12.2 to 19.8:


  1. Ensure that the latest PSU is applied to 12.2 Grid Infrastructure (GI) home.

  1. Review/Configure Operating system for 19c upgrade:

Ensure that server is having the supported operating system version, kernel release and all the required operating system packages, for 19c upgrade.

The below Oracle document can be referred for more details.


  1. You need root user access to run “rootupgrade.sh” script during upgrade

  1. Create directory structure for Oracle 19c GI home:

mkdir -p /u01/app/19.3.0/grid
chown -R grid:oinstall /u01/app/19.3.0
chmod -R 775 /u01/app/19.3.0

  1. Download the Oracle 19c (19.3) Grid Infrastructure software, for Linux x86-64 to a directory (Example – “/oracle_software/19.3.0”)

  1. Extract downloaded 19c GI software in above step to 19c GI home directory created in above step # 4

cd /oracle_software/19.3.0/

unzip LINUX.X64_193000_grid_home.zip -d /u01/app/19.3.0/grid

  1. Download and extract the latest Oracle 19c GI RU patch to a working directory (“/oracle_software/19.3.0/31305339”).

I used the current available latest RU patch
"Patch 31305339: GI Release Update 19.8.0.0.200714" patch.

  1. Use Cluster Verification Utility (CVU) to perform pre-checks for upgrade

cd /u01/app/19.3.0/grid

./runcluvfy.sh stage -pre crsinst -upgrade -rolling -src_crshome /u01/app/12.2.0.1/grid -dest_crshome /u01/app/19.3.0/grid -dest_version 19.0.0.0.0 -fixup -verbose

Review the Cluster Verification Utility (CVU) output and fix the errors/failures.

  1. Start the 19.8 Grid Infrastructure rolling upgrade as below:

The patch # 31305339 "GI Release Update 19.8" will be applied to the 19c GI home during upgrade.

Note - Ensure to set the environment, so that you can open the Graphical User Interface (GUI) for Oracle GI 19c installer.

$ unset ORACLE_BASE
$ unset ORACLE_HOME
$ unset ORACLE_SID

cd /u01/app/19.3.0/grid

./gridSetup.sh -applyPSU /oracle_software/19.3.0/31305339

Oracle Grid Infrastructure 19c Installer will start. There are 9 steps in the Installer.

Step 1 - Select Configuration option:
Choose “Upgrade Oracle Grid Infrastructure” and then click “Next”

Step 2 - Select all Cluster nodes and then click “Next”.

Step 3 - Specify Management options:
Update the details to perform registration with Enterprise Manager or uncheck and click “Next”.

Step 4 - Specify installation Location:
Review/Update Oracle Base and GI home path and then click “Next”.

Step 5 - Root script execution configuration:
            Uncheck “Automatically run configuration scripts” and then click “Next”.

Step 6 - Perform prerequisites check:
Review and fix if there are any errors. Click “Next” once there are no errors in the prerequisites check page.

Step 7 - Summary:
Review Summary and click “Submit”.

Step 8 - Install Product:
Installer will proceed with the upgrade.

Run “rootupgrade.sh” script, as root user, when installer prompts.
Run on local node first, after successful completion you can run on all other nodes in parallel.

Once the “rootupgrade.sh” script execution is completed on all nodes, come back to the Installer window and then click “OK” on the Installer UI window.

Step 9 - Finish
The rolling upgrade of Oracle Grid Infrastructure 19.8 completed successfully.
Completed all 9 steps of upgrade in GI 19c installer.
Click “Close”.

  1. Verify Grid infrastructure version using below commands, post upgrade:

$ <GI Home>/bin/crsctl query crs softwareversion -all

Oracle Clusterware version on node [Node1] is [19.0.0.0.0]
Oracle Clusterware version on node [Node2] is [19.0.0.0.0]

$ <GI Home>/bin/crsctl query crs activeversion

Oracle Clusterware active version on the cluster is [19.0.0.0.0]

  1. Verify all GI services are online using below commands, post upgrade:

<GI Home>/bin/crsctl check crs
<GI Home>/bin/crsctl stat res -t

November 26, 2019

How to access the Geo-Blocked websites using AWS


Hello Friends, hope you are doing well.

I just wanted to share my findings/observations, on how to access geographically blocked websites using AWS.

Hope this may be helpful, when you are trying to access your personal information back home, while travelling abroad. Also, if you want to watch your favorite programs while travelling, for example Indian application “Voot”, which works only in India.

VPNs are also good option for accessing the content that is blocked in your own country. There are lots of VPNs out there to choose from, for watching streaming media and services from another country.

AWS is another option to access the Geo-blocked websites.

The below listed high level steps can be followed to access Geo-blocked websites using AWS.

·       Login to AWS console
·       Change Region in AWS to specific region, from where you are trying access website.
Example: “Asia Pacific (Mumbai)” to access Indian application “Voot”
·       Open EC2 service page and create an EC2 instance with Windows OS
·       Connect to this EC2 instance using “Remote Desktop Connection” from your laptop/PC
·    Open IE (Internet Explorer) in EC2 instance and then you can browse the website that you couldn’t access from your laptop/PC

Hope this may be helpful, if you are trying to access Geo-blocked websites.