- Home
- Work-in-Progress Documentation
- Configuration
- Hive Metastore
Hive Metastore
Unravel should connect to your Hive Metastore to populate the Data page by analyzing your table usage in conjunction with Hive query information. Unravel Server collects information through the Hive API, which is similar to the beeline CLI and uses the JDBC database connection protocol. Data
This topic takes you through the steps of connecting Unravel to the Hive Metastore. If you are using Oracle you also have to configure it to grant Unravel access.
Connecting to Hive metastore
Hive metastore connection can be set either with auto-configuration or manual configuration.
Option 1: Auto-configuration
Stop Unravel
<Unravel installation directory>/unravel/manager stop
Run auto-configuration.
<unravel_installation_directory>/unravel/manager config auto
Note
In a multi-cluster deployment, the command is run differently based on whether the hive metastore is on core node or the edge node. In both the cases, the following commands must be run only from the core node.
When the hive metastore is on the core node, run the following from the core node:
<unravel_installation_directory>/unravel/manager config auto
When the hive metastore is on the edge node, run the following command also from the core node:
<unravel_installation_directory>/unravel/manager config edge auto <edge-key>
Auto-configuration will set all the properties for the JDBC connection to the database of the Hive metastore on the current node except for the password. In case, the Hive metastore password is not found during auto-configuration, you will be prompted to set the password.
Set the Hive metastore password.
Note
This step is required only if the password is not found during the auto-configuration.
Tip
Run the manager config edge show command to get the
<HIVE_KEY>
<CLUSTER_KEY>
<HIVE_KEY>
is the definition of the Hive service.CLUSTER_KEY
is the name of the cluster where you set the Hive configurations.
In a single cluster deployment set the password as follows:
<Unravel installation directory>/unravel/manager config hive metastore password
<CLUSTER_KEY>
<password>
## For example <Unravel installation directory>/unravel/manager config hive metastore password my-cluster passwordIn a multi-cluster deployment, where edge nodes are monitoring, set the password from the core node as follows:
<Unravel installation directory>/unravel/manager config hive metastore password
<CLUSTER_KEY>
<HIVE_KEY>
<password>
(If core is monitoring) <Unravel installation directory>/unravel/manager config edge hive metastore password<CLUSTER-KEY>
<HIVE-KEY>
<password>
##For example: <Unravel installation directory>/unravel/manager config edge hive metastore password my-cluster hive passwordAlso, refer to Encrypting/Decrypting passwords.
Apply the changes.
<Unravel installation directory>/unravel/manager config apply
Start Unravel
<Unravel installation directory>/unravel/manager start
Go to Unravel UI's Jobs > Applications page to confirm that Hive queries are displayed. Approximately twenty-four hours after configuration the Data page displays a list of your Hive Metastore tables along with their KPIs and other details.
Option 2: Manual configuration
Stop Unravel
<Unravel installation directory>/unravel/manager stop
Run the following command to connect to hive metastore.
Tip
Run the manager config edge show command to get the
<EDGE_KEY>
<CLUSTER_KEY>
<HIVE_KEY>
details.<EDGE_KEY>
is the label you provide to identify the edge node when you set the cluster.<CLUSTER_KEY>
is the name of the cluster where you set the Hive configurations.<HIVE_KEY>
is the definition of the Hive service.
Single cluster deployment
manager config hive metastore set
<CLUSTER_KEY>
HIVE_KEY
DRIVER
URL
USER
PASSWORD
##For example: manager config edge hive metastore set my-cluster hive oracle.jdbc.driver.OracleDriver jdbc:oracle:thin:@prodHost:1521:ORCL user passcodeMulti-cluster deployment
manager config edge hive metastore set
<EDGE_KEY>
<CLUSTER_KEY>
<HIVE_KEY>
DRIVER
URL
USER
PASSWORD
##For example: manager config edge hive metastore set my-edge my-cluster hive oracle.jdbc.driver.OracleDriver jdbc:oracle:thin:@prodHost:1521:ORCL user passcode
This will set the following metastore database information:
JDBC driver: JDBC Driver class name for the data store containing the metadata. For example:
MySQL: com.mysql.jdbc.Driver
Oracle: oracle.jdbc.driver.OracleDriver
Microsoft: com.microsoft.sqlserver.jdbc.SQLServerDriver
JDBC URL: JDBC connection string for the datastore containing the metadata of the form.
jdbc:
DB_Driver
://HOST:PORT/
hiveFor example:
Oracle: jdbc:oracle:thin:@prodHost:1521:ORCL
Microsoft: jdbc:sqlserver://jdbc_url
Username: Username used to access the data store.
Password: Password used to access the data store.
Apply the changes.
<Unravel installation directory>/unravel/manager config apply
Start Unravel
<Unravel installation directory>/unravel/manager start