Home

Hive Metastore

Unravel should connect to your Hive Metastore to populate the Data page by analyzing your table usage in conjunction with Hive query information. Unravel Server collects information through the Hive API, which is similar to the beeline CLI and uses the JDBC database connection protocol. Data

This topic takes you through the steps of connecting Unravel to the Hive Metastore. If you are using Oracle you also have to configure it to grant Unravel access.

Connecting to Hive metastore

Hive metastore connection can be set either with auto-configuration or manual configuration.

Option 1: Auto-configuration

  1. Stop Unravel

    <Unravel installation directory>/unravel/manager stop
    
  2. Run auto-configuration.

    <unravel_installation_directory>/unravel/manager config auto
    

    Note

    In a multi-cluster deployment, the command is run differently based on whether the hive metastore is on core node or the edge node. In both the cases, the following commands must be run only from the core node.

    • When the hive metastore is on the core node, run the following from the core node:

      <unravel_installation_directory>/unravel/manager config auto
      
    • When the hive metastore is on the edge node, run the following command also from the core node:

      <unravel_installation_directory>/unravel/manager config edge auto <edge-key>
      

    Auto-configuration will set all the properties for the JDBC connection to the database of the Hive metastore on the current node except for the password. In case, the Hive metastore password is not found during auto-configuration, you will be prompted to set the password.

  3. Set the Hive metastore password.

    Note

    This step is required only if the password is not found during the auto-configuration.

    Tip

    Run the manager config edge show command to get the <HIVE_KEY> <CLUSTER_KEY>

    • <HIVE_KEY> is the definition of the Hive service.

    • CLUSTER_KEY is the name of the cluster where you set the Hive configurations.

    In a single cluster deployment set the password as follows:

    <Unravel installation directory>/unravel/manager config hive metastore password <CLUSTER_KEY> <password>
    
    ## For example
    <Unravel installation directory>/unravel/manager config hive metastore password my-cluster password

    In a multi-cluster deployment, where edge nodes are monitoring, set the password from the core node as follows:

    <Unravel installation directory>/unravel/manager config hive metastore password <CLUSTER_KEY> <HIVE_KEY> <password> (If core is monitoring)
    <Unravel installation directory>/unravel/manager config edge hive metastore password <CLUSTER-KEY> <HIVE-KEY> <password> 
    
    ##For example: 
    <Unravel installation directory>/unravel/manager config edge hive metastore password my-cluster hive password
    

    Also, refer to Encrypting/Decrypting passwords.

  4. Apply the changes.

    <Unravel installation directory>/unravel/manager config apply
    
  5. Start Unravel

    <Unravel installation directory>/unravel/manager start

    Go to Unravel UI's Jobs > Applications page to confirm that Hive queries are displayed. Approximately twenty-four hours after configuration the Data page displays a list of your Hive Metastore tables along with their KPIs and other details.

Option 2: Manual configuration

  1. Stop Unravel

    <Unravel installation directory>/unravel/manager stop
    
  2. Run the following command to connect to hive metastore.

    Tip

    Run the manager config edge show command to get the <EDGE_KEY> <CLUSTER_KEY> <HIVE_KEY> details.

    • <EDGE_KEY> is the label you provide to identify the edge node when you set the cluster.

    • <CLUSTER_KEY> is the name of the cluster where you set the Hive configurations.

    • <HIVE_KEY> is the definition of the Hive service.

    • Single cluster deployment

      manager config hive metastore set <CLUSTER_KEY> HIVE_KEY DRIVER URL USER PASSWORD
      
      ##For example:
      manager config edge hive metastore set my-cluster hive oracle.jdbc.driver.OracleDriver jdbc:oracle:thin:@prodHost:1521:ORCL user passcode
    • Multi-cluster deployment

      manager config edge hive metastore set <EDGE_KEY> <CLUSTER_KEY> <HIVE_KEY> DRIVER URL USER PASSWORD
      
      ##For example:
      manager config edge hive metastore set my-edge my-cluster hive oracle.jdbc.driver.OracleDriver jdbc:oracle:thin:@prodHost:1521:ORCL user passcode

    This will set the following metastore database information:

    • JDBC driver: JDBC Driver class name for the data store containing the metadata. For example:

      • MySQL: com.mysql.jdbc.Driver

      • Oracle: oracle.jdbc.driver.OracleDriver

      • Microsoft: com.microsoft.sqlserver.jdbc.SQLServerDriver

    • JDBC URL: JDBC connection string for the datastore containing the metadata of the form.

      jdbc:DB_Driver://HOST:PORT/hive

      For example:

      • Oracle: jdbc:oracle:thin:@prodHost:1521:ORCL

      • Microsoft: jdbc:sqlserver://jdbc_url

    • Username: Username used to access the data store.

    • Password: Password used to access the data store.

  3. Apply the changes.

    <Unravel installation directory>/unravel/manager config apply
    
  4. Start Unravel

    <Unravel installation directory>/unravel/manager start