Skip to main content

Home

Configuring external hive metastore (Databricks)

You can configure external metastores of Databricks' workspaces for Unravel monitoring.

Refer to External Apache Hive metastore to set up hive metastores with Azure Databricks.

Copy database jar for unity catalog metastore configuration

Ensure that the database jar file for MySQL, MariaDB, or Oracle DB is copied to the following location before you configure Unity Catalog metastore. 

<Unravel installation directory>/unravel/share/java/

For example:

/opt/unravel/share/java/

To configure the metastore, do the following:

  1. Create a .txt or .prop file containing the following properties and their corresponding values in any location accessible to Unravel users. Following is a sample of the file:

    Tip

    To get the external hive metastore details, select Workspace Cluster and check for Spark and Hive configurations. For additional information, you can refer here. 

    com.unraveldata.hive.metastore.list=<workspacename1>,<workspacename2>
hive.metastore.cluster.ids=default
hive.metastore.<workspacename1>.workspace.ids=<workspace1_ID>
javax.jdo.option.<workspacename1>.ConnectionURL=<jdbc connection URL>
javax.jdo.option.<workspacename1>.ConnectionDriverName=<jdbc driver name>
javax.jdo.option.<workspacename1>.ConnectionPassword=<jdbc connection password>
javax.jdo.option.<workspacename1>.ConnectionUserName=<jdbc connection username>
hive.metastore.<workspacename2>.workspace.ids=<workspace2_ID>
javax.jdo.option.<workspacename2>.ConnectionURL=<jdbc connection URL>
javax.jdo.option.<workspacename2>.ConnectionDriverName=<jdbc driver name>
javax.jdo.option.<workspacename2>.ConnectionPassword=<jdbc connection password>
javax.jdo.option.<workspacename2>.ConnectionUserName=<jdbc connection username>

    Properties

    Description

    Workspace ID

    Workspace ID. You can get this workspace ID from the Unravel UI. Go to Manage > Workspaces.

    Workspace Name

    Name of your workspace.

    JDBC connection URL

    JDBC connection string for the hive metastore containing the metadata of the form: jdbc:DB_Driver://HOST:PORT/hive

    JDBC driver name

    JDBC Driver class name for the hive metastore containing the metadata

    JDBC connection password

    Password used to access the hive metastore.

    JDBC connection username

    Username used to access the hive metastore.

    For example:

    com.unraveldata.hive.metastore.list=Data-Science-Workspace,Bank-Application-Workspace
hive.metastore.cluster.ids=default
hive.metastore.Bank-Application-Workspace.workspace.ids=1953907393469835
hive.metastore.Data-Science-Workspace.workspace.ids=13032107253570
javax.jdo.option.Bank-Application-Workspace.ConnectionURL=jdbc:mysql://un4720mysql.mysql.database.azure.com:3306/testmetaold1?useSSL=true&requireSSL=false
javax.jdo.option.Bank-Application-Workspace.ConnectionDriverName=com.mysql.jdbc.Driver
javax.jdo.option.Bank-Application-Workspace.ConnectionPassword=AES(AES-encrypted-passcode)
javax.jdo.option.Bank-Application-Workspace.ConnectionUserName=mysql@un4720mysql
javax.jdo.option.Data-Science-Workspace.ConnectionDriverName=com.mysql.jdbc.Driver
javax.jdo.option.Data-Science-Workspace.ConnectionPassword=AES(AES-encrypted-passcode)
javax.jdo.option.Data-Science-Workspace.ConnectionURL=jdbc:mysql://un4720mysql.mysql.database.azure.com:3306/stockbrokerportfolio?useSSL=true&requireSSL=false
javax.jdo.option.Data-Science-Workspace.ConnectionUserName=mysql@un4720mysql

  2. From the installation directory, where Unravel binaries are installed, run the following command and provide the path to the properties file.

    <Unravel installation directory>/unravel/manager config properties import <path to the properties file>
For example:
/opt/unravel-install/unravel/manager config properties import /opt/properties.txt

  3. Ensure to stop Unravel.

    <Unravel installation directory>/unravel/manager stop

  4. Apply the changes.

    <Unravel installation directory>/unravel/manager config apply

  5. Start Unravel.

    <Unravel installation directory>/unravel/manager start
In this section: