Home

Configuring internal hive metastore (Databricks)

This topic provides instructions to configure the Databricks internal hive metastore with Unravel.

Prerequisites

  • You must have admin privileges for Databricks workspace.

  • Enable Web Terminal access for the workspace. This is needed only once for getting the internal hive metastore details. To enable Web terminal access, do the following:

    1. In your Databricks workspace, click Settings > Admin Console > Workspace Settings.

    2. Click the Web Terminal toggle.

      db-internal-metastore-enable-web-terminal.png

To configure the Databricks internal hive metastore with Unravel, do the following:

  1. Create a single-node cluster on Databricks and start it. This cluster is needed only once and can be deleted after you get the internal hive metastore configurations.

  2. After the cluster is started, launch a web terminal for that cluster. On the Cluster page, click the Apps tab, and then click Launch Web Terminal.

    db-internal-metastore-launch-web-terminal.png

  3. After launching the web terminal, using a vi editor, open the /databricks/hive/conf/hive-site.xml file.

  4. Copy the values of the following properties from the hive-site.xml file and create a temporary .txt or .prop file.

    Following is a sample of such a file:

    com.unraveldata.hive.metastore.list=<workspacename1>,<workspacename2>
hive.metastore.cluster.ids=default
hive.metastore.<workspacename1>.workspace.ids=<workspace1_ID>
javax.jdo.option.<workspacename1>.ConnectionURL=jdbc:mariadb://<db_instance_name>:3306/organizationxxxxxxxxxx?useSSL=true&amp;enabledSslProtocolSuites=TLSv1,TLSv1.1,TLSv1.2&amp;serverSslCert=/tmp/mysql-ssl-ca-cert.crt
javax.jdo.option.<workspacename1>.ConnectionDriverName=org.mariadb.jdbc.Driver
javax.jdo.option.<workspacename1>.ConnectionUserName=XXXxxxXXXXX4UXXXXXXXxxxxXXXX
javax.jdo.option.<workspacename1>.ConnectionPassword=XXXxxxXXXXX4UXXXXXXXxxxxXXXX
hive.metastore.<workspacename2>.workspace.ids=<workspace2_ID>
javax.jdo.option.<workspacename2>.ConnectionURL=jdbc:mariadb://<db_instance_name>:3306/organizationxxxxxxxxxx?useSSL=true&amp;enabledSslProtocolSuites=TLSv1,TLSv1.1,TLSv1.2&amp;serverSslCert=/tmp/mysql-ssl-ca-cert.crt
javax.jdo.option.<workspacename2>.ConnectionDriverName=org.mariadb.jdbc.Driver
javax.jdo.option.<workspacename2>.ConnectionUserName=XXXxxxXXXXX4UXXXXXXXxxxxXXXX
javax.jdo.option.<workspacename2>.ConnectionPassword=XXXxxxXXXXX4UXXXXXXXxxxxXXXX

    For the ConnectionURL, you must provide a valid SSL certificate path. Ensure to change the path of the SSL certificate in the JDBC URL to the path of the Unravel node where the SSL certificate is downloaded.

    Note

    SSL certificate for Azure Databricks can be downloaded from here.

  5. Using a terminal from the installation directory, where Unravel binaries are installed, run the following command to import the temporary properties file that you have created. 

    <Unravel installation directory>/unravel/manager config properties import <path to the properties file>
For example:
/opt/unravel-install/unravel/manager config properties import /opt/properties.txt

  6. Ensure to stop Unravel. 

    <Unravel installation directory>/unravel/manager config stop

  7. Apply the changes. 

    <Unravel installation directory>/unravel/manager config apply

  8. Start Unravel. 

    <Unravel installation directory>/unravel/manager start
In this section: