Microsoft Azure Databricks

Before installing Unravel in Azure Databricks, check and ensure that the Unravel installation requirements are completed and follow the below instructions to install and configure Unravel:

1. Create Unravel VM, create Azure Databricks, and install Unravel Prerequisites on Azure VM

2. Download Unravel

3. Deploy Unravel binaries

4. Install Unravel either with Interactive Precheck or manually

1. Create Unravel VM and Azure Databricks

Create Azure VM

Select Virtual Machines > Add and enter the following information in the Basics tab:

Project Details
Subscription	Choose the applicable subscription.
Resource group	Create a new group or choose an existing one.
Instance Details
Virtual Machine Name:	The Unravel server name.
Region:	Select the Azure region.
Availability Options	Select `No infrastructure is redundancy required`.
Image	Select the appropriate image. Both `Centos-based 7.x`+ and `Red Hat Enterprise Linux 7.x`+ are supported.
Size	Click Change Size. In the modal, select Memory-optimized image with at least `128 GB memory` and `Premium Disk support`, for example, E16s_v3 in East US 2)
Administrator account
Authentication type	Select `password` or `SSH Key`.
Username and Password	Enter your VM login information.
Inbound Port Rules
Public inbound ports	Select Allow selected ports.
Selected Inbound ports	Select both `HTTPS` and `SSH`.

Click Next: Disks > and enter the following information in the Disks tab.
Disk Options
- OS disk type: Select Premium SSD.
Data Disk
- Click Create and attach a new disk.
  Caution
  This disk is formatted, so do not choose the Attach an existing disk option.
  - Enter a Name.
  - Select Source typeNone (empty disk).
  - Set Size to at least 512 GiB.
  - Account type: Select premium SSD.
Click Next: Networking > and enter the following information:
- Virtual network: Create a new one or choose an existing one.
- Subnet: Create a new or choose an existing one.
- Public IP: Create a new one or choose an existing one.
- Select Inbound ports: Select HTTPS and SSH.
Click Review + create. Your deployment is now created.

Select Go to Resource > Networking > Inbound port rules > Add inbound port rule and include the following ports.

Rule Name	Destination	Destination IP Address	Destination Port Ranges
Unravel_3000	IP Addresses	NIC Private IP	3000
Unravel_443	IP Addresses	NIC Private IP	443
Unravel_4043	IP Addresses	NIC Private IP	4043
Unravel_4443	IP Addresses	NIC Private IP	443

Click OK.

Create Azure Databricks

Select Create a resource > Azure Databricks > Create. Go directly to step#3 if you already have workspaces.
Select Workspace name, Subscription, Resource group, Location, and Pricing tier.

Review VNET Peering options to connect Databricks with Unravel VM.

Databricks Workspace	Unravel VM	Region	VNET peering option
Deployed with VNET Injection	Deployed in the same VNET as Azure Workspace.	-	-
Deployed with VNET Injection	Deployed in different VNET from Azure Workspace.	Any Azure region	Create VNET Peering between the two VNETs
Deployed with Default VNET		Any Azure region	Create VNET Peering between the two VNETs

2. Download Unravel

Important

Before you download Unravel for your platform, ensure to get the username and password from Unravel Support.

Download the Unravel package (RPM or Tar):

You can refer to Download section for the complete list of Unravel product downloads.

For example:

Tar

curl -LO -u username:password https://preview.unraveldata.com/unravel/RPM/4.7.4/unravel-4.7.4.0.tar.gz

RPM

curl -LO -u username:password https://preview.unraveldata.com/unravel/RPM/4.7.4/unravel-4.7.4.0.rpm

3. Deploy Unravel

Unravel binaries are available as a tar file or RPM package. You can deploy the Unravel binaries in any directory on the server. However, the user who installs Unravel must have the write permissions to the directory where the Unravel binaries are deployed.

After deploying the Unravel binaries, the directory layout for both the Tar and RPM will be unravel/versions/<Directories and files>. The binaries are deployed to <Unravel_installation_directory>, and Unravel will be available in <Unravel_installation_directory/unravel.

The following steps to deploy Unravel from a tar file should be performed by a user, who will run Unravel.

Create an Installation directory.

mkdir /path/to/installation/directory
## For example: mkdir /opt/

Extract Unravel tar file to the installation directory, which you have created in the first step. After you extract the contents of the tar file, unravel directory is created within the installation directory.
```
tar zxf unravel-<version>tar.gz -C /path/to/installation/directory
## For example: tar zxf unravel-4.7.0.0.tar.gz -C /opt
## The unravel directory will be available within /opt
```

Grant ownership of the directory to a user who will run Unravel.

chown -R username:groupname /opt/unravel/

## For example: chown -R hadoop:hadoop /opt/unravel/

Important

The following steps to deploy Unravel from an RPM package should be performed by a root user. After the RPM package is deployed, the remaining installation procedures should be performed by unravel user.

Create an installation directory.
```
mkdir /usr/local/unravel
```

Run the following command:

rpm -i unravel-<version>.rpm
## For example: rpm -i unravel-4.7.0.0.rpm 
## The unravel directory will be available in /usr/local

If you want to provide a different location, use the --prefix command. For example:

mkdir /opt/unravel
rpm -i unravel-4.7.0.0.rpm --prefix /opt

## The unravel directory will be available in /opt

Grant ownership of the directory to a user who will run Unravel. This user executes all the processes involved in Unravel installation.
```
chown -R username:groupname /usr/local/unravel
## For example: chown -R hadoop:hadoop /usr/local/unravel
```
Continue with the installation procedures as unravel user.

4. Install Unravel

You can install Unravel either with Interactive Precheck or manually without Interactive Precheck.

Note

Unravel recommends installation with Interactive Precheck.

Install Unravel with Interactive Precheck

An Interactive Precheck utility is run to validate the required configurations before installing Unravel. When you run the Interactive Precheck utility various checks are prompted for gathering configuration information. The responses you provide for these checks are used to generate a bootstrap configuration file. This file, which contains the configuration information, is then used to bootstrap Unravel.

Do the following to install and configure Unravel with Interactive Precheck.

After you download and deploy the Unravel, run the precheck.sh script from unravel/versions/X.Y.Z/healthcheck/For example:
```
/opt/unravel/versions/X.Y.Z/healthcheck/precheck.sh
```

Enter the necessary details when you are prompted for the following configuration information:

This section covers general information about your Unravel install. You are prompted for the following:

Data platform you want to monitor.
For Hadoop: type of Unravel node you want to configure.
For edge nodes: core node location and test connectivity.

You must answer the following prompts:

-- General information
Which data platform are you installing for?
   1- Hadoop
   2- EMR
   3- HDI
   4- Databricks
   5- Dataproc

   Select one of the above [Hadoop]: 

Which type of Unravel node is this?
   1- Single node installation
   2- Core node
   3- Edge node

Select one of the above [Single node installation]: 
-- Core node - Log receiver information
   Hostname [None]: some.host.exampl
   Port (integer) [4043]: 
   TLS (y/n) [No]: y

   Attempting to connect to: https://some.host:4043/status

   Testing connection...
   Connection information:
     - Resolved IP:  89.163.146.49
     - Reverse lookup: ('app-002-de.some.host', [], ['89.163.146.49'])
     - Connection:     [Errno 111] Connection refused

   LR access: ERROR
   - ERROR: Fail to verify the connection

This check allows you to configure database-related information and an external database for Unravel.

-- Database configuration
 Configure an external database? (y/n) [No]:
 ##If you answer “No”, Unravel will use an Unravel managed database.
 ## If you answer “Yes”, you are further prompted for database information and test connectivity to that database.
 ##If you have answered “No” for the external database, you are further prompted to confirm if you want to use MySQL or MariaDB.

 Will you be using an Unravel managed MySQL or MariaDB? (y/n) [No]: 
 ##If you answer “No”,  the PostgreSQL bundled with Unravel is used.
 ##If you answer “Yes”, the “Extra packages” section searches for the required MySQL/MariaDB server tarball.

 Will Unravel connect to a MySQL or MariaDB database (ex: hive metastore) ? (y/n) [No]: 
 ##If you answer “Yes”, the “Extra packages” section searches for the required jdbc drivers.

The Extra packages check shows if you use Unravel managed MySQL/MariaDB or need JDBC drivers. Else, this check is automatically skipped.

-- Extra package location

*** JDBC drivers are required for Unravel managed MySQL or MariaDB.
*** Database software package is required for Unravel managed MySQL or MariaDB.

External package location [None]: /<my-extra-packages>
##This is the path to the directory where the required packages are located.

If the required packages are located, then the following message is shown:

The following packages will be installed:
   Database server: /my-extra-packages/mysql-5.7.27-linux-glibc2.12-x86_64.tar.gz
   JDBC driver:
     - /my-extra-packages/mysql-connector-java-5.1.48.tar.gz

 External package: Ok

If the required packages are not found, then the following error message is showing:

External package: ERROR
 - ERROR: Couldn't find jdbc drivers in /my-extra-packages
 - ERROR: Looked for:
   mysql-connector-java-*.tar.gz
   mysql-connector-java-*.jar
   mariadb-java-client-*.jar
 - ERROR: Couldn't find database server package in /my-extra-packages
 - ERROR: Looked for:
   mysql-*-linux-glibc2.12-x86_64.tar.gz
   mariadb-*-linux-x86_64.tar.gz

This check allows you to configure and test HTTPS for the unravel UI. This check prompts you for the certificate, key, password, and hostname details that are used to access Unravel.

Use HTTPS to access unravel? (y/n) [Yes]:  
##If you answer “Yes”, you are prompted for the path to the certificate and key. Unravel uses this information to configure TLS during installation. 
##If you answer “No”, you are shown a warning message for confirmation.

The information provided is verified for the following:

If the Key and Certificate match
If the certificate is valid
If the certificate applies for the provided hostname

This check allows you to set the Unravel UI port and verify the connectivity.

-- Unravel default port
  
  Port number (integer) [3000]: 

  Do you want to test if the port is accessible? (y/n) [Yes]: 
    
  This will open port 3000 and listen for connection for 120 seconds.
  Use your browser to test if the Unravel UI will be accessible on that port.
    
  We have detected the following hostnames:
  - some.host.example
    
 Browse to: http://some.host.example:3000
    
 ATTENTION: This address is an example. You should test with the URL that will be used to access Unravel.

A connection on port 3000 is tried and established. If the connection is successful, Unravel Port Test: OK is shown on the browser, and Unravel port: Reached is shown on the server.

This check allows you to set a custom data directory and verify the access if the directories exist. You will always find the software location where you deploy the Unravel binaries. In this check, only the space and access are tested. That data location that you have configured will be used.

-- Unravel directories

  Software [/opt/unravel]: 

  Data [/opt/unravel/data]: 

  Directories: ERROR
  - OK: 33 GB of free disk space for software.
  - ERROR: SYSH0026: Space for data 33 GB is low, recommended minimum is 100 GB.

Note

For more information, refer to Using the Interactive Precheck utility.

The responses that you have provided for the configuration information are used to generate a configuration file. You can use this configuration file when you run the setup command to install and configure Unravel.

After you have completed the responses, you are prompted to confirm if you want to generate the bootstrap configuration file. Press ENTER if you want to generate the bootstrap configuration file.
```
-- Unravel bootstrap configuration

   Generate a unravel bootstrap configuration file? (y/n) [Yes]:
```
The bootstrap configuration file is generated and located at /tmp/unravel-interactive-precheck/unravel-bootstrap.yaml.

Install Unravel with the bootstrap configuration file.

<unravel_installation_directory>/unravel/versions/<Unravel version>/setup --bootstrap /tmp/unravel-interactive-precheck/unravel-bootstrap.yaml

Install Unravel manually

To install Unravel manually, do the following:

1. Run setup

You can run the setup command to install Unravel.

About the setup command

The setup command allows you to do the following:

Runs Precheck automatically to detect possible issues that prevent a successful installation. Suggestions are provided to resolve issues. Refer to Precheck filters for the expected value for each filter.
Let you run extra parameters to integrate the database of your choice.
The setup command allows you to use a managed database shipped with Unravel or an external database. When you run the setup command run without any additional parameters, the Unravel managed PostgreSQL database is used. Otherwise, you can specify any of the following databases, which is supported by Unravel, with the setup command:
- MySQL (Unravel managed as well as external MySQL database)
- MariaDB (Unravel managed as well as external MariaDB database)
- PostgreSQL (External PostgreSQL)
Refer to Integrate database for details.
Let you specify a separate path for the data directory other than the default path.
The Unravel data and configurations are located in the data directory. By default, the installer maintains the data directory under <Unravel installation directory>/data. You can also change the data directory's default location by running additional parameters with the setup command. To install Unravel with the setup command.
Provides more options for setup.

To install Unravel with the setup command, do the following:

After deploying the binaries, if you are the root user, switch to Unravel user.
```
  su - <unravel user>
```
Notice
Only the Unravel user who owns the installation directory should run the setup command to install Unravel.

Run setup command with any of the following databases (PostgreSQL, MySQL, MariaDB). Refer to setup options for all the additional parameters that you can run with the setup command.

Tip

Run --help with the setup command and any combination of the setup command for complete usage details.

<unravel_installation_directory>/unravel/versions/<Unravel version>/setup --help

Refer to Integrate database topic and complete the prerequisites before running the setup command with any other database other than Unravel managed PostgreSQL, which is shipped with the product. Extra parameters must be passed with the setup command when you use another database.

Optionally, if you want to provide a different data directory, you can pass an extra parameter (--data-directory) with the setup command as shown below:

<unravel_installation_directory>/unravel/versions/<Unravel version>/setup --enable-databricks --data-directory /the/data/directory

Similarly, you can configure separate directories for other unravel directories—contact support for assistance.

Unravel managed PostgreSQL
```
<unravel_installation_directory>/unravel/versions/<Unravel version>/setup --enable-databricks
```
Notice
If you are using Unravel managed PostgreSQL database, and the Hive metastore is using MySQL, refer Set up Unravel Managed PostgreSQL for Hive metastore with MySQL

External PostgreSQL

<unravel_installation_directory>/unravel/versions/<Unravel version>/setup --enable-databricks --external-database postgresql <HOST> <PORT> <SCHEMA> <USERNAME> <PASSWORD>/

##The HOST, PORT, SCHEMA, USERNAME, PASSWORD are optional fields and are prompted if missing.

##For example:
/opt/unravel/versions/abcd.992/setup --enable-databricks --external-database postgresql xyz.unraveldata.com 5432 unravel_db_prod unravel unraveldata

Unravel managed MySQL

<unravel_installation_directory>/unravel/versions/<Unravel version>/setup --enable-databricks --extra /tmp/mysql

External MySQL

<unravel_installation_directory>/unravel/versions/<Unravel version>/setup --enable-databricks --extra /tmp/<MySQL-directory> --external-database mysql <HOST> <PORT> <SCHEMA> <USERNAME> <PASSWORD>

##The HOST, PORT, SCHEMA, USERNAME, PASSWORD are optional fields and are prompted if missing.

Unravel managed MariaDB

<unravel_installation_directory>/unravel/versions/<Unravel version>/setup --enable-databricks  --extra /tmp/mariadb

External MariaDB

<unravel_installation_directory>unravel/versions/<Unravel version>/setup --enable-databricks  --extra /tmp/<MariaDB-directory> --external-database mariadb <HOST> <PORT> <SCHEMA> <USERNAME> <PASSWORD>

##The HOST, PORT, SCHEMA, USERNAME, PASSWORD are optional fields and are prompted if missing.

When you run the setup command, the Precheck utility, which identifies the issues that prevent a successful installation, is automatically run. Refer to Precheck filters list to view details of each item in the precheck run output.

The Precheck output displays the issues that prevent a successful installation and also provides suggestions to resolve them. You must resolve each of the issues before proceeding. After the prechecks are resolved, you must re-login or reload the shell to execute the setup command again.

Note

In certain situations, you can skip the precheck using the setup --skip-precheck.

For example:

/opt/unravel/versions/<Unravel version>/setup --cluster-access abc1011.p2g.net.eu.xyz --skip-precheck

You can also skip the checks that you know can fail. For example, if you want to skip the Check limits option and check_network_ports, run the setup command as follows:

setup --filter-precheck ~check_limits,~check_network_ports

Filters	Description	Expected Value
System
Check uptime	Verifies the period since the last server reboot.	>24h
Clock sync	Verifies if the clock synchronization service is running on the server.	The clock synchronization service is up and running.
CPU requirement	Verifies if the server has enough CPUs to run Unravel efficiently.	Check requirements.
Memory requirement	Verifies that the server has enough memory to run Unravel efficiently.	Check requirements.
Disk access	Verifies that the user who runs unravel has access to the configured disk locations.	Unravel users can access the configured disk locations.
Disk Freespace	Verifies if the disk locations have enough free space.	Check requirements
Kerberos tools	Verifies that the Kerberos tools are available on the server to support kerberized environments.	Kerberos tools are installed.
Network ports	Verifies that the network ports used by Unravel are available.	Check requirements
OS libraries	Verifies the required libraries if run with Unravel managed MySQL.	The following packages must be installed for fulfilling the OS level requirements for MySQL: `numactl-libs` (for libnuma.so) `libaio` (for libaio.so)
OS release	Verifies that the OS distribution is supported.	Check compatibility matrix
OS settings	Verifies vm.max_map_count recommended.	Check requirements
SELinux	Verifies if the SELinux status is enabled or not and provides in which mode it is(Permissive, Disabled, Enforcing).	Check product documentation.
Check limits	Verifies that user limits are set to values	Check requirements
Healthcheck report bundle	Healthcheck report tarball. This report provides the summary and information gathered by the healthcheck with the location.

setup Options	Description
-h, --help	Shows help for setup.
--config CONFIG	Specify a different path to the configuration file. <unravel_installation_directory>/unravel/versions/`<Unravel version>`/setup --config `path/to/config/directory`
--enable-core	Enables core node support for non-Hadoop clusters.
--cluster-access (Edge node parameter)	Enables cluster access to the core node in a multi-cluster environment.
--data-forwarder host:port cluster-type cluster-id	Data forwarder, main unravel node.
--data-directory	Specify a different path to the data directory.
--external-database [param [param ...]]	Enable external database.
--external-database-ssl	Enable external database with SSL.
--log-file	Setup log file location. Default is `/tmp/unravel-setup-YYYYMMDD-HHMMSS.log`.
--extra DIR, -e DIR	Specify extra packages location.
--precheck	Run the preinstallation check.

Following is a sample of the setup command run result:

/opt/unravel/versions/abcd.1004/setup 
2021-04-05 15:51:30 Sending logs to: /tmp/unravel-setup-20210405-155130.log
2021-04-05 15:51:30 Running preinstallation check...
2021-04-05 15:51:31 Gathering information ................. Ok
2021-04-05 15:51:51 Running checks .................. Ok
--------------------------------------------------------------------------------
system
 Check limits        : PASSED
 Clock sync          : PASSED
 CPU requirement     : PASSED, Available cores: 8 cores
 Disk access         : PASSED, /opt/unravel/versions/abcd.1004/healthcheck/healthcheck/plugins/system is writable
 Disk freespace      : PASSED, 229 GB of free disk space is available for precheck dir.
 Kerberos tools      : PASSED
 Memory requirement  : PASSED, Available memory: 79 GB
 Network ports       : PASSED
 OS libraries        : PASSED
 OS release          : PASSED, OS release version: centos 7.6
 OS settings         : PASSED
 SELinux             : PASSED
--------------------------------------------------------------------------------
hadoop
 Clients            : PASSED
                 - Found hadoop
                 - Found hdfs
                 - Found yarn
                 - Found hive
                 - Found beeline
 Distribution         : PASSED, found CDH 6.3.3
 RM HA Enabled/Disabled    : PASSED, Disabled
Healthcheck report bundle: /tmp/healthcheck-20210405155130-xyz.unraveldata.com.tar.gz
2021-04-05 15:51:53 Prepare to install with: /opt/unravel/versions/abcd.1004/installer/installer/../installer/conf/presets/default.yaml
2021-04-05 15:51:57 Sending logs to: /opt/unravel/logs/setup.log
2021-04-05 15:51:57 Instantiating templates ................................................................................................................................................................................................................................ Ok
2021-04-05 15:52:05 Creating parcels .................................... Ok
2021-04-05 15:52:20 Installing sensors file ............................ Ok
2021-04-05 15:52:20 Installing pgsql connector ... Ok
2021-04-05 15:52:22 Starting service monitor ... Ok
2021-04-05 15:52:27 Request start for elasticsearch_1 .... Ok
2021-04-05 15:52:27 Waiting for elasticsearch_1 for 120 sec ......... Ok
2021-04-05 15:52:35 Request start for zookeeper .... Ok
2021-04-05 15:52:35 Request start for kafka .... Ok
2021-04-05 15:52:35 Waiting for kafka for 120 sec ...... Ok
2021-04-05 15:52:37 Waiting for kafka to be alive for 120 sec ..... Ok
2021-04-05 15:52:42 Initializing pgsql ... Ok
2021-04-05 15:52:46 Request start for pgsql .... Ok
2021-04-05 15:52:46 Waiting for pgsql for 120 sec ..... Ok
2021-04-05 15:52:47 Creating database schema ................. Ok
2021-04-05 15:52:50 Generating hashes .... Ok
2021-04-05 15:52:52 Loading elasticsearch templates ............ Ok
2021-04-05 15:52:55 Creating kafka topics .................... Ok
2021-04-05 15:53:36 Creating schema objects ....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Ok
2021-04-05 15:54:03 Request stop ....................................................... Ok
2021-04-05 15:54:16 Done
[unravel@xyz ~]$

Review and update Unravel Log Receiver (LR) endpoint. By default, this is set to local FQDN only visible to workspaces within the same network. If this is not the case, run the following to set the LR endpoint:
```
<unravel_installation_directory>/unravel/manager config databricks set-lr-endpoint <hostname> ''

## For example: 
/opt/unravel/manager config databricks set-lr-endpoint <hostname> ''
```
After you run this command, you are prompted to specify the port number. Ensure to press ENTER and leave it empty.

Apply the changes.

<Unravel installation directory>/unravel/manager config apply
<Unravel installation directory>/unravel/manager refresh databricks

Start all the services.

<unravel_installation_directory>/unravel/manager start

Check the status of services.
```
<unravel_installation_directory>/unravel/manager report 
```
The following service statuses are reported:
- OK: Service is up and running.
- Not Monitored: Service is not running. (Has stopped or has failed to start)
- Initializing: Services are starting up.
- Does not exist: The process unexpectedly disappeared. A restart will be attempted ten times.
You can also get the status and information for a specific service. Run the manager report command as follows:
```
<unravel_installation_directory>/unravel/manager report <service> 
## For example: /opt/unravel/manager report auto_action
```

2. Connect Databricks cluster to Unravel

Run the following steps to connect the Databricks cluster to Unravel.

Click Add Workspace and enter the following details.

Field	Description
Workspace Id	Databricks workspace ID, which can be found in the Databricks URL.
Workspace Name	Databricks workspace name, which can be found in the Databricks URL.
Instance (Region) URL	Regional URL where the Databricks workspace is deployed.
Tier	Select a subscription option: Standard or Premium.
Token	Personal access token to access Databricks REST APIs. Refer to Authentication using Databricks personal access tokens to create personal access tokens. Note Personal access tokens can be created with admin or non-admin roles.

Note

After you click the Add button, it will take around 2-3 minutes to register the Databricks Workspace with Unravel.

Add Unravel configuration to Databricks clusters using any of the following options:
- global init script
  Global init script applies the Unravel configurations at Workspace level. To set up global init scripts, open this file and follow the instructions.
- cluster init script
  Cluster init script applies the Unravel configurations at cluster level. To setup cluster init scripts from the cluster UI, do the following:
  1. Go to Unravel UI, click Manage > Workspaces > Cluster configuration to get the configuration details.
  2. Follow the instructions and update each cluster (Automated /Interactive) that you want to monitor with Unravel.
    Add Unravel configuration to Databricks clusters. Go to Unravel UI and from the upper right click Manage > Workspaces > Cluster configuration to get the configuration details. Follow the instructions and update every cluster (Automated /Interactive) in your workspace.
    Tip
    By default, the Ganglia metrics are enabled with Dcom.unraveldata.agent.metrics.ganglia_enabled property set to true.
    Note
    To add Unravel configurations to job clusters via API, refer How to setup cluster init scripts via cluster API.
Set additional configurations if required.

Configure the Workspace for Data page.

Ensure that at least one of the workspaces is populated before you configure a workspace for the Data page.

To configure the Databricks for Data page, do the following:

Stop Unravel

<Unravel installation directory>/unravel/manager stop

Set the following property.

<Unravel installation directory>/unravel/manager config properties set hive.metastore.<X>.workspace.ids <Comma-separated list of Databricks workspaces>

##Replace <X> with  the metastore variables listed in com.unraveldata.hive.metastore.list.

Apply the changes.

<Unravel installation directory>/unravel/manager config apply

Start Unravel

<Unravel installation directory>/unravel/manager start

Optionally, you can run healthcheck, at this point, to verify that all the configurations and services are running successfully.
```
<unravel_installation_directory>/unravel/manager healthcheck
```
Healthcheck is run automatically on an hourly basis in the backend. You can set the healthcheck intervals and email alerts to receive the healthcheck reports.

Tip

The workspace setup can be done at any time and does not impact the running clusters or jobs.

Validate Databricks workspace setup

Refer to Databricks FAQ.

In this section:

Home

Microsoft Azure Databricks

1. Create Unravel VM and Azure Databricks

Caution

2. Download Unravel

Important

3. Deploy Unravel

Important

4. Install Unravel

Note

Note

Notice

Tip

Notice

Note

Note

Note

Tip

Note

Tip

Search results