Skip to main content

Home

Microsoft Azure Databricks

Before installing Unravel in Azure Databricks, check and ensure that the Unravel installation requirements are completed and follow the below instructions to install and configure Unravel:

1. Create Unravel VM and Azure Databricks resource

2. Download Unravel

3. Deploy Unravel binaries

4. Install Unravel either with Interactive Precheck or manually

5. Configure Unravel Log Receiver

6. Connect Databricks cluster to Unravel

1. Create Unravel VM and Azure Databricks resource
  1. Sign in to the Azure portal.

  2. Select Virtual Machines > Add and enter the following information in the Basics tab:

    Project Details

    Subscription

    Choose the applicable subscription.

    Resource group

    Create a new resource group or choose an existing one.

    Instance Details

    Virtual Machine Name:

    The Unravel server name.

    Region:

    Select the Azure region.

    Availability Options

    Select No infrastructure redundancy required.

    Image

    Select the appropriate image. Both Centos-based 7.x+ and Red Hat Enterprise Linux 7.x+ are supported.

    Size

    Click Change Size. In the modal, select Memory-optimized image with at least 128 GB memory and Premium Disk support, for example, E16s_v3 in East US 2)

    Administrator account

    Authentication type

    Select password or SSH Key.

    Username and Password

    Enter your VM login information.

    Inbound Port Rules

    Public inbound ports

    Select Allow selected ports.

    Selected Inbound ports

    Select both HTTPS and SSH.

  3. Click Next: Disks > and enter the following information in the Disks tab.

    Disk Options

    • OS disk type: Select Premium SSD.

    Data Disk

    • Click Create and attach a new disk.

      Caution

      This disk is formatted, so do not choose the Attach an existing disk option.

      • Enter a Name.

      • Select Source typeNone (empty disk).

      • Set Size to at least 512 GiB.

      • Account type: Select premium SSD.

  4. Click Next: Networking > and enter the following information:

    • Virtual network: Create a new one or choose an existing one.

    • Subnet: Create a new or choose an existing one.

    • Public IP: Create a new one or choose an existing one.

    • Select Inbound ports: Select HTTPS and SSH.

  5. Click Review + create. Your deployment is now created.

  6. Select Go to Resource > Networking > Inbound port rules > Add inbound port rule and include the following ports.

    Name

    Destination

    Destination IP Address

    Destination Port Ranges

    Unravel_3000

    IP Addresses

    NIC Private IP

    3000

    Unravel_443

    IP Addresses

    NIC Private IP

    443

    Unravel_4043

    IP Addresses

    NIC Private IP

    4043

    Unravel_4443

    IP Addresses

    NIC Private IP

    4443

  7. Click OK.

  1. Select Create a resource > Azure Databricks > Create. Go directly to step#3 if you already have workspaces.

  2. Select Workspace name, Subscription, Resource group, Location, and Pricing tier.

  3. Review VNET Peering options to connect Databricks with Unravel VM.

2. Download Unravel
3. Deploy Unravel

Unravel binaries are available as a TAR file or RPM package. You can deploy the Unravel binaries in any directory on the server. However, the user who installs Unravel must have the write permissions to the directory where the Unravel binaries are deployed.

After you extract the contents of the TAR file or RPM package, unravel directory is created within the installation directory (<unravel_installation_directory>), and Unravel will be available in <Unravel_installation_directory>/unravel. The directory layout will be unravel/versions/<Directories and files>.

The following steps to deploy Unravel from a TAR file should be performed by a user, who will run Unravel.

  1. Create an Installation directory.

    mkdir </path/to/installation/directory>
    

    For example: mkdir /opt/

  2. Extract Unravel tar file to the installation directory, which you have created in the first step. After you extract the contents of the TAR file, unravel directory is created within the installation directory.

    tar zxf unravel-<version>tar.gz -C </path/to/installation/directory>
    

    For example: tar zxf unravel-4.7.x.x.tar.gz -C /opt

    The unravel directory will be available within /opt

  3. Grant ownership of the directory to a user who will run Unravel.

    chown -R username:groupname </path/to/installation/directory>
    

    For example: chown -R unravel:unravelgroup /opt/unravel

Important

The following steps to deploy Unravel from an RPM package should be performed by a root user. After the RPM package is deployed, the remaining installation procedures should be performed by the Unravel user.

  1. Create an installation directory.

    mkdir </path/to/installation/directory>
    

    For example: mkdir /usr/local/unravel

  2. Run the following command:

    rpm -i unravel-<version>.rpm
    

    For example: rpm -i unravel-4.7.x.x.rpm

    The unravel directory will be available in /usr/local

    If you want to provide a different location, use the --prefix command.

    For example:

    mkdir /opt/unravel rpm -i unravel-4.7.x.x.rpm --prefix /opt

    The unravel directory will be available in /opt.

  3. Grant ownership of the directory to a user who will run Unravel. This user executes all the processes involved in Unravel installation.

    chown -R username:groupname </path/to/installation/directory>
    

    For example:chown -R unravel:unravelgroup /usr/local/unravel

  4. Continue with the installation procedures as unravel user.

4. Install Unravel

You can install Unravel either with Interactive Precheck or manually without Interactive Precheck.

Note

Unravel recommends installation with Interactive Precheck.

To install Unravel with Interactive precheck, you must run the Interactive Precheck utility to generate a bootstrap configuration file for installation, configure the Unravel log receiver, and then connect the Databricks cluster to Unravel.

5. Configure Unravel Log Receiver
6. Connect Databricks cluster to Unravel

Run the following steps to connect the Databricks cluster to Unravel.