Sign up for Unravel SaaS free trial
You can start a free trial of Unravel monitoring via SaaS with the following steps:
Sign-up for Unravel free trials
Go to https://unraveldata.com/saas-free-trial and enter the following details:
First Name
Last Name
Company Name
Company Email
Trial Type: Amazon EMR or Databricks
Click START FREE TRIAL. You will receive a mail for email verification.
After your email is verified, you will receive a welcome email with the following details:
Unravel URL from where you can log in.
Login credentials.
Steps for instrumenting Amazon EMR/Databricks cluster with instructions.
Following is a sample of the welcome email for reference:
EMR cluster - Welcome email
Databricks cluster - Welcome email
Click the Unravel service link in the email and login to Unravel using the default username and password provided in the email.
Connect your Amazon Elastic MapReduce (EMR) cluster or Databricks cluster to Unravel SaaS.
This section provides instructions to connect an Amazon EMR cluster to Unravel SaaS.
On the AWS console, select the EMR service and click Create cluster.
In the Create Cluster - Quick Options screen, click Go to advanced options.
In Step 1: Software and Steps, select any release up to emr-5.29.0.
In Step 2: Hardware, enter a configuration for your EMR cluster and click Next.
In Step 3: General Cluster Settings, specify the following settings in Add Bootstrap Action and click Add. The Amazon EMR cluster will start with this bootstrap action.
Setting
Action
Name
Select Custom action.
Script location
Specify the following bootstrap location:
s3://unravel-saas-bootstrap/unravel_emr_bootstrap.py
Note
If you want to monitor MR jobs, then you must pass additional optional argument --all
Optional arguments
Enter the following:
--unravel-server
ENTER-YOUR-UNRAVEL-INSTANCE-HOSTNAME-HERE
Click Configure and add.
In Step 4: Security, edit the configuration for the cluster as required. For example:
Choose the EC2 key pair.
Select the EC2 security groups. AWS EMR service automatically applies additional rules that are required for EMR nodes.
Click Create cluster. Your new EMR cluster finishes the bootstrap process and will be in the Waiting state.
This section provides instructions to connect a Databricks cluster to Unravel SaaS.
Create a Workspace token in Databricks.
Go to Workspace > Admin Console > Access Control and enable Personal Access Tokens. For more details, refer to Manage personal access tokens.
Go to Workspace > User Settings > Access Tokens and click Generate New Token. For more details, refer to Authentication using Databricks personal access tokens.
Register a new Databricks workspace or edit details of an existing Databricks workspace.
Go to the Unravel Instance UI and click Workspaces. The Workspaces Manager page is displayed.
In STEP-1 : Configure Unravel with Databricks Workspaces, click Add Workspace and enter the following details.
Field
Description
Workspace Id
Databricks workspace ID.
Workspace Name
Databricks workspace name.
Instance (Region) URL
Regional URL where the Databricks workspace is deployed.
Tier
Select a subscription option: Standard or Premium.
Token
Personal access token to authenticate to and access Databricks REST APIs. Refer to Authentication using Databricks personal access tokens to create personal access tokens.
Note
After you click the Add button, it will take around 2-3 minutes to register the Databricks Workspace with Unravel.
Configure Databricks cluster with Unravel.
On the Unravel Instance UI, click Workspaces and follow the instructions in the Workspaces Manager > STEP-2 to configure the Databricks cluster with Unravel.
Based on these instructions, go to Configure Cluster > Advanced Options and update the following configurations. These configurations must be updated for every cluster (Automated /Interactive) in your workspace.
Spark
Copy the following snippet to Spark > Spark Conf. Replace
Unravel DNS or IP Address
.spark.eventLog.enabled true spark.eventLog.dir dbfs:/databricks/unravel/eventLogs/ spark.unravel.server.hostport
<Unravel DNS or IP Address>
:4043 spark.unravel.shutdown.delay.ms 300 spark.executor.extraJavaOptions -Dcom.unraveldata.client.rest.request.timeout.ms=1000 -Dcom.unraveldata.client.rest.conn.timeout.ms=1000 -javaagent:/dbfs/databricks/unravel/unravel-agent-pack-bin/btrace-agent.jar=config=executor,libs=spark-2.3 spark.driver.extraJavaOptions -Dcom.unraveldata.client.rest.request.timeout.ms=1000 -Dcom.unraveldata.client.rest.conn.timeout.ms=1000 -javaagent:/dbfs/databricks/unravel/unravel-agent-pack-bin/btrace-agent.jar=config=driver,script=StreamingProbe.btclass,libs=spark-2.3Logging
Select DBFS as Destination, and maintain the default Cluster Log Path.
dbfs:/cluster-logs/
Init Scripts
In the Init Scripts tab, select Destination as DBFS. Copy the following as the Init script path and click Add.
dbfs:/databricks/unravel/unravel-db-sensor-archive/dbin/install-unravel.sh
Monitor clusters with Unravel
After your cluster is up and running. You can run some jobs, log on to Unravel, and monitor your applications.
Unravel sends an Insight email daily that provides details about your cluster activities.
Free trial expiry
The free trial expires after 14 days. After the free trial has expired, you will receive a trial expiry email. You can extend the trial period by contacting trials@unraveldata.com
.