Home

Allocate a cluster gateway/edge/client host with HDFS access.

Enable the hadoop fs command, Hive, and Spark.

Although Unravel Server doesn't launch Hive or Spark jobs, it's convenient to have Hive and Spark installed on this gateway/edge/client host.

1. Run the following commands this gateway/edge/client host as root, substituting your site-specific values for name, cldb-list, and history-server.

   sudo yum install mapr-client.x86_64
   sudo /opt/mapr/server/configure.sh -N name -c -C cldb-list -HS history-server
   sudo yum install mapr-hive.noarch
   sudo yum install mapr-spark.noarch

Check/add/modify these MapR settings:

1. Run the following commands on Unravel Server as root:

   sudo useradd -g mapr unravel
   hadoop fs -mkdir /user/unravel
   hadoop fs -chown unravel:mapr /user/unravel

2. If MapR tickets are enabled, make sure you have tickets for users unravel and mapr on the target host.

   You might need to export ticket environment variables (such as MAPR_TICKETFILE_LOCATION) in /srv/unravel/unravel_ctl first.

3. Check/adjust available RAM on the Unravel gateway/client host:

   free -g

   For instructions on adjusting RAM allocated to MapR-FS (mfs), see https://community.mapr.com/docs/DOC-1209.

   For example, edit /opt/mapr/conf/warden.conf as follows:

   service.command.mfs.heapsize.maxpercent=10

4. Restart MapR-FS (mfs).

Download the Unravel Server RPM.

Ensure that the host machine's local disks have the minimum space required.

Unravel Server uses two separate disks: one for binaries (/usr/local/unravel) and one for data (/srv/unravel). The separate disk /srv/unravel is beneficial for performance. If either disk doesn't have the minimum space required, create symbolic links for them to another disk drive.

df -h /srv

Install the Unravel Server RPM.

sudo rpm -ivf unravel-version.rpm*

(Optional) Enable additional daemons for high-volume workloads.

In /usr/local/unravel/etc/unravel.properties, set general properties for Unravel Server.

Point Unravel Server to logs on HDFS.

Unravel collects HDFS logs for analysis. To point Unravel Server to these logs, set the following properties in /usr/local/unravel/etc/unravel.properties:

If the Application Timeline Server (ATS) requires user authentication, set the following properties:

Connect to the Oozie server by setting oozie.server.url.

Enable https access to Resource Manager by setting https.protocol.

Define the monitoring frequency.

If Kerberos is enabled, add authentication for HDFS:

1. Create or identify a principal and keytab for Unravel daemons to access HDFS and REST when Kerberos is enabled.

2. Find and verify the principal keytab by running this command:

   klist -kt KEYTAB_FILE

   Note

   Set the Linux file permissions of the keytab file to 500 (chmod 500) and set its owner set to unravel or to your chosen user, as explained in Run Unravel Daemons with Custom User.

3. In /usr/local/unravel/etc/unravel.properties, add/set these properties for Kerberos:

   com.unraveldata.kerberos.principal=unravel/my-host.my-domain@my-realm
   com.unraveldata.kerberos.keytab.path=/usr/local/unravel/etc/unravel.keytab

If Sentry is enabled, add these permissions:

1. Define your own alt principal with narrow privileges and the access permissions shown in the table below.

   The alt principal can be unravel (default) or one of your choosing. The corresponding kerberos principal does not need to be the same as the local user.

2. Verify that the user running the Unravel daemon /etc/unravel_ctl has the access permissions shown in the table below.

   Resource	Principal	Access	Purpose
   hdfs://user/spark/applicationHistory	mapr or alternate	read+execute	Spark event log
   hdfs://usr/history/done	mapr or alternate	read+execute	MapReduce logs
   hdfs://tmp/logs	mapr or alternate	read+execute	YARN aggregation folder
   hdfs://user/hive/warehouse	mapr or alternate	read+execute	Obtain table partition sizes
   Hive Metastore access	hive	read+execute	Hive table information

Find the hostname of Unravel Server.

echo "http://$(hostname -f):3000/"

If you're using an SSH tunnel or HTTP proxy, you might need to make adjustments.

Using a supported web browser, navigate to http://unravel-host:3000 and log in with username admin with password unraveldata.

Unravel UI displays collected data.