Skip to main content

Home

Configuring Unravel to monitor multiple clusters

Multi-cluster support allows you to see all your clusters through a "single pane of glass". You need only manage one Unravel instance to connect to and analyze multiple clusters. For example, you can use one Unravel instance to monitor Test, UAT (User Acceptance Test), and Production clusters.

Note

Multi-cluster support is only available for fresh installs.

Basic Configuration
arch-config.png
Unravel core to Unravel edge node configuration to cluster

An Unravel Edge Node needs to be configured for each cluster you are monitoring.

MC-Configuration.png
Prerequisites

The Unravel edge node does not have to be a dedicated node. It is a lightweight installation with one child process.

See the installation instructions for your platform for the prerequisites of the Unravel core node.

Unravel core Installation
  1. Install Unravel v4.6.0.0 on your server for your particular platform.

  2. In /usr/local/unravel/etc/unravel.properties set com.unraveldata.multi-cluster.enable=true.

  3. Restart Unravel daemons.

    /etc/init.d/unravel_all.sh start
Installation for Unravel edge node

Important

For each edge node, you must perform the following steps.

  1. Create a directory for the unravel files.

    mkdir /usr/local/unravel
  2. cd a directory and create unravel.yaml.

    cd /usr/local/unravel
    vi /home/unravel/unravel.yaml
  3. Configure Unravel Core Node and the Unravel edge node.

    hosts:
      host_master:
        name: "Unravel Core Node"
        hostname: "core_node_fully_qualified_domain_name"
        roles:
          - core
          - mysql_server
      edge_#:
        name: "Unravel Edge Node #"
        hostname: "edge_node_fully_qualified_domain_name"
        roles:
          - cluster_access
  4. Change the owner of unravel.yaml to unravel.

    chown unravel /home/unravel/unravel.yaml
  5. Download the package and unpack it in the directory you created.

    curl http://caballo.unraveldata.com:8081/artifactory/dist-internal/unravel/release/4.6.0.0/unravel-4.6.0.0.2.tar.gz | tar xz -C /usr/local
  6. Change the owner and group of /usr/local/unravel and its contents to unravel.

  7. Run the setup script.

    Important

    The setup script must be run as $unravel-user.

    chown -R unravel:unravel /usr/local/unravel
    /usr/local/unravel/versions/4.6.0.0.2/setup.sh --config=/home/unravel/unravel.yaml
    
    2019-12-05 14:25:20 Instanciating templates ...................................................................... Ok

    If run the script as root or another user the script exits. For example,

    [root@unravelEdgeCluster]# /usr/local/unravel/versions/4.6.0.0.2/setup.sh --config=/home/unravel/unravel.yaml
    
    2019-12-11 14:24:57 FATAL *** /usr/local/unravel/versions/4.6.0.0.2/setup.sh was run as root. This is not supported. Exiting
  8. Run manager.sh to configure Unravel. This script sets up the cluster access and lr proxy related cfgs.

    /usr/local/unravel/services/manager.sh configA
  9. Review the /usr/local/unravel/data/conf/unravel.properties. The following properties should be configured as below

    cluster_access.host=unravel edge node
    cluster_access.port=4120
    cluster_access.external.host=unravel edge node
    cluster_access.external.port=4120
    sregistry.mode=proxy
    com.unraveldata.lrproxy=true
    remote_lr.host=unravel core node
    remote_lr.port=4043
  10. Start Unravel on the edge node.

    /usr/local/unravel/services/manager.sh start
  11. Configure the edge node Unravel properties. You must set the following properties

    1. Set these properties as below.

      // set the host port
      unravel.server.hostport=unravel edge node
      
      // Configure Spark event logs, mapreduce done directory, yarn aggregated logs directory  cluster type cfgs 
      com.unraveldata.cluster.type=CDH
      com.unraveldata.job.collector.done.log.base=/user/history/done
      com.unraveldata.spark.eventlog.location=hdfs:///user/spark/applicationHistory
      com.unraveldata.job.collector.log.aggregation.base=/tmp/logs/*/logs/
      
    2. If you are running Impala you set the following properties.

      com.unraveldata.cloudera.manager.url=http://cm-host:port
      com.unraveldata.cloudera.manager.username=userName
      com.unraveldata.cloudera.manager.password=PassWord
    3. See basic log properties for all the log properties you need to set.