Configuring multiple hosts for Unravel Server

Home

Configuring multiple hosts for Unravel Server

This topic explains how to configure multiple hosts for Unravel Server. This is useful for performance or reliability requirements.

When you complete the steps below, the expected result is

Multiple Unravel hosts work together in an ensemble.
Each host has a unique role, and is identified by a daemon named unravel_xyz or unravel_xyz_n (where n is 1, 2, 3,...), which runs on exactly one host.
Unravel Web UI (unravel_tc) runs on host1.
Port 4043 log_receiver runs on host2.
If you do not use an external database (db), unravel_db runs on host1. However, unravel_db can also run harmlessly if an external db is used.
/usr/local/unravel/etc/unravel.properties is identical on all Unravel hosts in the ensemble.
Note
The file unravel.properties is never changed by an RPM upgrade because it contains site-specific information, but some one-time changes occur when setting up multiple hosts. It must be identical on all hosts, so if you are using data center automation like Puppet, Chef, Salt, Ansible, CloudFormation, Cfengine, and so on, you must maintain one "golden" unravel.properties per Unravel ensemble. After you modify it as described below, you must update the "golden" unravel.properties file for your site.
Daemons are enabled/disabled via chkconfig. chkconfig sets symbolic links which are persistent across Unravel Server RPM upgrades.

Prerequisites

If you use Kerberos, set that up first on host1.
Install the same Unravel RPM on one or two additional hosts, hereafter referred to as host2 and host3. This distinction is important because certain instructions only apply to specific hosts.

Note

Each host is assigned unique roles identified by daemon names that start with unravel_.

Expected result

Stop Unravel Server.
On each Unravel host, run this command:
```
# sudo /etc/init.d/unravel_all.sh stop
```
Modify unravel.properties on host1.
Pick a machine to be host1, where the Unravel Web UI will run. If the bundled db is in use, edit /usr/local/unravel/etc/unravel.properties on host1 to change:
Note
UNRAVEL_HOST_IP must be a fully qualified DNS or IP address. Replace 3316 with your port number and unravel_mysql_prod with your database name.
To find your fully qualified hostname, enter.
Replace all text with your local variables.
```
# hostname -I 
```
```
unravel.jdbc.url=jdbc:mysql://unravel-host-ip:3316/unravel-mysql-prod
```
Copy host1's unravel.properties to other hosts.
Copy /usr/local/unravel/etc/unravel.properties, /usr/local/unravel/etc/unravel.ext.sh, and /etc/unravel_ctl (if present) from host1 to host2 (and host3, if you are using three hosts). For example on host1:
```
# scp /usr/local/unravel/etc/unravel.properties host2:/usr/local/unravel/etc/
# scp /usr/local/unravel/etc/unravel.ext.sh host2:/usr/local/unravel/etc/
# scp /etc/unravel_ctl host2:/etc/
```
Verify that the ownership of unravel.properties and unravel.ext.sh is unravel:unravel. If /etc/unravel_ctl is used, it should be owned by root:root.
Important
The scripts invoked below will make an identical change to the unravel.properties file on each machine.
Assign roles.
Use these scripts to assign unique roles to the hosts. To reduce the chance of errors, the command line arguments are the same on each host, but notice that the script name is different. The arguments are the hostnames IP addresses of the hosts in the ensemble.
These scripts establish the roles each host plays in the ensemble. The main effect is to assign specific Unravel logical daemons to one host and only one host.
Note
Note that some daemons have names like unravel_xyz_1 or unravel_xyz_2, and so on. The entire name with the instance numeric suffix is set to run on one host in the ensemble. In some cases, multiple xyz numeric instances run on one machine, but the overall name with suffix runs on one host only.
The switch_to_* scripts change unravel.properties in a coordinated fashion and also create a file, unravel.id.properties, to hold the integer property indicating which role the particular machine has (1, 2, or 3).
- For a 2-host ensemble (substitute host):
  on host1
```
# sudo /usr/local/unravel/install_bin/switch_to_1of2.sh host1 host2
```
  on host2
```
# sudo /usr/local/unravel/install_bin/switch_to_2of2.sh host1 host2
```
- For a 3-host ensemble (substitute host):
  on host1
```
# sudo /usr/local/unravel/install_bin/switch_to_1of3.sh host1 host2 host3
```
  on host2
```
# sudo /usr/local/unravel/install_bin/switch_to_2of3.sh host1 host2 host3
```
  on host3
```
#sudo /usr/local/unravel/install_bin/switch_to_3of3.sh host1 host2 host3
```
Set-up Zookeeper and Kafka.
1. Assign Kafka Partitions
  Kafka partition assignment (for 3 host installs) is done by evenly distributing a topic over the hosts that exist at topic create time. Topics must be created anew when a new host is added in order to have proper distribution.
2. Redistribute Zookeeper Topics
  Perform these steps, in sequential order on the specific hosts as indicated by the prompt name. Skip host3 if you are only using 2 hosts for Unravel Server:
  1. Stop all and clear Zookeeper and Kafka data areas on each host:
    on host1
```
# sudo /usr/local/unravel/install_bin/kafka_clear.sh
```
    on host2
```
# sudo /usr/local/unravel/install_bin/kafka_clear.sh
```
    on host3
```
# sudo /usr/local/unravel/install_bin/kafka_clear.sh
```
  2. Start up Zookeeper ensemble:
    on host1
```
# sudo /etc/init.d/unravel_all.sh start-zk
```
    on host2
```
# sudo /etc/init.d/unravel_all.sh start-zk 
```
    on host3
```
# sudo /etc/init.d/unravel_all.sh start-zk
```
  3. Wait 15 seconds for Zookeeper quorum to settle.
```
# sleep 15
```
  4. Start up Kafka ensemble:
    on host1
```
# sudo /etc/init.d/unravel_all.sh start-k
```
    on host2
```
# sudo /etc/init.d/unravel_all.sh start-k 
```
    on host3
```
# sudo /etc/init.d/unravel_all.sh start-k 
```
  5. Wait 10 seconds for Kafka coordination:.
```
# sleep 10
```
  6. Create the Kafka topics (only on one host):
    on host1
```
# sudo /usr/local/unravel/install_bin/kafka_create_topics.sh
```

Start the Unravel Server.

Finish multi-host installation by starting up Unravel Server:

on host1

# sudo /etc/init.d/unravel_all.sh start 
echo "http://unravel-host-ip:3000/"

on host2

# sudo /etc/init.d/unravel_all.sh start

Edit Hive-site Snippet for Hive-Hook.
The port 4043 is on host2 and that means the hive-site.xml file changes needed for Unravel hive-hook are in /usr/local/unravel/hive-hook/hive-site.xml.snip on host2. If hive-site.xml was already configured for host1, then modify it for host2.
Snapshot unravel.properties as new golden file.

In this section:

Would you like to provide feedback? Just click here to suggest edits.

Home

Configuring multiple hosts for Unravel Server

Note

Prerequisites

Note

Expected result

Note

Important

Note

Search results