Upgrading Unravel from version 4.6.2.x to 4.7.x

Home

Upgrading Unravel from version 4.6.2.x to 4.7.x

This topic provides instructions to upgrade from Unravel version 4.6.2.x to 4.7.x.

Caution

If you have manually added certificates to unravel’s JVM, those will not be imported when you upgrade to Unravel 4.7x. Therefore, you must add these certificates again. Refer to Configuring Truststore certificates and follow the instructions to add the Truststore certificates.
If you have configured Unravel for HTTPS, those configurations will not be imported. Hence, you must re-configure this again. Refer to Enabling Transport Layer Security (TLS) for Unravel UI and follow the instructions to re-configure TLS.
For the CDH and HDP platforms, the Hive metastore password must be reset after you upgrade. Refer to Set the Hive metastore password instructions in the Hive Metastore topic.

Caution

Before upgrading Unravel, you must take a backup of the data directory and external database (if using). Perform the following steps:

Stop Unravel.

<Unravel installation directory>/unravel/manager stop

Archive the entire data directory.

For example,

tar czf /archive/destination/unravel-backup.tar.gz /opt/unravel/data

If using an external database, perform a backup of the database.
Note
For the details on how to back up the database, refer to the documentation for your corresponding database.

Upgrading Unravel in a single cluster deployment (4.6.2.x to 4.7.x)

Download and extract the Unravel binaries of the Unravel version that you want to upgrade/downgrade to.

Stop Unravel.

<Unravel installation directory>/unravel/manager stop

Activate the version that you want to upgrade/downgrade to.

<Unravel installation directory>/unravel/manager activate <Unravel-version>

Run auto-configuration and refresh to automatically reload the updates.

<unravel_installation_directory>/unravel/manager config auto --refresh

After the auto-configuration completes, apply the changes.

<unravel_installation_directory>/unravel/manager config apply

Start all the services.

<unravel_installation_directory>/unravel/manager start

Check the status of services.
```
<unravel_installation_directory>/unravel/manager report 
```
The following service statuses are reported:
- OK: Service is up and running.
- Not Monitored: Service is not running. (Has stopped or has failed to start)
- Initializing: Services are starting up.
- Does not exist: The process unexpectedly disappeared. Restarts will be attempted 10 times.
Delete the old install directory from unravel/versions/<THE.OLD.VERSION>.

Upgrading Unravel in multi-cluster deployment (4.6.2.x to 4.7.x)

To upgrade Unravel in a multi-cluster environment, you must upgrade all the edge nodes, upgrade the core node, and pull in all the edge node updates to the core node.

Edge nodes
Run the following steps to upgrade Unravel on the edge nodes involved in Unravel monitoring.
1. Download and extract the Unravel binaries of the Unravel version that you want to upgrade/downgrade to.
2. Stop Unravel.
```
<Unravel installation directory>/unravel/manager stop
```
3. Activate the version that you want to upgrade/downgrade to.
```
<Unravel installation directory>/unravel/manager activate <Unravel-version>
```
  Caution
  You may get a Network port failed precheck error if you run the manager activate command immediately after executing the manager stop command.
  To avoid this precheck error, stop Unravel and then ensure to wait for a minute or two before executing the manager activate command.
  Retry the manager activate command in case of the Network port failed precheck error.
4. Run auto-configuration and refresh to automatically reload the updates.
```
<unravel_installation_directory>/unravel/manager config auto --refresh
```
  After the auto-configuration completes, apply the changes.
```
<unravel_installation_directory>/unravel/manager config apply
```
5. Start all the services.
```
<unravel_installation_directory>/unravel/manager start 
```
6. Check the status of services.
```
<unravel_installation_directory>/unravel/manager report 
```
  The following service statuses are reported:
  - OK: Service is up and running.
  - Not Monitored: Service is not running. (Has stopped or has failed to start)
  - Initializing: Services are starting up.
  - Does not exist: The process unexpectedly disappeared. Restarts will be attempted 10 times.

Core node

Run the following steps only on the core node.

Download and extract the Unravel binaries of the Unravel version that you want to upgrade/downgrade to.

Stop Unravel.

<Unravel installation directory>/unravel/manager stop

Activate the version that you want to upgrade/downgrade to.

<Unravel installation directory>/unravel/manager activate <Unravel-version>

Optional: If you have run the config auto command previously on the cluster, then run auto-configuration on the core node and refresh to automatically reload the updates.
```
<unravel_installation_directory>/unravel/manager config auto --refresh
```
Caution
If you have NOT run the manager config auto earlier on the core node, an error will be displayed, which can be ignored.

Run the following command to get the list of edge clusters that are configured.

<Unravel installation directory>/unravel/manager config edge show

-- Running: config edge show
------------ | ---------------------------------------- | ------------
    EDGE KEY | - local-node                             | Enabled
             |     Cluster manager:                     | Enabled
             |     Clusters:                            | 
 CLUSTER KEY |       - HDP47314                         | Enabled
             |           - No services found            | 
------------ | ---------------------------------------- | ------------
-- OK

Remove the edge configuration that shows EDGE KEY as local-node.

<Unravel installation directory>/unravel/manager config edge remove <EDGE KEY>

Configure the required edge nodes. Refer to the instructions in Configuring edge nodes.

Apply the changes.

<unravel_installation_directory>/unravel/manager config apply

Start all the services.

<unravel_installation_directory>/unravel/manager start

Check the status of services.

<unravel_installation_directory>/unravel/manager report

Check the list of services, which are enabled for the edge node after running the auto-configurations.

<unravel_installation_directory>/unravel/manager config edge show

-- Running: config edge show
------------ | ---------------------------------------- | ------------
    EDGE KEY | - edge-a                                 | Enabled
             |     Cluster manager:                     | Enabled
             |     Clusters:                            | 
 CLUSTER KEY |       - Cluster_Name                     | Enabled
             |           HBASE:                         | 
 SERVICE KEY |             - hbase                      | Enabled
             |           HDFS:                          | 
 SERVICE KEY |             - hdfs                       | Enabled
             |           HIVE:                          | 
 SERVICE KEY |             - hive                       | Enabled
 SERVICE KEY |             - hive2                      | Enabled
             |           IMPALA:                        | 
 SERVICE KEY |             - impala                     | Enabled
 SERVICE KEY |             - impala2                    | Enabled
             |           KAFKA:                         | 
 SERVICE KEY |             - kafka                      | Enabled
 SERVICE KEY |             - kafka2                     | Enabled
             |           SPARK_ON_YARN:                 | 
 SERVICE KEY |             - spark_on_yarn              | Enabled
             |           YARN:                          | 
 SERVICE KEY |             - yarn                       | Enabled
             |           ZOOKEEPER:                     | 
 SERVICE KEY |             - zookeeper                  | Enabled
------------ | ---------------------------------------- | ------------
-- OK

You can disable any of the services. For example, you want to disable the HBase services:

<unravel_installation_directory>/unravel/manager config edge hbase disable <EDGE_KEY> <CLUSTER KEY> <SERVICE NAME>
Example: /opt/unravel/manager config edge hbase disable edge-a Cluster_Name hbase

Run healthcheck to verify that all the configurations and services are running successfully.
```
<unravel_installation_directory>/manager healthcheck
```
Healthcheck is run automatically, on an hourly basis, in the backend. You can set your email to receive the healthcheck reports.
Delete the old install directory from unravel/versions/<THE.OLD.VERSION>.

Configuring edge nodes

Obtain the Cluster Access ID, which must be provided when you add the edge nodes. Run the following command on the edge node:
```
<unravel_installation_directory>/unravel/manager support show cluster_access_id
```
Important
Ensure to repeat the following steps (2-7) on the core node for each of the edge nodes.
Add each of the edge nodes, involved with Unravel monitoring, to the core node.
Important
- EDGE_KEY is any label that you provide to identify the edge node. Preferably provide the hostname of the edge node for quick identification later.
```
<unravel_installation_directory>/unravel/manager config edge add <EDGE_KEY> <CLUSTER_ACCESS_ID>
##Example: /opt/unravel/manager config edge add edge-a 123-123-123-123
```
Notice
When you are adding edge nodes for CDH or CDP platforms and If you want to configure Migration or Forecasting reports for these clusters, then you must have one of the following roles: MigrationsForecasting
- Full Administrator
- Cluster Administrator
- Operator
- Configurator
These roles are required only in the case of the Cloudera manager.
Run auto-configuration.
Important
- EDGE_KEY is the label that you have configured in step 3 as <EDGE_KEY>.
```
<unravel_installation_directory>/unravel/manager config edge auto <EDGE_KEY> 
//For example: /opt/unravel/manager config edge auto edge-a 
```
When prompted, you can provide the following:
- Cluster manager URL: Provide the URL of the cluster manager. For example: https://abcd773.unraveldata.com:8443
- Username
- Password

The Hive metastore database password can be recovered automatically only for a cluster manager with an administrative account. Otherwise, it must be set manually as follows:

Run the manager config edge show command to get the <EDGE_KEY>, <HIVE_KEY>, and <CLUSTER_KEY>, which must be provided when you set the Hive metastore password.

<EDGE_KEY> is the label you provide to identify the edge node when you add the edge node in Step 3.
CLUSTER_KEY is the name of the cluster where you set the Hive configurations.
<HIVE_KEY> is the definition of the Hive service. In the output of the manager config edge show command, this is shown as the <SERVICE_KEY> for Hive.

-- Running: config edge show
------------ | ---------------------------------------- | ------------
    EDGE KEY | - edge-a                                 | Enabled
             |     Cluster manager:                     | Enabled
             |     Clusters:                            | 
 CLUSTER KEY |       - Cluster_Name                     | Enabled
             |           HBASE:                         | 
 SERVICE KEY |             - hbase                      | Enabled
             |           HDFS:                          | 
 SERVICE KEY |             - hdfs                       | Enabled
             |           HIVE:                          | 
 SERVICE KEY |             - hive                       | Enabled
 SERVICE KEY |             - hive2                      | Enabled
             |           IMPALA:                        | 
 SERVICE KEY |             - impala                     | Enabled
 SERVICE KEY |             - impala2                    | Enabled
             |           KAFKA:                         | 
 SERVICE KEY |             - kafka                      | Enabled
 SERVICE KEY |             - kafka2                     | Enabled
             |           SPARK_ON_YARN:                 | 
 SERVICE KEY |             - spark_on_yarn              | Enabled
             |           YARN:                          | 
 SERVICE KEY |             - yarn                       | Enabled
             |           ZOOKEEPER:                     | 
 SERVICE KEY |             - zookeeper                  | Enabled
------------ | ---------------------------------------- | ------------
-- OK

In a multi-cluster deployment, where edge nodes are monitoring, set the password on the core node as follows:

<Unravel installation directory>/unravel/manager config edge hive metastore password <EDGE_KEY> <CLUSTER-KEY> <HIVE-KEY> <password> 
##Example: /opt/unravel/manager config edge hive metastore password local-node cluster1 hive password

In case, the core node is monitoring the Hadoop cluster directly, run the following command from the core node.

<Unravel installation directory>/unravel/manager config hive metastore password <CLUSTER_KEY> <HIVE_KEY> <password> 
##Example: /opt/unravel/unravel/manager config edge hive metastore password cluster1 hive P@SSw0rd

Configure the FSImage. Refer to configuring FSImage .

Enable hive_worker daemon

After upgrade from versions prior to v4.7.x, you must manually enable the hive_worker daemon.

Open the <Unravel_installation_directory>/unravel/data/conf/unravel.yaml file using an editor.

In the unravel.yaml file, set the hive_worker to true to enable the hive_worker daemon.

services:
  aws_worker:
    enabled: true
  hive_worker:
    enabled: true
  pgsql:
    enabled: true
  yarn_jc_sensor:
    enabled: false

Stop Unravel, refresh the files, and start Unravel again.

<Unravel installation directory>/unravel/manager stop
<Unravel installation directory>/unravel/manager refresh all
<Unravel installation directory>/unravel/manager start

Upgrading Unravel sensors

Refer to Upgrading sensors.

Important

In a multi-cluster deployment, it is mandatory to Upgrade the core node before you upgrade the sensors on the edge node.

Enable Migration reports

If you had migrations reports on CDH or CDP clusters, you must run the following steps to continue using the migration reports.

Stop Unravel.

<Unravel installation directory>/unravel/manager stop

Enable the Migration reports as an admin user. Run the following command:

<Unravel installation directory>/unravel/manager config ondemand cloud-migration enable

Apply changes.

<Unravel installation directory>/unravel/manager config apply

Start Unravel
```
<Unravel installation directory>/unravel/manager start
```
The Migration tab will be visible on the Unravel UI. Also, refer to the Migrations topics for more details.

In this section:

Would you like to provide feedback? Just click here to suggest edits.

Home

Upgrading Unravel from version 4.6.2.x to 4.7.x

Caution

Caution

Note

Upgrading Unravel in a single cluster deployment (4.6.2.x to 4.7.x)

Upgrading Unravel in multi-cluster deployment (4.6.2.x to 4.7.x)

Caution

Caution

Configuring edge nodes

Important

Important

Notice

Important

Enable hive_worker daemon

Upgrading Unravel sensors

Important

Enable Migration reports

Search results