Skip to main content

Home

Upgrading Unravel from version 4.6.2.x to 4.7.x

This topic provides instructions to upgrade from Unravel version 4.6.2.x to 4.7.x.

Caution

Caution

Before upgrading Unravel, you must take a backup of the data directory and external database (if using). Perform the following steps:

  1. Stop Unravel.

    <Unravel installation directory>/unravel/manager stop
  2. Archive the entire data directory.

    For example,

    tar czf /archive/destination/unravel-backup.tar.gz /opt/unravel/data
  3. If using an external database, perform a backup of the database.

    Note

    For the details on how to back up the database, refer to the documentation for your corresponding database.

Upgrading Unravel in a single cluster deployment (4.6.2.x to 4.7.x)
  1. Download and extract the Unravel binaries of the Unravel version that you want to upgrade/downgrade to.

  2. Stop Unravel.

    <Unravel installation directory>/unravel/manager stop
  3. Activate the version that you want to upgrade/downgrade to.

    <Unravel installation directory>/unravel/manager activate <Unravel-version>
  4. Run auto-configuration and refresh to automatically reload the updates.

    <unravel_installation_directory>/unravel/manager config auto --refresh
    

    After the auto-configuration completes, apply the changes.

    <unravel_installation_directory>/unravel/manager config apply
    
  5. Start all the services.

    <unravel_installation_directory>/unravel/manager start 
    
  6. Check the status of services.

    <unravel_installation_directory>/unravel/manager report 
    

    The following service statuses are reported:

    • OK: Service is up and running.

    • Not Monitored: Service is not running. (Has stopped or has failed to start)

    • Initializing: Services are starting up.

    • Does not exist: The process unexpectedly disappeared. Restarts will be attempted 10 times.

  7. Delete the old install directory from unravel/versions/<THE.OLD.VERSION>.

Upgrading Unravel in multi-cluster deployment (4.6.2.x to 4.7.x)

To upgrade Unravel in a multi-cluster environment, you must upgrade all the edge nodes, upgrade the core node, and pull in all the edge node updates to the core node.

  1. Edge nodes

    Run the following steps to upgrade Unravel on the edge nodes involved in Unravel monitoring.

    1. Download and extract the Unravel binaries of the Unravel version that you want to upgrade/downgrade to.

    2. Stop Unravel.

      <Unravel installation directory>/unravel/manager stop
    3. Activate the version that you want to upgrade/downgrade to.

      <Unravel installation directory>/unravel/manager activate <Unravel-version>

      Caution

      You may get a Network port failed precheck error if you run the manager activate command immediately after executing the manager stop command.

      To avoid this precheck error, stop Unravel and then ensure to wait for a minute or two before executing the manager activate command.

      Retry the manager activate command in case of the Network port failed precheck error.

    4. Run auto-configuration and refresh to automatically reload the updates.

      <unravel_installation_directory>/unravel/manager config auto --refresh
      

      After the auto-configuration completes, apply the changes.

      <unravel_installation_directory>/unravel/manager config apply
      
    5. Start all the services.

      <unravel_installation_directory>/unravel/manager start 
      
    6. Check the status of services.

      <unravel_installation_directory>/unravel/manager report 
      

      The following service statuses are reported:

      • OK: Service is up and running.

      • Not Monitored: Service is not running. (Has stopped or has failed to start)

      • Initializing: Services are starting up.

      • Does not exist: The process unexpectedly disappeared. Restarts will be attempted 10 times.

  2. Core node

    Run the following steps only on the core node.

    1. Download and extract the Unravel binaries of the Unravel version that you want to upgrade/downgrade to.

    2. Stop Unravel.

      <Unravel installation directory>/unravel/manager stop
    3. Activate the version that you want to upgrade/downgrade to.

      <Unravel installation directory>/unravel/manager activate <Unravel-version>
    4. Optional: If you have run the config auto command previously on the cluster, then run auto-configuration on the core node and refresh to automatically reload the updates.

      <unravel_installation_directory>/unravel/manager config auto --refresh
      

      Caution

      If you have NOT run the manager config auto earlier on the core node, an error will be displayed, which can be ignored.

    5. Run the following command to get the list of edge clusters that are configured.

      <Unravel installation directory>/unravel/manager config edge show
      -- Running: config edge show
      ------------ | ---------------------------------------- | ------------
          EDGE KEY | - local-node                             | Enabled
                   |     Cluster manager:                     | Enabled
                   |     Clusters:                            | 
       CLUSTER KEY |       - HDP47314                         | Enabled
                   |           - No services found            | 
      ------------ | ---------------------------------------- | ------------
      -- OK
      1. Remove the edge configuration that shows EDGE KEY as local-node.

        <Unravel installation directory>/unravel/manager config edge remove <EDGE KEY>
      2. Configure the required edge nodes. Refer to the instructions in Configuring edge nodes.

    6. Apply the changes.

      <unravel_installation_directory>/unravel/manager config apply
      
    7. Start all the services.

      <unravel_installation_directory>/unravel/manager start 
      
    8. Check the status of services.

      <unravel_installation_directory>/unravel/manager report 
      
    9. Check the list of services, which are enabled for the edge node after running the auto-configurations.

      <unravel_installation_directory>/unravel/manager config edge show
      -- Running: config edge show
      ------------ | ---------------------------------------- | ------------
          EDGE KEY | - edge-a                                 | Enabled
                   |     Cluster manager:                     | Enabled
                   |     Clusters:                            | 
       CLUSTER KEY |       - Cluster_Name                     | Enabled
                   |           HBASE:                         | 
       SERVICE KEY |             - hbase                      | Enabled
                   |           HDFS:                          | 
       SERVICE KEY |             - hdfs                       | Enabled
                   |           HIVE:                          | 
       SERVICE KEY |             - hive                       | Enabled
       SERVICE KEY |             - hive2                      | Enabled
                   |           IMPALA:                        | 
       SERVICE KEY |             - impala                     | Enabled
       SERVICE KEY |             - impala2                    | Enabled
                   |           KAFKA:                         | 
       SERVICE KEY |             - kafka                      | Enabled
       SERVICE KEY |             - kafka2                     | Enabled
                   |           SPARK_ON_YARN:                 | 
       SERVICE KEY |             - spark_on_yarn              | Enabled
                   |           YARN:                          | 
       SERVICE KEY |             - yarn                       | Enabled
                   |           ZOOKEEPER:                     | 
       SERVICE KEY |             - zookeeper                  | Enabled
      ------------ | ---------------------------------------- | ------------
      -- OK

      You can disable any of the services. For example, you want to disable the HBase services:

      <unravel_installation_directory>/unravel/manager config edge hbase disable <EDGE_KEY> <CLUSTER KEY> <SERVICE NAME>
      Example: /opt/unravel/manager config edge hbase disable edge-a Cluster_Name hbase
    10. Run healthcheck to verify that all the configurations and services are running successfully.

      <unravel_installation_directory>/manager healthcheck
      

      Healthcheck is run automatically, on an hourly basis, in the backend. You can set your email to receive the healthcheck reports.

    11. Delete the old install directory from unravel/versions/<THE.OLD.VERSION>.

Configuring edge nodes
  1. Obtain the Cluster Access ID, which must be provided when you add the edge nodes. Run the following command on the edge node:

    <unravel_installation_directory>/unravel/manager support show cluster_access_id

    Important

    Ensure to repeat the following steps (2-7) on the core node for each of the edge nodes.

  2. Add each of the edge nodes, involved with Unravel monitoring, to the core node.

    Important

    • EDGE_KEY is any label that you provide to identify the edge node. Preferably provide the hostname of the edge node for quick identification later.

    <unravel_installation_directory>/unravel/manager config edge add <EDGE_KEY> <CLUSTER_ACCESS_ID>
    ##Example: /opt/unravel/manager config edge add edge-a 123-123-123-123
    

    Notice

    When you are adding edge nodes for CDH or CDP platforms and If you want to configure Migration or Forecasting reports for these clusters, then you must have one of the following roles: MigrationsForecasting

    • Full Administrator

    • Cluster Administrator

    • Operator

    • Configurator

    These roles are required only in the case of the Cloudera manager.

  3. Run auto-configuration.

    Important

    • EDGE_KEY is the label that you have configured in step 3 as <EDGE_KEY>.

    <unravel_installation_directory>/unravel/manager config edge auto <EDGE_KEY> 
    //For example: /opt/unravel/manager config edge auto edge-a 

    When prompted, you can provide the following:

    • Cluster manager URL: Provide the URL of the cluster manager. For example: https://abcd773.unraveldata.com:8443

    • Username

    • Password

  4. The Hive metastore database password can be recovered automatically only for a cluster manager with an administrative account. Otherwise, it must be set manually as follows:

    1. Run the manager config edge show command to get the <EDGE_KEY>, <HIVE_KEY>, and <CLUSTER_KEY>, which must be provided when you set the Hive metastore password.

      • <EDGE_KEY> is the label you provide to identify the edge node when you add the edge node in Step 3.

      • CLUSTER_KEY is the name of the cluster where you set the Hive configurations.

      • <HIVE_KEY> is the definition of the Hive service. In the output of the manager config edge show command, this is shown as the <SERVICE_KEY> for Hive.

      -- Running: config edge show
      ------------ | ---------------------------------------- | ------------
          EDGE KEY | - edge-a                                 | Enabled
                   |     Cluster manager:                     | Enabled
                   |     Clusters:                            | 
       CLUSTER KEY |       - Cluster_Name                     | Enabled
                   |           HBASE:                         | 
       SERVICE KEY |             - hbase                      | Enabled
                   |           HDFS:                          | 
       SERVICE KEY |             - hdfs                       | Enabled
                   |           HIVE:                          | 
       SERVICE KEY |             - hive                       | Enabled
       SERVICE KEY |             - hive2                      | Enabled
                   |           IMPALA:                        | 
       SERVICE KEY |             - impala                     | Enabled
       SERVICE KEY |             - impala2                    | Enabled
                   |           KAFKA:                         | 
       SERVICE KEY |             - kafka                      | Enabled
       SERVICE KEY |             - kafka2                     | Enabled
                   |           SPARK_ON_YARN:                 | 
       SERVICE KEY |             - spark_on_yarn              | Enabled
                   |           YARN:                          | 
       SERVICE KEY |             - yarn                       | Enabled
                   |           ZOOKEEPER:                     | 
       SERVICE KEY |             - zookeeper                  | Enabled
      ------------ | ---------------------------------------- | ------------
      -- OK
    2. In a multi-cluster deployment, where edge nodes are monitoring, set the password on the core node as follows:

      <Unravel installation directory>/unravel/manager config edge hive metastore password <EDGE_KEY> <CLUSTER-KEY> <HIVE-KEY> <password> 
      ##Example: /opt/unravel/manager config edge hive metastore password local-node cluster1 hive password
      

      In case, the core node is monitoring the Hadoop cluster directly, run the following command from the core node.

      <Unravel installation directory>/unravel/manager config hive metastore password <CLUSTER_KEY> <HIVE_KEY> <password> 
      ##Example: /opt/unravel/unravel/manager config edge hive metastore password cluster1 hive P@SSw0rd
  5. Configure the FSImage. Refer to configuring FSImage .

Enable hive_worker daemon

After upgrade from versions prior to v4.7.x, you must manually enable the hive_worker daemon.

  1. Open the <Unravel_installation_directory>/unravel/data/conf/unravel.yaml file using an editor.

  2. In the unravel.yaml file, set the hive_worker to true to enable the hive_worker daemon.

    services:
      aws_worker:
        enabled: true
      hive_worker:
        enabled: true
      pgsql:
        enabled: true
      yarn_jc_sensor:
        enabled: false
  3. Stop Unravel, refresh the files, and start Unravel again.

    <Unravel installation directory>/unravel/manager stop
    <Unravel installation directory>/unravel/manager refresh all
    <Unravel installation directory>/unravel/manager start
Upgrading Unravel sensors

Refer to Upgrading sensors.

Important

In a multi-cluster deployment, it is mandatory to Upgrade the core node before you upgrade the sensors on the edge node.

Enable Migration reports

If you had migrations reports on CDH or CDP clusters, you must run the following steps to continue using the migration reports.

  1. Stop Unravel.

    <Unravel installation directory>/unravel/manager stop
    
  2. Enable the Migration reports as an admin user. Run the following command:

    <Unravel installation directory>/unravel/manager config ondemand cloud-migration enable
    
  3. Apply changes.

    <Unravel installation directory>/unravel/manager config apply
  4. Start Unravel

    <Unravel installation directory>/unravel/manager start

    The Migration tab will be visible on the Unravel UI. Also, refer to the Migrations topics for more details.