Home

Healthcheck and Precheck

This section provides information about the healthcheck and precheck scripts that are packaged with the Unravel installation.

Healthcheck

Healthcheck is a series of verifications and information gathering, such as:

  • Preinstallation checks (CPU, memory, free space, etc.)

  • Status of Unravel services

  • Health of Unravel services

  • Detection of errors and issues using log parsing

  • Information gathering to assist support

All healthcheck actions are defined in healthcheck/conf/actions. directory. Actions can be for either precheck, healthcheck, or both. Usually, precheck actions are also part of the healthcheck actions.

  • Precheck: Verifying Installation requirements.

  • Healthcheck: Making sure that unravel is healthy.

Actions can be either of the following or both:

  • Gather: Get some information for self-validation or as a source for a check.

  • Check: A verification either by directly interacting with services or by reading data from a gather action.

Actions can be dependent on other actions. For example, a check action is dependent on a gather action. Healthcheck ensures that the required actions are run first. If a dependency fails, the action also fails.

Actions can run any of the following checks or gather information:

  • Service: The action runs for each and every service specified. Either for a specific service or for all services. This runs only if those services are configured.

  • Statistics: These usually gather and extract information about unravel.

  • System: Provides information and verification on the server that runs unravel.

  • Hadoop: Hadoop specific checks such as HDFS access.

  • Configuration: Configuration extraction actions. Unravel settings and related information is gathered.

  • State: The current global state of Unravel (service enabled, running, etc). This is not service-specific.

Note

Any failures in gather action or check action are not reported and it does not fail the healthcheck.

Running healthcheck

You can use healthcheck to run all checks on all the services, checks on specific services, or to run a specific check.

Action

Command

To run all checks on all the services.

manager healthcheck

To run checks on a specific services

manager healthcheck <service>

For example:

manager healthcheck database,zookeeper,elasticsearch

To run a specific check

manager healthcheck <checks code>

For example:

manager healthcheck kafka_is_alive

Healthcheck generates a tar file. For example, healthcheck-20200717212312-server.domain.com.tar.gz. This tar file contains:

  • Text reports (healthcheck.report.txt)

  • Raw healthcheck data in YAML format (healthcheck.yaml)

  • Configuration, migration, statistics, etc. (global, services)

  • Log extracts (healthcheck.log)

Following is a sample list of the healthcheck files:

healthcheck.log --> Logfile of the healthcheck run that generated the bundle
healthcheck.report.txt --> The text report
healthcheck.yaml --> The raw healthcheck data

global --> Data for global actions
global/configuration
global/statistics
global/hadoop
global/system
global/state

services --> Data of per services actions
services/elasticsearch_1
services/kafka_1
services/ngui_1
services/zookeeper_1
Precheck

Precheck verifies all the installation requirements. You can run prechecks in any of the following stages:

Running precheck - before installation

You can get the healthcheck package and run precheck.sh before the installation. Following is the sample output of running the precheck.

2020-07-17 21:23:12 Sending logs to: /tmp/healthcheck-20200717212312-server.domain.com/healthcheck.log
2020-07-17 21:23:13 Gathering information ................ Ok
2020-07-17 21:23:36 Running checks .................... Ok
--------------------------------------------------------------------------------
system
---------------------------------------
Check uptime                  : OK
Clock sync                    : OK
CPU requirement               : OK, 8 cores
Disk access                   : OK, /tmp/healthcheck/healthcheck/plugins/system is writable
Disk freespace                : OK, precheck, found 152 GB.
Kerberos tools                : OK
Memory requirement            : OK, 98 GB
Network ports                 : OK
OS libraries                  : OK
OS release                    : OK, centos 7.6
OS settings                   : OK
SELinux                       : OK
Tmp noexec                    : OK


--------------------------------------------------------------------------------
hadoop
---------------------------------------
Clients                       : OK
                              - Found hadoop
                              - Found hdfs
                              - Found yarn
                              - Found hive
                              - Found beeline
Distribution                  : OK, found CDH 6.1.1
HDFS access                   : OK, /
YARN access                   : OK


Healthcheck bundle: /tmp/healthcheck-20200717212312-server.domain.com.tar.gz
Running precheck - after deploying Unravel binaries

Precheck is run automatically whenever you run the setup command. After deploying the binaries, run precheck as follows:

setup --precheck

After running the precheck, the results are added to /tmp .

To skip the precheck run the setup command as follows:

setup --skip-precheck

You can also skip the checks that you know can fail. For example, if you want to skip memory and Hadoop checks, run the setup command as follows:

setup --filter-precheck ~hadoop,~mem_minimum
Running precheck - After installation

The preinstallation checks are run automatically when running healthcheck. However, post-installation also you can run the precheck from the manager tool as follows:

manager precheck
In this section: