Corporate logotype
  • Free Trial
  • Blog
  • Unravel
  • Select version
    • v4.6.0 (Preview)
    • v4.5.x (Current)
    • v4.4.x
    • v4.3.x
    • v4.2.x
    • v4.1.x
Corporate logotype
  • Overview
    • Unravel 4.5 Documentation
      • Formatting conventions
      • Installation
        • Running a pre-check
        • Quick start guide
          • Unravel deployment for high availability
          • Platforms
            • Cloudera Distribution of Apache Hadoop (CDH)
              • Prerequisites
              • Part 1: Installing Unravel Server on CDH+CM
              • Part 2: Enabling additional instrumentation
              • Creating an alternate principal
              • Adding a new node in an existing CDH cluster
              • Troubleshooting
            • Hortonworks Data Platform (HDP)
              • Prerequisites
              • Part 1: Installing Unravel Server
              • Part 2: Enabling additional instrumentation
              • Adding a new node in an existing HDP cluster monitored by Unravel
            • MapR
              • Prerequisites
              • Part 1: Installing Unravel Server on MapR
              • Part 2: Enabling additional instrumentation
            • Amazon Elastic MapReduce (EMR)
              • Prerequisites
              • Architecture
              • Planning guidance
              • Installing Unravel Server on an EC2 instance
              • Connecting Unravel Server to a new or existing EMR cluster
              • Setting up Amazon RDS (optional)
              • Setting up VPC peering (optional)
              • Adding instrumentation for a new node
              • Testing and troubleshooting
              • Operational guidance
              • Deploying Unravel from the AWS Marketplace
            • Amazon Athena (preview only)
              • Creating private subnets for Unravel's Lambda function
            • Google Dataproc
              • Prerequisites
              • Architecture
              • Installing Unravel Server on an GCE VM
              • Connecting Unravel Server to a new Dataproc cluster
              • Setting up Google Cloud SQL (optional)
            • Microsoft Azure HDInsight
              • Architecture
              • Prerequisites
              • Creating Azure storage
              • Part 1: Installing Unravel on a Separate Azure VM
              • Part 2: Connecting Unravel to an HDInsight cluster
              • Finding properties in Azure
              • Using Azure HDInsight APIs
              • Deploying Unravel for Azure HDInsight from Azure Marketplace
              • Adding a new node in an existing HDI cluster monitored by Unravel
              • Upgrading Unravel Server
              • Setting up Azure MySQL for Unravel (Optional)
            • Microsoft Azure Databricks
              • Deploying Unravel for Azure Databricks from Azure Marketplace
              • Prerequisites
              • Architecture
              • Create Azure VM
              • Create Azure Database for MySQL
              • Create Azure Databricks
              • Install Unravel on Azure VM
              • Configure and restart Unravel
              • Configure Azure Databricks Automated (Job) Clusters with Unravel
              • Running the Databricks_setup.sh script
              • Uninstalling Unravel Server and Sensors on Azure Databricks
          • MySQL
            • Installing MySQL
              • Configuring MySQL
                • Setting up Azure MySQL for Unravel (Optional)
                • Add-ons
                  • OnDemand
                    • Installing or upgrading OnDemand
                    • Library versions and licensing for OnDemand
                  • Matomo (preview only)
                  • Next steps
                  • Upgrading Unravel Server
                  • Upgrading Unravel Sensors
                    • Upgrading sensors on CDH
                    • Upgrading sensors on EMR
                      • Upgrading sensors on HDP
                      • Upgrading sensors on MapR
                  • User guide
                    • Getting started
                      • Operations page
                        • Applications page
                          • All applications
                          • Running applications
                          • Sessions
                            • Workflow
                          • Reports page
                            • Operational insights
                              • Data insights
                                • Migration planning
                                  • Reports archive
                                    • Scheduled reports
                                    • Scheduling reports
                                  • Unravel for Azure Databricks
                                    • Operations page
                                      • Applications page
                                        • Reports page
                                          • Operational Insights
                                          • Data insights
                                      • Application Performance Management (APM)
                                        • Typical APM layout
                                        • Spark APM
                                          • Hive APM
                                            • Impala APM
                                              • Kafka APM
                                                • MapReduce APM
                                                  • Tez APM
                                                    • Workflow APM
                                                      • Athena APM (preview)
                                                        • Cascading and Pig APM
                                                          • HBase alerts and metrics
                                                          • Auto Actions
                                                            • Overview
                                                              • Limitations
                                                                • Templates
                                                                  • Creating AutoActions
                                                                  • Expert rule
                                                                    • Same logical operator
                                                                      • Snooze feature
                                                                        • AutoActions examples
                                                                          • Running AutoAction demos
                                                                            • Supported cluster metrics
                                                                            • Use Cases
                                                                              • Detecting resource contention in the cluster
                                                                                • Detecting apps using resources inefficiently
                                                                                • Identifying rogue apps
                                                                                • End-to-end monitoring of HBase databases and clusters
                                                                                  • Best practices for end-to-end monitoring of Kafka
                                                                                    • Kafka detecting lagging or stalled partitions
                                                                                      • Optimizing the performance of Spark apps
                                                                                        • Using Unravel to tune Spark data skew and partitioning
                                                                                          • How to intelligently monitor Kafka/Spark Streaming data pipeline
                                                                                            • Using RDD caching to improve a Spark app's performance
                                                                                          • Common UI features
                                                                                          • Miscellaneous
                                                                                            • Event panel & insights
                                                                                              • Some keywords and error message
                                                                                                • Resource metrics
                                                                                            • Advanced topics
                                                                                              • Advanced instrumentation options
                                                                                                • Enabling the JVM sensor
                                                                                                  • Enabling the JVM sensor on HDP cluster-wide for MapReduce2 (MR)
                                                                                                • Adding additional Spark instrumentation
                                                                                                  • Monitoring individual Spark apps
                                                                                                  • Uploading Spark programs to Unravel
                                                                                                • Integrating with Informatica Big Data Management
                                                                                                  • Monitoring individual Hive queries
                                                                                                  • Advanced installation options
                                                                                                    • Modifying the behavior of unravel_es
                                                                                                      • Deploying Unravel on security-enhanced Linux
                                                                                                        • Enabling multiple daemons for high-volume data
                                                                                                        • Running verification scripts and benchmarks
                                                                                                          • Uninstalling Unravel Server
                                                                                                        • Configurations
                                                                                                          • Cloud
                                                                                                            • Creating an AWS RDS CloudWatch Alarm for Free Storage Space
                                                                                                          • HBase
                                                                                                            • Elasticsearch storage requirements on the Unravel Node
                                                                                                            • Configuring HBase
                                                                                                          • Connecting to the Hive Metastore
                                                                                                            • Populating the Unravel Data Insights page
                                                                                                            • Configuring access for an Oracle database
                                                                                                          • Kafka
                                                                                                            • Connecting to a Kafka cluster
                                                                                                            • Kafka security
                                                                                                          • Kerberos
                                                                                                            • Changing the Kerberos keytab files
                                                                                                            • Creating Active Directory Kerberos principals and keytabs for Unravel
                                                                                                          • Miscellaneous
                                                                                                            • Configuring email alerts
                                                                                                            • Creating Active Directory Kerberos principals and keytabs for Unravel
                                                                                                            • Enable authentication for the Unravel Elastic daemon
                                                                                                            • Encrypting passwords in Unravel properties and settings
                                                                                                            • Importing a private certificate into Unravel truststore
                                                                                                            • Running Unravel daemons with a custom user
                                                                                                            • Setting retention time in Unravel server
                                                                                                            • Using a private certificate authority with Unravel
                                                                                                          • OnDemand reports
                                                                                                            • OnDemand configurations
                                                                                                            • Configuring forecasting and migration planning reports
                                                                                                            • Configuring small files and files reports
                                                                                                            • Triggering an import of FSimage
                                                                                                          • Secure UI access
                                                                                                            • Adding SSL and TLS to Unravel web UI
                                                                                                            • Enabling LDAP authentication for Unravel UI
                                                                                                            • Enabling SAML authentication for Unravel Web UI
                                                                                                            • Enabling TLS to Unravel Web UI directly
                                                                                                          • Spark
                                                                                                            • Configure Spark properties for Spark Worker daemon @ Unravel
                                                                                                            • Configuring notebooks for Spark
                                                                                                            • Enable/disable live monitoring of Spark Streaming applications
                                                                                                          • User interface
                                                                                                            • Defining a custom banner
                                                                                                            • Defining a custom web UI port
                                                                                                            • Disabling browser telemetry
                                                                                                            • Disabling support/comments panel
                                                                                                            • Restricting direct access to Unravel UI
                                                                                                            • Specifying a cluster ID or name
                                                                                                          • Unravel admins
                                                                                                            • Adding more admins to Unravel
                                                                                                            • Adding read-only admins to Unravel
                                                                                                            • Changing Unravel admin's password
                                                                                                          • Workflows
                                                                                                            • Monitoring Airflow workflows
                                                                                                            • Monitoring Oozie workflows
                                                                                                        • Miscellaneous
                                                                                                          • Backup and archive metric database
                                                                                                          • Cluster wide report
                                                                                                            • Stopping, restarting, and configuring the AutoAction daemon
                                                                                                            • Moving MySQL to a custom location
                                                                                                              • Partitioning MySQL and migrating data
                                                                                                              • MySQL
                                                                                                                • Backing up the MySQL database
                                                                                                                • Restoring the MySQL database
                                                                                                                • Moving MySQL to a custom location
                                                                                                                  • Partitioning MySQL and migrating data
                                                                                                                  • REST API
                                                                                                                    • Swagger content
                                                                                                                    • Authentication through Unravel UI
                                                                                                                    • Applications API
                                                                                                                    • Apps API
                                                                                                                    • AutoActions API
                                                                                                                    • Capacity forecasting API
                                                                                                                    • Chargeback reports API
                                                                                                                    • Cluster optimization reports API
                                                                                                                    • Data insights API
                                                                                                                    • File reports API
                                                                                                                    • Jobs API
                                                                                                                    • Migration reports API
                                                                                                                    • Operational
                                                                                                                    • Operations API
                                                                                                                    • Queue analysis reports API
                                                                                                                    • Report archives API
                                                                                                                    • Scheduled reports API
                                                                                                                    • Small files reports API
                                                                                                                    • Top "X" reports API
                                                                                                                    • Workflows API
                                                                                                                    • Use case - AutoActions and Pagerduty
                                                                                                                      • All endpoints by resource name
                                                                                                                        • Apps endpoints
                                                                                                                        • AutoAction endpoints
                                                                                                                        • Celery endpoints
                                                                                                                        • Chargeback reports
                                                                                                                        • Cluster reports
                                                                                                                        • Common endpoints
                                                                                                                        • Job endpoints
                                                                                                                        • Kafka endpoints
                                                                                                                        • Report endpoints
                                                                                                                        • Search endpoints
                                                                                                                        • Authentication API
                                                                                                                        • Workflow endpoints
                                                                                                                        • YARN resource manager endpoints
                                                                                                                    • Roles and Role Based Access Control (RBAC)
                                                                                                                      • RBAC roles
                                                                                                                        • Configuring RBAC general properties
                                                                                                                        • Configuring LDAP or SAML RBAC properties
                                                                                                                          • Example RBAC configurations
                                                                                                                            • Manage page
                                                                                                                            • RBAC UI
                                                                                                                            • Tagging
                                                                                                                              • What is tagging?
                                                                                                                                • Tagging applications
                                                                                                                                  • Tagging workflows
                                                                                                                                    • Tagging a Hive on Tez query
                                                                                                                                    • Unravel properties
                                                                                                                                      • Basic
                                                                                                                                      • Executor logs
                                                                                                                                        • Airflow
                                                                                                                                        • Analytics
                                                                                                                                        • AutoAction
                                                                                                                                        • Azure
                                                                                                                                        • Btrace
                                                                                                                                        • Celery
                                                                                                                                        • Cluster
                                                                                                                                        • Cluster manager
                                                                                                                                        • Custom banner
                                                                                                                                        • Data insights
                                                                                                                                        • Email
                                                                                                                                        • Experimental
                                                                                                                                        • File reports
                                                                                                                                        • Forecasting report
                                                                                                                                        • FSimage
                                                                                                                                        • HBase
                                                                                                                                        • Hive Hook SSL
                                                                                                                                        • Hive Metastore access
                                                                                                                                        • Hive Metastore JDBC
                                                                                                                                        • HiveServer2
                                                                                                                                        • Impala
                                                                                                                                        • Kafka
                                                                                                                                        • LDAP
                                                                                                                                        • Cloud migration reports
                                                                                                                                        • Miscellaneous
                                                                                                                                        • Oozie
                                                                                                                                        • Queue analysis
                                                                                                                                        • Role Based Access Control (RBAC)
                                                                                                                                        • S3 monitoring
                                                                                                                                        • SAML
                                                                                                                                        • Sensor
                                                                                                                                        • Sessions
                                                                                                                                        • Small files report
                                                                                                                                        • Small files and file reports
                                                                                                                                        • Spark
                                                                                                                                        • Spark S3
                                                                                                                                        • SSL/TLS
                                                                                                                                        • Tagging
                                                                                                                                        • Tez
                                                                                                                                        • Yarn Timeline
                                                                                                                                        • Top X
                                                                                                                                        • Workflow
                                                                                                                                    • Appendices
                                                                                                                                      • Unravel server reference
                                                                                                                                      • Troubleshooting
                                                                                                                                        • Sending Diagnostics to Unravel Support
                                                                                                                                      • Release notes
                                                                                                                                        • v4.6.x Release notes
                                                                                                                                          • v4.6.0 Release notes
                                                                                                                                            • v4.6.x - Updates to Unravel properties
                                                                                                                                            • v4.5.x Release notes
                                                                                                                                              • v4.5.5.2 Release notes
                                                                                                                                                • v4.5.5.1 Release notes
                                                                                                                                                  • v4.5.5.0 Release notes
                                                                                                                                                    • v4.5.4.0 Release notes
                                                                                                                                                      • v4.5.4.4 Release notes
                                                                                                                                                        • v4.5.4.3 Release notes
                                                                                                                                                          • v4.5.4.2 Release notes
                                                                                                                                                          • v4.5.3 Release notes
                                                                                                                                                            • v4.5.3.3 Release notes
                                                                                                                                                              • v4.5.3.2 Release notes
                                                                                                                                                                • v4.5.3.1 Release notes
                                                                                                                                                                • v4.5.2 Release notes
                                                                                                                                                                  • v4.5.2.3 Release notes
                                                                                                                                                                    • v4.5.2.2 Release notes
                                                                                                                                                                      • v4.5.2.1 Release notes
                                                                                                                                                                      • v4.5.1 Release notes
                                                                                                                                                                        • v4.5.1.1 Release notes
                                                                                                                                                                        • v4.5.0 Release notes
                                                                                                                                                                          • v4.5.0.9 Release notes
                                                                                                                                                                            • v4.5.0.8 Release notes
                                                                                                                                                                              • v4.5.0.7 Release notes
                                                                                                                                                                                • v4.5.0.6 Release notes
                                                                                                                                                                                  • v4.5.0.5 Release notes
                                                                                                                                                                                    • v4.5.0.4 Release notes
                                                                                                                                                                                      • v4.5.0.3 Release notes
                                                                                                                                                                                        • v4.5.0.2 Release notes
                                                                                                                                                                                        • v4.5.x Upgrades
                                                                                                                                                                                          • v4.5.3.x - Upgrade instructions
                                                                                                                                                                                          • v4.5.2.x - Upgrade instructions
                                                                                                                                                                                          • v4.5.1.x - Upgrade instructions
                                                                                                                                                                                          • v4.5.x - Updates to Unravel properties
                                                                                                                                                                                          • v4.4.x Release notes
                                                                                                                                                                                            • v4.4.2.0 Release notes
                                                                                                                                                                                              • v4.4.2.1 Release notes
                                                                                                                                                                                              • v4.4.1 Release notes
                                                                                                                                                                                                • v4.4.1 - New configuration properties
                                                                                                                                                                                                • v4.4.1.1 Release notes
                                                                                                                                                                                                • v4.4.0 Release notes
                                                                                                                                                                                                  • v4.4.x New Unravel properties
                                                                                                                                                                                                    • v4.4.x - Upgrade instructions
                                                                                                                                                                                                    • v4.3.x Release notes
                                                                                                                                                                                                      • v4.3.1.0 Release notes
                                                                                                                                                                                                        • v4.3.1.10
                                                                                                                                                                                                        • v4.3.1.9 Release notes
                                                                                                                                                                                                        • v4.3.1.7 Release notes
                                                                                                                                                                                                        • v4.3.1.6 Release notes
                                                                                                                                                                                                        • v4.3.1.5 Release notes
                                                                                                                                                                                                        • v4.3.1.4 Release notes
                                                                                                                                                                                                        • v4.3.1.3 Release notes
                                                                                                                                                                                                        • v4.3.1.2 Release notes
                                                                                                                                                                                                        • v4.3.1.1 Release notes
                                                                                                                                                                                                      • v4.3.0 Release notes
                                                                                                                                                                                                      • v4.2.x Release notes
                                                                                                                                                                                                        • v4.2.7 Release notes
                                                                                                                                                                                                          • v4.2.6 Release notes
                                                                                                                                                                                                            • v4.2.5 Release notes
                                                                                                                                                                                                              • v4.2.4 Release notes
                                                                                                                                                                                                                • v4.2.3 Release notes
                                                                                                                                                                                                                  • v4.2 Release notes
                                                                                                                                                                                                                  • v4.0-v4.1 Release notes
                                                                                                                                                                                                                    • v4.1.9.4 Release notes
                                                                                                                                                                                                                  • Newsletters
                                                                                                                                                                                                                  • Compatibility matrix
                                                                                                                                                                                                                    • Downloads
                                                                                                                                                                                                                      • Unravel 4.6.x
                                                                                                                                                                                                                        • v4.6.0.0
                                                                                                                                                                                                                      • Unravel 4.5
                                                                                                                                                                                                                        • v4.5.5.2
                                                                                                                                                                                                                          • v4.5.5.1
                                                                                                                                                                                                                            • v4.5.5.0
                                                                                                                                                                                                                              • v4.5.4.4
                                                                                                                                                                                                                                • v4.5.4.3
                                                                                                                                                                                                                                  • v4.5.4.2
                                                                                                                                                                                                                                    • v4.5.4.0
                                                                                                                                                                                                                                      • v4.5.3.2
                                                                                                                                                                                                                                        • v4.5.3.1
                                                                                                                                                                                                                                          • v4.5.3.0
                                                                                                                                                                                                                                          • v4.5.2.3
                                                                                                                                                                                                                                            • v4.5.2.2
                                                                                                                                                                                                                                              • v4.5.2.1
                                                                                                                                                                                                                                                • v4.5.2.0
                                                                                                                                                                                                                                                  • v4.5.1.1
                                                                                                                                                                                                                                                    • v4.5.1.0
                                                                                                                                                                                                                                                      • v4.5.0.9
                                                                                                                                                                                                                                                        • v4.5.0.8
                                                                                                                                                                                                                                                          • v4.5.0.7
                                                                                                                                                                                                                                                            • v4.5.0.6
                                                                                                                                                                                                                                                              • v4.5.0.5
                                                                                                                                                                                                                                                                • v4.5.0.4
                                                                                                                                                                                                                                                                  • v4.5.0.3
                                                                                                                                                                                                                                                                    • v4.5.0.2
                                                                                                                                                                                                                                                                      • v4.5.0.0
                                                                                                                                                                                                                                                                      • Unravel 4.4
                                                                                                                                                                                                                                                                        • v4.4.2.1
                                                                                                                                                                                                                                                                          • Unravel 4.4.2.0
                                                                                                                                                                                                                                                                            • Unravel 4.4.1.1
                                                                                                                                                                                                                                                                              • Unravel 4.4.1.0
                                                                                                                                                                                                                                                                                • Unravel 4.4.0
                                                                                                                                                                                                                                                                                • Unravel 4.3
                                                                                                                                                                                                                                                                                  • Unravel 4.3.1.9
                                                                                                                                                                                                                                                                                    • Unravel 4.3.1.8 - Not Available
                                                                                                                                                                                                                                                                                    • Unravel 4.3.1.7
                                                                                                                                                                                                                                                                                      • Unravel 4.3.1.6
                                                                                                                                                                                                                                                                                        • Unravel 4.3.1.5
                                                                                                                                                                                                                                                                                          • Unravel 4.3.1.4
                                                                                                                                                                                                                                                                                            • Unravel 4.3.1.3
                                                                                                                                                                                                                                                                                              • Unravel 4.3.1.2
                                                                                                                                                                                                                                                                                                • Unravel 4.3.1.1
                                                                                                                                                                                                                                                                                                  • v4.3.1.0
                                                                                                                                                                                                                                                                                                  • Unravel 4.2
                                                                                                                                                                                                                                                                                                    • v4.2.7
                                                                                                                                                                                                                                                                                                      • v4.2.6
                                                                                                                                                                                                                                                                                                        • v4.2.5
                                                                                                                                                                                                                                                                                                        • v4.5.4.4
                                                                                                                                                                                                                                                                                                        • Open Source Information
                                                                                                                                                                                                                                                                                                        • SOC 2 compliance

                                                                                                                                                                                                                                                                                                        Home

                                                                                                                                                                                                                                                                                                        • Free Trial
                                                                                                                                                                                                                                                                                                        • Blog
                                                                                                                                                                                                                                                                                                        • Unravel
                                                                                                                                                                                                                                                                                                        • Select version
                                                                                                                                                                                                                                                                                                          • v4.6.0 (Preview)
                                                                                                                                                                                                                                                                                                          • v4.5.x (Current)
                                                                                                                                                                                                                                                                                                          • v4.4.x
                                                                                                                                                                                                                                                                                                          • v4.3.x
                                                                                                                                                                                                                                                                                                          • v4.2.x
                                                                                                                                                                                                                                                                                                          • v4.1.x
                                                                                                                                                                                                                                                                                                        print
                                                                                                                                                                                                                                                                                                        • Prev
                                                                                                                                                                                                                                                                                                        • Next
                                                                                                                                                                                                                                                                                                        •  Home
                                                                                                                                                                                                                                                                                                        • Unravel 4.5 Documentation
                                                                                                                                                                                                                                                                                                        • Advanced topics
                                                                                                                                                                                                                                                                                                        • REST API
                                                                                                                                                                                                                                                                                                        • Operations API

                                                                                                                                                                                                                                                                                                        Operations API

                                                                                                                                                                                                                                                                                                        /clusters

                                                                                                                                                                                                                                                                                                        /clusters/{cluster_name}/nodes

                                                                                                                                                                                                                                                                                                        /clusters/{cluster_name}/resources/cpu/allocated

                                                                                                                                                                                                                                                                                                        /clusters/{cluster_name}/resources/cpu/total

                                                                                                                                                                                                                                                                                                        /clusters/{cluster_name}/resources/memory/allocated

                                                                                                                                                                                                                                                                                                        /clusters/{cluster_name}/resources/memory/total

                                                                                                                                                                                                                                                                                                        /clusters/nodes

                                                                                                                                                                                                                                                                                                        /clusters/resources/cpu/allocated

                                                                                                                                                                                                                                                                                                        /clusters/resources/cpu/total

                                                                                                                                                                                                                                                                                                        /clusters/resources/memory/allocated

                                                                                                                                                                                                                                                                                                        /clusters/resources/memory/total

                                                                                                                                                                                                                                                                                                        /clusters/resources/tagged/{query_type}?applicationType={value}

                                                                                                                                                                                                                                                                                                        /clusters/resources/{resource_type}/{query_type}?to={time}&from={time}&interval={polling_interval}

                                                                                                                                                                                                                                                                                                        In this section:

                                                                                                                                                                                                                                                                                                        Search results

                                                                                                                                                                                                                                                                                                          No results found

                                                                                                                                                                                                                                                                                                          Would you like to provide feedback? Just click here to suggest edits.

                                                                                                                                                                                                                                                                                                          • Prev
                                                                                                                                                                                                                                                                                                          • Next
                                                                                                                                                                                                                                                                                                          © 2016-2020 Unravel Data Inc. All rights reserved.
                                                                                                                                                                                                                                                                                                          Last modified: 2019-07-02
                                                                                                                                                                                                                                                                                                          Publication date: 2020-05-15