Creating AutoActions
Note
See here for limitations on AutoActions.
Select Manage > AutoActions.
The tab displays all defined AutoActions separated into an Active and Inactive list. You can enable/disable an AutoAction by clicking the check box on the right side of its row. You can edit or delete an AutoAction regardless of its status by clicking edit (
) or delete (
) on the right side of the row.
To create a new AutoAction click either Create from Template, Build Rule, or Expert Rule.
Create from Template: Provides a partially completed template designed for the task. The only ruleset options available are relevant to the task being defined. Fields that you need to fill in are highlighted.
Build Rule: Provides an empty template, with all the options available.
Expert Rule: provides only a text box for defining your AutoAction using JSON.
Create an AutoAction.
See AutoActions Templates for descriptions and use of the template's sections.
Using Create From Template
Click Create from Template. All the available templates are displayed. Click the template you wish to use.
Resource contention
Resource contention in cluster: Monitors overall memory or vCore usage on a cluster and the number of pending/running jobs in order to detect when the cluster is struggling to accommodate all submitted jobs and falls into a resource contention pitfall. Once detected, the specified AutoAction is taken.
Resource contention in queue: Monitors overall memory or vCore usage in a queue and number of pending/running jobs in order to detect a state when the queue is struggling to accommodate all submitted jobs and falls into a resource contention pitfall. Once detected, the specified action is taken.
Rogue identification
Rogue user: Identifies so-called “rogue” users on a cluster who can potentially affect other users and the cluster as a whole, that is, users submitting jobs that use too much of the cluster resources (memory or vCores). Once the rogue user is detected, the specified action is taken.
Rogue app: Identifies so-called “rogue” apps on a cluster which can potentially affect other apps and the cluster as a whole, that is, apps using too much of the cluster resources (memory or vCores). Once the rogue app is detected, the specified action is taken.
Rogue Impala query (HDFS Read/Write) - identifies so-called “rogue” Impala queries on a cluster which can potentially affect other apps and the cluster as a whole, that is, queries using too much of the cluster resources (memory or vCores). Once the rogue app is detected, the specified action is taken.
Long-running Jobs
Long-running YARN app: Monitors elapsed time of a running YARN app, MapReduce, Spark, Hive, etc., and executes the action if the job runs longer than desired.
Long-running Hive query: Tracks Hive jobs by monitoring the total elapsed time of a running Hive query and executes the action if the job runs longer than desired.
Long-running workflow: Tracks workflow jobs by monitoring the total elapsed time of a running workflow and executes the action if the job runs longer than desired.
Long-Impala query: Tracks Impala jobs by monitoring the total elapsed time of a running workflow and executes the action if the job runs longer than desired.
Complete the template.
Note
Changing the predefined ruleset can result in the action not behaving as anticipated.
Using Build Rule
Click Build Rule and complete the template.
Using Expert Rule
The AutoActions engine is capable of much more than is available through the templates. Expert Rule is a very powerful mode giving you total access the engine's capabilities. You can create complex rulesets, using JSON, to accommodate almost any cluster monitoring requirements.
Before using this mode, you should have a clear understanding of AutoActions concepts. See the Expert Rule section for a detailed explanation on how to define an AutoAction. See the Sample AutoActions section for examples of rules and actions.
Click Save AutoAction.
Your AutoAction is now listed in the Manage > AutoActions tab.