AutoAction endpoints

/autoactions

Gets a list of all active and inactive AutoActions.

Request

GET http://unravel-host:3000/api/v1/autoactions?enabled={boolean}

Path parameters

None.

Query parameters

Required parameters are highlighted.

Name

Type

Description

enabled

boolean

Filters by active or inactive status.

Valid values:

true: Show only active AutoActions

false: Show only inactive AutoActions

Response body

Name

Type

Description

enabled

boolean

true: active AutoAction

false: inactive AutoAction

admin

boolean

Indicates an admin has access the AutoAction template. This is always true.

policy_name

string

Defined by Unravel. Default: AutoAction2.

policy_id

string

Defined by Unravel. Default: 10.

instance_id

string

An unique ID associated with the AutoAction; generated by Unravel.

name_by_user

string

AutoAction name assigned by the user who defined the AutoAction.

description_by_user

string

Description assigned by the user.

created_by

string

User who created the AutoAction.

last_edited_by

string

Last user who edited the AutoAction.

created_at

string

The time the AutoAction was created. Format: Unix epoch format.

updated_at

string

The time the AutoAction was last edited. Format: Unix epoch format.

rules

array

The AutoActions rules. When there are two or more trules are defined they are evaulated in conjunction using either AND, OR, or SAME, in other words, rule 1 AND rule2.

Logical Operator

array

Logic operator for evaulating multiple rules. AND, OR, or SAME. See SAME logical operator for comprehensive description and example of the SAME operator.

scope

string

The scope the violation must take place in: User, Queue, Cluster, or App Name,

metric

string

Metric to use for comparision. You can obtain the list of available metrics using the AutoAction Metrics API or in the User Guide SAME page.

compare

string

>, >=, ==, <, <;=

state

string

State the app must be in for the violation to occurr. Valid states are: new, new_saving, submitted, accepted, scheduled, allocated, allocatedSaving, launched, running, finishing, finished, killed, failed, undefined, newAny, allocatedAny, pending, and * (all).

value

string

Value for comparision, for example, allocated memory <= 10.

actions

array

Actions to take upon violation. If no actions are specified the violation is logged.

cluster_mode

integer

Mode to use for cluster scope evaluation. The violation can only occur if it falls within the scope as defined below.

0:: Cluster scope is not being used.

1: All clusters (the cluster_list is irrelevant).

2: Only the clusters listed.

3: All clusters except the clusters listed.

cluster_list

array

List of cluster names.

cluster_transform

string

Regular expression (regex) to be used to generate names for the cluster_list. All names which match the expression are added to the cluster_list and used when evaluating the scope as defined by cluster_mode.

queue_mode

integer

Mode to use for queue scope evaluation. The violation can only occur if it falls within the scope as defined below.

0: Queue scope is not being used.

1: All queues (the queue_list is irrelevant).

2: Only the queues listed.

3: All clusters except the queues listed are considered.

queue_list

array

List of queue names.

queue_transform

integer

Regular expression (regex) to be used to generate names for the queue_list. All names which match the expression are added to the queue_list All names which match the expression are added to the cluster_list and used when evaluating the scope as defined by queue_mode.

user_mode

integer

Mode to use for user evaluation. The action is triggered for:

0: User scope is not being used.

1: All users (the user_list is irrelevant).

2: Only the users listed.

3: All users except the users listed.

user_list

array

List of user names.

user_transform

integer

Regular expression (regex) to be used to generate names for the user_list. All names which match the expression are added to the user_list and used when evaluating the scope as defined by user_mode.

app_mode

integer

Mode to use for app evaluation. The action is triggered for:

0: App scope is not being used

1: All apps (the app_listis irrelevant).

2: Only the apps listed are considered.

3: All apps except the apps listed are considered.

app_list

array

List of app names.

app_transform

integer

Regular expression (regex) to be used to generate names for the app_list. All names which match the expression are added to the user_list and used when evaluating the scope as defined by app_mode.

sustain_mode

integer

The length of time a violation must continually occur before the AutoAction is triggered, allowing time for the violator to "self correct".

0: Off.

1: On.

sustain_time

integer

Time period to use when sustain_modeis on.

time_mode

integer

The time period during which the AutoAction is active, i.e., can be triggered. It is always 2.

1: AutoAction is always active.

2: AutoAction is active daily..

Examples

Fetch active AutoActions:

curl -X GET "http://node.companh.com:3000/api/v1/autoactions?enabled=true" -H"accept: application/json" -H "accept: application/json" -H  "Authorization: JWT 
			token"

Response body:

[
  {
   "enabled": true,
   "admin": true,
   "policy_name": "AutoAction2",
   "policy_id": 10,
   "instance_id": "7952742540109537242",
   "name_by_user": "check for queue cluster user ",
   "description_by_user": "",
   "created_by": "admin",
   "last_edited_by": "admin",
   "created_at": 1579173259517,
   "updated_at": 1579503284485,
   "rules": [
     {
      "scope": "user",]
      "metric": "appCount",
      "compare": ">",]
      "state": "new",]"
      "value": 34
     }
   ],
   "actions": [
    ],
   "cluster_mode": 2,
   "cluster_list": [
     "default",
     "QACDH63B1"
    ],
    "cluster_transform": "",
    "queue_mode": 2,
    "queue_list": [
      "root.unravel",
      "default",
      "root.users.unravel"
    ],
    "queue_transform": "",
    "user_mode": 2,
    "user_list": [
      "unravel@UNRAVELDATA.COM"
    ],
    "user_transform": "",
    "app_mode": 2,  
    "app_list": [
      "appi1"
    ],
    "app_transform": "",
    "sustain_mode": 0,
    "sustain_time": 0,
    "time_mode": 2
  }
]

/autoactions/metrics

Gets a list of AutoAction metrics.

Request

GET http://unravel-host:3000/api/v1/autoactions/metrics

Path parameters

None.

Query parameters

None.

Response body

Name	Type	Description
[no name]	array	JSON structure containing a list of metrics.

Examples

Request:

curl -X GET "http://myserver.com:3000/api/v1/autoactions/metrics" -H  "accept: application/json" -H  "Authorization: JWT token"

Response body:

[
  "appCount",
  "elapsedTime",
  "allocatedMB",
  "allocatedVCores",
  "runningContainers",
  "memorySeconds",
  "vcoreSeconds",
  "jobCount",
  "elapsedAppTime",
  "mapsTotal",
  "mapsCompleted",
  "reducesTotal",
  "reducesCompleted",
  "mapsPending",
  "mapsRunning",
  "reducesPending",
  "reducesRunning",
  "newReduceAttempts",
  "runningReduceAttempts",
  "failedReduceAttempts",
  "killedReduceAttempts",
  "successfulReduceAttempts",
  "newMapAttempts",
  "runningMapAttempts",
  "failedMapAttempts",
  "killedMapAttempts",
  "successfulMapAttempts",
  "badId",
  "connection",
  "ioError",
  "wrongLength",
  "wrongMap",
  "wrongReduce",
  "fileBytesRead",
  "fileBytesWritten",
  "fileReadOps",
  "fileLargeReadOps",
  "fileWriteOps",
  "hdfsBytesRead",
  "hdfsBytesWritten",
  "hdfsReadOps",
  "hdfsLargeReadOps",
  "hdfsWriteOps",
  "mapInputRecords",
  "mapOutputRecords",
  "mapOutputBytes",
  "mapOutputMaterializedBytes",
  "splitRawBytes",
  "combineInputRecords",
  "combineOutputRecords",
  "reduceInputGroups",
  "reduceShuffleBytes",
  "reduceInputRecords",
  "reduceOutputRecords",
  "spilledRecords",
  "shuffledMaps",
  "failedShuffle",
  "mergedMapOutputs",
  "gcTimeMillis",
  "cpuMilliseconds",
  "physicalMemoryBytes",
  "virtualMemoryBytes",
  "committedHeapBytes",
  "totalLaunchedMaps",
  "totalLaunchedReduces",
  "dataLocalMaps",
  "slotsMillisMaps",
  "slotsMillisReduces",
  "millisMaps",
  "millisReduces",
  "vcoresMillisMaps",
  "vcoresMillisReduces",
  "mbMillisMaps",
  "mbMillisReduces",
  "bytesRead",
  "bytesWritten",
  "duration",
  "totalDfsBytesRead",
  "totalDfsBytesWritten",
  "inputRecords",
  "outputRecords",
  "outputToInputRecordsRatio",
  "totalJoinInputRowCount",
  "totalJoinOutputRowCount",
  "inputPartitions",
  "outputPartitions",
  "joinInputRowCount",
  "joinOutputRowCount",
  "joinOutputToInputRowRatio"
]

/autoactions/recent_violations

Gets a list of recent AutoAction violations.

Request

GET http://unravel-host:3000/api/v1/autoactions/recent_violations?start_time={timestamp}&end_time={timestamp}&limit={integer}"

Path parameters

None.

Query parameters

Required parameters are highlighted.

Name	Type	Description
`start_time`	string	Starting date of date range. Format: `YYYY-MM-DD`.
`end_time`	string	Ending date of date range. Format: `YYYY-MM-DD`.
`limit`	integer	Maximum number of violations to return. Default: 100

Name

Type

Description

start_time

string

Starting date of date range. Format: YYYY-MM-DD.

end_time

string

Ending date of date range. Format: YYYY-MM-DD.

limit

integer

Maximum number of violations to return.

Default: 100

Response body

Name	Type	Description
[violation]	array	JSON structure containing a list of violations

Violation

Name

Type

Description

entityType

integer

Application type:

1: Hive

2: MapReduce

3: Spark

clusterName

string

Cluster name on which the AutAction violation occurred.

appId

string

App Id which violated the AutoAction.

eventTime

integer

Time of the violation in Unix epoch format.

eventName

string

AutoAction name assigned at creation.

entityId

string

Each violation has an entityId assigned by Unravel.

user

string

The user who created the AutoAction, typically admin.

queue

string

The queue name on which the AuotAction violation occurred.

Examples

Request:

curl -X GET "http://tnode75.unraveldata.com:3000/api/v1/autoactions/recent_violations?start_time=2019-11-01&end_time=2019-11-31&limit=1" -H "accept: application/json" -H  "Authorization: JWT token"

Response body:

[
  {
   "entityType": 0,
   "clusterName": "N/A",
   "appId": "http://node.company.com:3000/#/app/operations/charts/impala?from=1578397319224&to=1578397919224&at=1578397619224&interval=1m",
   "eventTime": 1578397619224,
   "eventName": "Long running impala query",
   "entityId": "8496537073757152411",
   "user": "N/A",
   "queue": "N/A"
   }
]

/autoactions/recent_violations?eventType=AAVA

Gets a list of recent AutoAction violations filtered by app type.

Request

GET http://unravel-host:3000/api/v1/autoactions/recent_violations?eventType=AAVA?start_time={timestamp}&end_time={timestamp}&limit={integer}"

Path parameters

None.

Query parameters

Required parameters are highlighted.

Name

Type

Description

User

string

The app's owner.

appId

string

The app's Id.

appType

string

The type of app. You must specify one or more of the following. Use a comma separated list to specify multiple app types.

0: MR

1: Hive

2: Spark

13: Workflows

15: Tez

16: Impala

start_time

string

Starting date of date range. Format: YYYY-MM-DD.

end_time

string

Ending date of date range. Format: YYYY-MM-DD.

limit

integer

Maximum number of violations to return.

Default: 100

Response body

Name	Type	Description
[violation]	array	JSON structure containing a list of violations grouped by event type.

Violation

Name

Type

Description

entityType

integer

App type.

0: MR

1: Hive

2: Spark

13: Workflows

15: Tez

16: Impala

clusterName

string

Cluster the violating app was on at time of violation.

appId

string

Violating app's Id. This is unique to the app.

eventTime

integer

Time the violation occurred. (Unix epoch format)

eventName

string

Name assigned when AutoAction was created.

entityId

string

An unique ID associated with the AutoAction; generated by Unravel.

user

string

Username of AutoAction's creator.

queue

string

Queue the violating app was in at time of violation.

Examples

Request:

curl -X GET "http://node.company.com:3000/api/v1/autoactions/recent_violations?start_time=2019-11-01&end_time=2019-11-31&limit=1" -H "accept: application/json" -H  "Authorization: JWT token"

Response body:

[
  {
   "entityType": 0,
   "clusterName": "N/A",
   "appId": "http://node.company.com:3000/#/app/operations/charts/impala?from=1578397319224&to=1578397919224&at=1578397619224&interval=1m",
   "eventTime": 1578397619224,
   "eventName": "Long running impala query",
   "entityId": "8496537073757152411",
   "user": "N/A",
   "queue": "N/A"
   }
]

/autoactions/templates?is_enabled=true&is_admin=true

Gets a list of Unravel's predefined AutoAction templates.

Request

GET http://unravel-host:3000/api/v1/autoactions/templates?is_enabled={boolean}&is_admin=true

Path parameters

None.

Query parameters

Name	Type	Description
`is_enabled`	boolean	This must be set to true.
`is_admin`	boolean	This must be set to true.

Response Body

Name	Type	Description
[violation]	array	JSON structure containing a list of violations grouped by event type.

Template

Name			Type	Value
`admin`			boolean	`true`
`enabled`			boolean	`true`
`policy_name`			string	`AutoAction 2`
`policy_id`			string	10
`instance_id`			string	Unique id of violation generated by Unravel.
`name_by_user`			string	AutoAction name, for instance Resource contention in cluster (allocated memory).
`description_by_user`			string
`created_by`			string	admin
`last_edited_by`			string	admin
`created_at`			integer	Time, in Unix epoch format, tempalte was cretaed.
`updated_at`			string	Time, in Unix epoch format, template was last edited.
`rules` Note: The rules are defined based upon the purpose of the template, for example, Resource contention in cluster (allocated memory).			array	One or more rules. If more than one, the rules must be logically combined
	Logical operator			AND, OR, or SAME when more than one rule
		`scope`	string
		`metric`	string
		`compare`	string
		`state`	string
		`value`	string
`actions`			array	Actions to take upon violation
`cluster_mode`			integer	1
`cluster_list`			array
`queue_mode`			integer	0
`queue_list`			array
`app_mode`			integer	0
`applist`			array
`user_mode`			integer	0
`user_list`			array
`sustain_mode`			integer
`sustain_time`			integer
`time_mode`			integer
`start_hour`			integer	0
`start__min`			integer	0
`end_hour`			integer	0
`end_min`			integer	0
`dt_zone`			integer	America/Los_Angeles
`tool_tip`			integer	This reflect the purpose of the template. For example, "AutoAction policy for a cluster where the application running in this cluster has allocated memory exceeding the specified threshold"

Examples

Request:

curl -X GET "http://no.company.com:3000/api/v1/autoactions/templates?is_enabled=true&is_admin=true" -H "accept: application/json "Authorization: JWT token"

Response body:

[
  {
    "admin": true,
    "enabled": true,
    "policy_name": "AutoAction2",
    "policy_id": 10,
    "instance_id": "8876002422463862009",
    "name_by_user": "Resource contention in cluster (allocated memory)",
    "description_by_user": "",
    "created_by": "admin",
    "last_edited_by": "admin",
    "created_at": 1516155266280,
    "updated_at": 1516155344636,
    "rules": [
      {
        "SAME": [
          {
            "scope": "clusters",
            "metric": "allocatedMB",
            "compare": ">",
            "value": 0
          },
          {
            "scope": "clusters",
            "metric": "appCount",
            "compare": ">",
            "state": "pending",
            "value": 0
          }
        ]
      }
    ],
    "actions": [
    ],
    "cluster_mode": 1,
    "cluster_list": [
      
    ],
    "queue_mode": 0,
    "queue_list": [
      
    ],
    "app_mode": 0,
    "app_list": [
      
    ],
    "user_mode": 0,
    "user_list": [
      
    ],
    "time_mode": 0,
    "start_hour": 0,
    "start_min": 0,
    "end_hour": 0,
    "end_min": 0,
    "dt_zone": "America/Los_Angeles",
    "tooltip": "AutoAction policy for a cluster where the application running in this cluster has allocated memory exceeding the specified threshold"
  } 
]

In this section:

Home

AutoAction endpoints

/autoactions

/autoactions/metrics

/autoactions/recent_violations

/autoactions/recent_violations?eventType=AAVA

/autoactions/templates?is_enabled=true&is_admin=true

Search results