Home

AutoAction endpoints
/autoactions

Gets a list of all active and inactive AutoActions.

Request
GET http://unravel-host:3000/api/v1/autoactions?enabled={boolean} 
Path parameters

None.

Query parameters

Required parameters are highlighted.

Name

Type

Description

enabled

boolean

Filters by active or inactive status.

Valid values:

true: Show only active AutoActions
false: Show only inactive AutoActions
Response body

Name

Type

Description

enabled

boolean

true: active AutoAction
false: inactive AutoAction

admin

boolean

Indicates an admin has access the AutoAction template. This is always true.

policy_name

string

Defined by Unravel. Default: AutoAction2.

policy_id

string

Defined by Unravel. Default: 10.

instance_id

string

An unique ID associated with the AutoAction; generated by Unravel.

name_by_user

string

AutoAction name assigned by the user who defined the AutoAction.

description_by_user

string

Description assigned by the user.

created_by

string

User who created the AutoAction.

last_edited_by

string

Last user who edited the AutoAction.

created_at

string

The time the AutoAction was created. Format: Unix epoch format.

updated_at

string

The time the AutoAction was last edited. Format: Unix epoch format.

rules

array

The AutoActions rules. When there are two or more trules are defined they are evaulated in conjunction using either AND, OR, or SAME, in other words, rule 1 AND rule2.

Logical Operator

array

Logic operator for evaulating multiple rules. AND, OR, or SAME. See SAME logical operator for comprehensive description and example of the SAME operator.

scope

string

The scope the violation must take place in: User, Queue, Cluster, or App Name,

metric

string

Metric to use for comparision. You can obtain the list of available metrics using the AutoAction Metrics API or in the User Guide SAME page.

compare

string

>, >=, ==, <, <;=

state

string

State the app must be in for the violation to occurr. Valid states are: new, new_saving, submitted, accepted, scheduled, allocated, allocatedSaving, launched, running, finishing, finished, killed, failed, undefined, newAny, allocatedAny, pending, and * (all).

value

string

Value for comparision, for example, allocated memory <= 10.

actions

array

Actions to take upon violation. If no actions are specified the violation is logged.

cluster_mode

integer

Mode to use for cluster scope evaluation. The violation can only occur if it falls within the scope as defined below.

0:: Cluster scope is not being used.
1: All clusters (the cluster_list is irrelevant).
2: Only the clusters listed.
3: All clusters except the clusters listed.

cluster_list

array

List of cluster names.

cluster_transform

string

Regular expression (regex) to be used to generate names for the cluster_list. All names which match the expression are added to the cluster_list and used when evaluating the scope as defined by cluster_mode.

queue_mode

integer

Mode to use for queue scope evaluation. The violation can only occur if it falls within the scope as defined below.

0: Queue scope is not being used.
1: All queues (the queue_list is irrelevant).
2: Only the queues listed.
3: All clusters except the queues listed are considered.

queue_list

array

List of queue names.

queue_transform

integer

Regular expression (regex) to be used to generate names for the queue_list. All names which match the expression are added to the queue_list All names which match the expression are added to the cluster_list and used when evaluating the scope as defined by queue_mode.

user_mode

integer

Mode to use for user evaluation. The action is triggered for:

0: User scope is not being used.
1: All users (the user_list is irrelevant).
2: Only the users listed.
3: All users except the users listed.

user_list

array

List of user names.

user_transform

integer

Regular expression (regex) to be used to generate names for the user_list. All names which match the expression are added to the user_list and used when evaluating the scope as defined by user_mode.

app_mode

integer

Mode to use for app evaluation. The action is triggered for:

0: App scope is not being used
1: All apps (the app_listis irrelevant).
2: Only the apps listed are considered.
3: All apps except the apps listed are considered.

app_list

array

List of app names.

app_transform

integer

Regular expression (regex) to be used to generate names for the app_list. All names which match the expression are added to the user_list and used when evaluating the scope as defined by app_mode.

sustain_mode

integer

The length of time a violation must continually occur before the AutoAction is triggered, allowing time for the violator to "self correct".

0: Off.
1: On.

sustain_time

integer

Time period to use when sustain_modeis on.

time_mode

integer

The time period during which the AutoAction is active, i.e., can be triggered. It is always 2.

1: AutoAction is always active.
2: AutoAction is active daily..
Examples

Fetch active AutoActions:

curl -X GET "http://node.companh.com:3000/api/v1/autoactions?enabled=true" -H"accept: application/json" -H "accept: application/json" -H  "Authorization: JWT 
			token"
		

Response body:

[
  {
   "enabled": true,
   "admin": true,
   "policy_name": "AutoAction2",
   "policy_id": 10,
   "instance_id": "7952742540109537242",
   "name_by_user": "check for queue cluster user ",
   "description_by_user": "",
   "created_by": "admin",
   "last_edited_by": "admin",
   "created_at": 1579173259517,
   "updated_at": 1579503284485,
   "rules": [
     {
      "scope": "user",]
      "metric": "appCount",
      "compare": ">",]
      "state": "new",]"
      "value": 34
     }
   ],
   "actions": [
    ],
   "cluster_mode": 2,
   "cluster_list": [
     "default",
     "QACDH63B1"
    ],
    "cluster_transform": "",
    "queue_mode": 2,
    "queue_list": [
      "root.unravel",
      "default",
      "root.users.unravel"
    ],
    "queue_transform": "",
    "user_mode": 2,
    "user_list": [
      "unravel@UNRAVELDATA.COM"
    ],
    "user_transform": "",
    "app_mode": 2,  
    "app_list": [
      "appi1"
    ],
    "app_transform": "",
    "sustain_mode": 0,
    "sustain_time": 0,
    "time_mode": 2
  }
]
/autoactions/metrics

Gets a list of AutoAction metrics.

Request
GET http://unravel-host:3000/api/v1/autoactions/metrics
Path parameters

None.

Query parameters

None.

Response body

Name

Type

Description

[no name]

array

JSON structure containing a list of metrics.

Examples

Request:

curl -X GET "http://myserver.com:3000/api/v1/autoactions/metrics" -H  "accept: application/json" -H  "Authorization: JWT token"

Response body:

[
  "appCount",
  "elapsedTime",
  "allocatedMB",
  "allocatedVCores",
  "runningContainers",
  "memorySeconds",
  "vcoreSeconds",
  "jobCount",
  "elapsedAppTime",
  "mapsTotal",
  "mapsCompleted",
  "reducesTotal",
  "reducesCompleted",
  "mapsPending",
  "mapsRunning",
  "reducesPending",
  "reducesRunning",
  "newReduceAttempts",
  "runningReduceAttempts",
  "failedReduceAttempts",
  "killedReduceAttempts",
  "successfulReduceAttempts",
  "newMapAttempts",
  "runningMapAttempts",
  "failedMapAttempts",
  "killedMapAttempts",
  "successfulMapAttempts",
  "badId",
  "connection",
  "ioError",
  "wrongLength",
  "wrongMap",
  "wrongReduce",
  "fileBytesRead",
  "fileBytesWritten",
  "fileReadOps",
  "fileLargeReadOps",
  "fileWriteOps",
  "hdfsBytesRead",
  "hdfsBytesWritten",
  "hdfsReadOps",
  "hdfsLargeReadOps",
  "hdfsWriteOps",
  "mapInputRecords",
  "mapOutputRecords",
  "mapOutputBytes",
  "mapOutputMaterializedBytes",
  "splitRawBytes",
  "combineInputRecords",
  "combineOutputRecords",
  "reduceInputGroups",
  "reduceShuffleBytes",
  "reduceInputRecords",
  "reduceOutputRecords",
  "spilledRecords",
  "shuffledMaps",
  "failedShuffle",
  "mergedMapOutputs",
  "gcTimeMillis",
  "cpuMilliseconds",
  "physicalMemoryBytes",
  "virtualMemoryBytes",
  "committedHeapBytes",
  "totalLaunchedMaps",
  "totalLaunchedReduces",
  "dataLocalMaps",
  "slotsMillisMaps",
  "slotsMillisReduces",
  "millisMaps",
  "millisReduces",
  "vcoresMillisMaps",
  "vcoresMillisReduces",
  "mbMillisMaps",
  "mbMillisReduces",
  "bytesRead",
  "bytesWritten",
  "duration",
  "totalDfsBytesRead",
  "totalDfsBytesWritten",
  "inputRecords",
  "outputRecords",
  "outputToInputRecordsRatio",
  "totalJoinInputRowCount",
  "totalJoinOutputRowCount",
  "inputPartitions",
  "outputPartitions",
  "joinInputRowCount",
  "joinOutputRowCount",
  "joinOutputToInputRowRatio"
]
/autoactions/recent_violations

Gets a list of recent AutoAction violations.

Request
GET http://unravel-host:3000/api/v1/autoactions/recent_violations?start_time={timestamp}&end_time={timestamp}&limit={integer}"
Path parameters

None.

Query parameters

Required parameters are highlighted.

Name

Type

Description

start_time

string

Starting date of date range. Format: YYYY-MM-DD.

end_time

string

Ending date of date range. Format: YYYY-MM-DD.

limit

integer

Maximum number of violations to return.

Default: 100

Response body

Name

Type

Description

[violation]

array

JSON structure containing a list of violations

Violation

Name

Type

Description

entityType

integer

Application type:

1: Hive
2: MapReduce
3: Spark

clusterName

string

Cluster name on which the AutAction violation occurred.

appId

string

App Id which violated the AutoAction.

eventTime

integer

Time of the violation in Unix epoch format.

eventName

string

AutoAction name assigned at creation.

entityId

string

Each violation has an entityId assigned by Unravel.

user

string

The user who created the AutoAction, typically admin.

queue

string

The queue name on which the AuotAction violation occurred.

Examples

Request:

curl -X GET "http://tnode75.unraveldata.com:3000/api/v1/autoactions/recent_violations?start_time=2019-11-01&end_time=2019-11-31&limit=1" -H "accept: application/json" -H  "Authorization: JWT token"

Response body:

[
  {
   "entityType": 0,
   "clusterName": "N/A",
   "appId": "http://node.company.com:3000/#/app/operations/charts/impala?from=1578397319224&to=1578397919224&at=1578397619224&interval=1m",
   "eventTime": 1578397619224,
   "eventName": "Long running impala query",
   "entityId": "8496537073757152411",
   "user": "N/A",
   "queue": "N/A"
   }
]
/autoactions/recent_violations?eventType=AAVA

Gets a list of recent AutoAction violations filtered by app type.

Request
GET http://unravel-host:3000/api/v1/autoactions/recent_violations?eventType=AAVA?start_time={timestamp}&end_time={timestamp}&limit={integer}"
Path parameters

None.

Query parameters

Required parameters are highlighted.

Name

Type

Description

User

string

The app's owner.

appId

string

The app's Id.

appType

string

The type of app. You must specify one or more of the following. Use a comma separated list to specify multiple app types.

0: MR
1: Hive
2: Spark
13: Workflows
15: Tez
16: Impala

start_time

string

Starting date of date range. Format: YYYY-MM-DD.

end_time

string

Ending date of date range. Format: YYYY-MM-DD.

limit

integer

Maximum number of violations to return.

Default: 100

Response body

Name

Type

Description

[violation]

array

JSON structure containing a list of violations grouped by event type.

Violation

Name

Type

Description

entityType

integer

App type.

0: MR
1: Hive
2: Spark
13: Workflows
15: Tez
16: Impala

clusterName

string

Cluster the violating app was on at time of violation.

appId

string

Violating app's Id. This is unique to the app.

eventTime

integer

Time the violation occurred. (Unix epoch format)

eventName

string

Name assigned when AutoAction was created.

entityId

string

An unique ID associated with the AutoAction; generated by Unravel.

user

string

Username of AutoAction's creator.

queue

string

Queue the violating app was in at time of violation.

Examples

Request:

curl -X GET "http://node.company.com:3000/api/v1/autoactions/recent_violations?start_time=2019-11-01&end_time=2019-11-31&limit=1" -H "accept: application/json" -H  "Authorization: JWT token"

Response body:

[
  {
   "entityType": 0,
   "clusterName": "N/A",
   "appId": "http://node.company.com:3000/#/app/operations/charts/impala?from=1578397319224&to=1578397919224&at=1578397619224&interval=1m",
   "eventTime": 1578397619224,
   "eventName": "Long running impala query",
   "entityId": "8496537073757152411",
   "user": "N/A",
   "queue": "N/A"
   }
]
/autoactions/templates?is_enabled=true&is_admin=true

Gets a list of Unravel's predefined AutoAction templates.

Request
GET http://unravel-host:3000/api/v1/autoactions/templates?is_enabled={boolean}&is_admin=true
Path parameters

None.

Query parameters

Name

Type

Description

is_enabled

boolean

This must be set to true.

is_admin

boolean

This must be set to true.

Response Body

Name

Type

Description

[violation]

array

JSON structure containing a list of violations grouped by event type.

Template

Name

Type

Value

admin

boolean

true

enabled

boolean

true

policy_name

string

AutoAction 2

policy_id

string

10

instance_id

string

Unique id of violation generated by Unravel.

name_by_user

string

AutoAction name, for instance Resource contention in cluster (allocated memory).

description_by_user

string

 

created_by

string

admin

last_edited_by

string

admin

created_at

integer

Time, in Unix epoch format, tempalte was cretaed.

updated_at

string

Time, in Unix epoch format, template was last edited.

rules

Note: The rules are defined based upon the purpose of the template, for example, Resource contention in cluster (allocated memory).

array

One or more rules. If more than one, the rules must be logically combined

Logical operator

AND, OR, or SAME when more than one rule

scope

string

 

metric

string

compare

string

 

state

string

value

string

actions

array

Actions to take upon violation

cluster_mode

integer

1

cluster_list

array

 

queue_mode

integer

0

queue_list

array

 

app_mode

integer

0

applist

array

 

user_mode

integer

0

user_list

array

 

sustain_mode

integer

sustain_time

integer

time_mode

integer

start_hour

integer

0

start__min

integer

0

end_hour

integer

0

end_min

integer

0

dt_zone

integer

America/Los_Angeles

tool_tip

integer

This reflect the purpose of the template. For example, "AutoAction policy for a cluster where the application running in this cluster has allocated memory exceeding the specified threshold"

Examples

Request:

curl -X GET "http://no.company.com:3000/api/v1/autoactions/templates?is_enabled=true&is_admin=true" -H "accept: application/json "Authorization: JWT token"

Response body:

[
  {
    "admin": true,
    "enabled": true,
    "policy_name": "AutoAction2",
    "policy_id": 10,
    "instance_id": "8876002422463862009",
    "name_by_user": "Resource contention in cluster (allocated memory)",
    "description_by_user": "",
    "created_by": "admin",
    "last_edited_by": "admin",
    "created_at": 1516155266280,
    "updated_at": 1516155344636,
    "rules": [
      {
        "SAME": [
          {
            "scope": "clusters",
            "metric": "allocatedMB",
            "compare": ">",
            "value": 0
          },
          {
            "scope": "clusters",
            "metric": "appCount",
            "compare": ">",
            "state": "pending",
            "value": 0
          }
        ]
      }
    ],
    "actions": [
    ],
    "cluster_mode": 1,
    "cluster_list": [
      
    ],
    "queue_mode": 0,
    "queue_list": [
      
    ],
    "app_mode": 0,
    "app_list": [
      
    ],
    "user_mode": 0,
    "user_list": [
      
    ],
    "time_mode": 0,
    "start_hour": 0,
    "start_min": 0,
    "end_hour": 0,
    "end_min": 0,
    "dt_zone": "America/Los_Angeles",
    "tooltip": "AutoAction policy for a cluster where the application running in this cluster has allocated memory exceeding the specified threshold"
  } 
]