Cluster reports

These endpoints provide detailed data on various ongoing cluster activities, memory and vCore usage by cluster, queue, application, and user, job status, and recent alerts.

/clusters

Gets a list of clusters.

Request

GET http://unravel-host:3000/api/v1/clusters

Path parameters

None.

Query parameters

None.

Response body

Name		Type	Description
`cluster-name`		string	Cluster name.

Examples

Request:

curl -X GET "http://myserver.com:3000/api/v1/clusters" -H "accept: application/json" -H  "Authorization: JWT token"

Response:

[
  "QACDH601A"
]

/clusters/{cluster_name}/nodes

Gets a time series of all the nodes of given cluster grouped by node state.

Request

GET http://unravel-host:3000/api/v1/clusters/{cluster_name}/nodes?from={timestamp}&to={timestamp}

Path parameters

Name	Description
`cluster_name`	Cluster name.

Query parameters

Required parameters are highlighted.

Name	Type	Description
`from`	string	Start date. Format `YYYY-MM-DD`
`to`	string	End date. Format `YYYY-MM-DD`

Response body

{
  "total": [
    {
      "ts": 0
    }
  ],
  "active": [
    {
      "ts": 0
    }
  ],
  "unhealthy": [
    {
      "ts": 0
    }
  ],
  "lost": [
    {
      "ts": 0
    }
  ],
  "rebooted": [
    {
      "ts": 0
    }
  ],
  "decommissioned": [
    {
      "ts": 0
    }
  ]
}

Examples

Request:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/QACDH601A/nodes?from=2019-03-28&to=2019-06-28" -H  "accept: application/json" -H  "Authorization: JWT token"

Response body:

{
  "date": [
    1561536000000,
    1561539600000,
    1561543200000,
    1561546800000,
    1561550400000,
    1561554000000,
    1561557600000,
    1561561200000,
    1561564800000,
    1561568400000,
    1561572000000,
    1561575600000,
    1561579200000,
    1561582800000,
    1561586400000,
    1561590000000,
    1561593600000,
    1561597200000,
    1561600800000,
    1561604400000,
    1561608000000,
    1561611600000,
    1561615200000
  ],
  "total": {
    "1561536000000": 1,
    "1561539600000": 1,
    "1561543200000": 1,
    "1561546800000": 1,
    "1561550400000": 1,
    "1561554000000": 1,
    "1561557600000": 1,
    "1561561200000": 1,
    "1561564800000": 1,
    "1561568400000": 1,
    "1561572000000": 1,
    "1561575600000": 1,
    "1561579200000": 1,
    "1561582800000": 1,
    "1561586400000": 1,
    "1561590000000": 1,
    "1561593600000": 1,
    "1561597200000": 1,
    "1561600800000": 1,
    "1561604400000": 1,
    "1561608000000": 1,
    "1561611600000": 1,
    "1561615200000": 1
  },
  "active": {
    "1561536000000": 1,
    "1561539600000": 1,
    "1561543200000": 1,
    "1561546800000": 1,
    "1561550400000": 1,
    "1561554000000": 1,
    "1561557600000": 1,
    "1561561200000": 1,
    "1561564800000": 1,
    "1561568400000": 1,
    "1561572000000": 1,
    "1561575600000": 1,
    "1561579200000": 1,
    "1561582800000": 1,
    "1561586400000": 1,
    "1561590000000": 1,
    "1561593600000": 1,
    "1561597200000": 1,
    "1561600800000": 1,
    "1561604400000": 1,
    "1561608000000": 1,
    "1561611600000": 1,
    "1561615200000": 1
  },
  "lost": {
    "1561536000000": 0,
    "1561539600000": 0,
    "1561543200000": 0,
    "1561546800000": 0,
    "1561550400000": 0,
    "1561554000000": 0,
    "1561557600000": 0,
    "1561561200000": 0,
    "1561564800000": 0,
    "1561568400000": 0,
    "1561572000000": 0,
    "1561575600000": 0,
    "1561579200000": 0,
    "1561582800000": 0,
    "1561586400000": 0,
    "1561590000000": 0,
    "1561593600000": 0,
    "1561597200000": 0,
    "1561600800000": 0,
    "1561604400000": 0,
    "1561608000000": 0,
    "1561611600000": 0,
    "1561615200000": 0
  },
  "unhealthy": {
    "1561536000000": 0,
    "1561539600000": 0,
    "1561543200000": 0,
    "1561546800000": 0,
    "1561550400000": 0,
    "1561554000000": 0,
    "1561557600000": 0,
    "1561561200000": 0,
    "1561564800000": 0,
    "1561568400000": 0,
    "1561572000000": 0,
    "1561575600000": 0,
    "1561579200000": 0,
    "1561582800000": 0,
    "1561586400000": 0,
    "1561590000000": 0,
    "1561593600000": 0,
    "1561597200000": 0,
    "1561600800000": 0,
    "1561604400000": 0,
    "1561608000000": 0,
    "1561611600000": 0,
    "1561615200000": 0
  },
  "decommissioned": {
    "1561536000000": 0,
    "1561539600000": 0,
    "1561543200000": 0,
    "1561546800000": 0,
    "1561550400000": 0,
    "1561554000000": 0,
    "1561557600000": 0,
    "1561561200000": 0,
    "1561564800000": 0,
    "1561568400000": 0,
    "1561572000000": 0,
    "1561575600000": 0,
    "1561579200000": 0,
    "1561582800000": 0,
    "1561586400000": 0,
    "1561590000000": 0,
    "1561593600000": 0,
    "1561597200000": 0,
    "1561600800000": 0,
    "1561604400000": 0,
    "1561608000000": 0,
    "1561611600000": 0,
    "1561615200000": 0
  },
  "rebooted": {
    "1561536000000": 0,
    "1561539600000": 0,
    "1561543200000": 0,
    "1561546800000": 0,
    "1561550400000": 0,
    "1561554000000": 0,
    "1561557600000": 0,
    "1561561200000": 0,
    "1561564800000": 0,
    "1561568400000": 0,
    "1561572000000": 0,
    "1561575600000": 0,
    "1561579200000": 0,
    "1561582800000": 0,
    "1561586400000": 0,
    "1561590000000": 0,
    "1561593600000": 0,
    "1561597200000": 0,
    "1561600800000": 0,
    "1561604400000": 0,
    "1561608000000": 0,
    "1561611600000": 0,
    "1561615200000": 0
  }
}

/clusters/{cluster_name}/resources/cpu/total

Gets a time series for total CPU usage in a given cluster.

Request

GET http://unravel-host:3000/api/v1/clusters/{cluster_name}/resources/cpu/total?from={timestamp}&to={timestamp}

Path parameters

Name	Description
`cluster_name`	Cluster name.

Query parameters

Required parameters are highlighted.

Name	Type	Description
`from`	string	Start date. Format `YYYY-MM-DD`
`to`	string	End date. Format `YYYY-MM-DD`

Response body

Examples

Request:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/QACDH601A/resources/cpu/total?from=2019-03-28&to=2019-06-28" -H  "accept: application/json" -H  "Authorization: JWT token"

Response body:

{
  "1561536000000": 24,
  "1561539600000": 24,
  "1561543200000": 24,
  "1561546800000": 24,
  "1561550400000": 24,
  "1561554000000": 24,
  "1561557600000": 24,
  "1561561200000": 24,
  "1561564800000": 24,
  "1561568400000": 24,
  "1561572000000": 24,
  "1561575600000": 24,
  "1561579200000": 24,
  "1561582800000": 24,
  "1561586400000": 24,
  "1561590000000": 24,
  "1561593600000": 24,
  "1561597200000": 24,
  "1561600800000": 24,
  "1561604400000": 24,
  "1561608000000": 24,
  "1561611600000": 24,
  "1561615200000": 24
}

/clusters/{cluster_name}/resources/cpu/allocated

Gets a time series for allocated CPU in a given cluster.

Request

GET http://unravel-host:3000/api/v1/clusters/{cluster_name}/resources/cpu/allocated?from={timestamp}&to={timestamp}

Name	Description
`cluster_name`	Cluster name.

Query parameters

Required parameters are highlighted.

Name	Type	Description
`from`	string	Start date. Format `YYYY-MM-DD`
`to`	string	End date. Format `YYYY-MM-DD`

Response body

The response contains time series of CPU usage in the given cluster.

Examples

Request:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/QACDH601A/resources/cpu/allocated?from=2019-03-28&to=2019-06-28" -H  "accept: application/json" -H  "Authorization: JWT token"

Response body:

{
  "1561539600000": "2.875",
  "1561543200000": "1.5"
}

/clusters/{cluster_name}/resources/memory/total

Gets a time series for total memory usage in a given cluster.

Request

GET http://unravel-host:3000/api/v1/clusters/{cluster_name}/resources/memory/total?from={timestamp}&to={timestamp}

Path parameters

Name	Description
`cluster_name`	Cluster name.

Query parameters

Required parameters are highlighted.

Name	Type	Description
`from`	string	Start date. Format `YYYY-MM-DD`
`to`	string	End date. Format `YYYY-MM-DD`

Response body

Examples

Request:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/QACDH601A/resources/memory/total?from=2019-03-28&to=2019-06-28" -H  "accept: application/json" -H  "Authorization: JWT token"

Response body:

{
  "1561536000000": 14663,
  "1561539600000": 14663,
  "1561543200000": 14663,
  "1561546800000": 14663,
  "1561550400000": 14663,
  "1561554000000": 14663,
  "1561557600000": 14663,
  "1561561200000": 14663,
  "1561564800000": 14663,
  "1561568400000": 14663,
  "1561572000000": 14663,
  "1561575600000": 14663,
  "1561579200000": 14663,
  "1561582800000": 14663,
  "1561586400000": 14663,
  "1561590000000": 14663,
  "1561593600000": 14663,
  "1561597200000": 14663,
  "1561600800000": 14663,
  "1561604400000": 14663,
  "1561608000000": 14663,
  "1561611600000": 14663,
  "1561615200000": 14663
}

/clusters/{cluster_name}/resources/memory/allocated

Gets a time series for allocated memory in a given cluster.

Request

GET http://unravel-host:3000/api/v1/clusters/{cluster_name}/resources/memory/allocated?from={timestamp}&to={timestamp}

Path parameters

Name	Description
`cluster_name`	Cluster name.

Query parameters

Required parameters are highlighted.

Name	Type	Description
`from`	string	Start date. Format `YYYY-MM-DD`
`to`	string	End date. Format `YYYY-MM-DD`

Response body

Examples

Request:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/QACDH601A/resources/memory/allocated?from=2019-03-28&to=2019-06-28" -H  "accept: application/json" -H  "Authorization: JWT token"

Response body:

{
  "1561539600000": "4416",
  "1561543200000": "2304"
}

/clusters/nodes

Gets a summary of cluster nodes.

Request

GET http://unravel-host:3000/api/v1/clusters/nodes/start_time=(date}&end_time={date}

Path parameters

None.

Query parameters

Name	Type	Description
`start_time`	string	Start date for report. Format: YYYY-MM-DD
`end_time`	string	End date for report. Format: YYYY-MM-DD

Name

Type

Description

start_time

string

Start date for report.

Format: YYYY-MM-DD

end_time

string

End date for report.

Format: YYYY-MM-DD

Response body

{
  "total": [
    {
      "ts": 0
    }
  ],
  "active": [
    {
      "ts": 0
    }
  ],
  "unhealthy": [
    {
      "ts": 0
    }
  ],
  "lost": [
    {
      "ts": 0
    }
  ],
  "rebooted": [
    {
      "ts": 0
    }
  ],
  "decommissioned": [
    {
      "ts": 0
    }
  ]
}

Examples

Request:

curl -X GET "http://myserver.com:3000/api/v1/clusters/nodes" -H "accept: application/json" -H  "Authorization: JWT token"

Response:

{
  "date": [
    1561611600000,
    1561615200000
  ],
  "total": {
    "1561611600000": 1,
    "1561615200000": 1
  },
  "active": {
    "1561611600000": 1,
    "1561615200000": 1
  },
  "lost": {
    "1561611600000": 0,
    "1561615200000": 0
  },
  "unhealthy": {
    "1561611600000": 0,
    "1561615200000": 0
  },
  "decommissioned": {
    "1561611600000": 0,
    "1561615200000": 0
  },
  "rebooted": {
    "1561611600000": 0,
    "1561615200000": 0
  }
}

/clusters/resources/cpu/allocated

Gets a time series of CPU allocations across all clusters.

Request

GET http://unravel-host:3000/api/v1/clusters/resources/cpu/allocated?from={timestamp}&to={timestamp}

Path parameters

None.

Query parameters

Required parameters are highlighted.

Name	Type	Description
`from`	string	Start date. Format `YYYY-MM-DD`
`to`	string	End date. Format `YYYY-MM-DD`

Response body

Examples

Request:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/cpu/allocated?from=2019-03-28&to=2019-06-28" -H  "accept: application/json" -H  "Authorization: JWT token"

Response:

{
  "1561539600000": "2.875",
  "1561543200000": "1.5"
}

/clusters/resources/cpu/total

Gets total CPU usage across all clusters.

Request

GET http://unravel-host:3000/api/v1/clusters/resources/cpu/total?from={timestamp}&to={timestamp}

Path parameters

None.

Query parameters

Required parameters are highlighted.

Name	Type	Description
`from`	string	Start date. Format `YYYY-MM-DD`
`to`	string	End date. Format `YYYY-MM-DD`

Response body

Examples

Request:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/cpu/total?from=2019-03-28&to=2019-06-28" -H  "accept: application/json" -H  "Authorization: JWT token"

Response body:

{
  "1561536000000": 24,
  "1561539600000": 24,
  "1561543200000": 24,
  "1561546800000": 24,
  "1561550400000": 24,
  "1561554000000": 24,
  "1561557600000": 24,
  "1561561200000": 24,
  "1561564800000": 24,
  "1561568400000": 24,
  "1561572000000": 24,
  "1561575600000": 24,
  "1561579200000": 24,
  "1561582800000": 24,
  "1561586400000": 24,
  "1561590000000": 24,
  "1561593600000": 24,
  "1561597200000": 24,
  "1561600800000": 24,
  "1561604400000": 24,
  "1561608000000": 24,
  "1561611600000": 24,
  "1561615200000": 24
}

/clusters/resources/memory/total

Gets total memory usage across all clusters.

Request

GET http://unravel-host:3000/api/v1/clusters/resources/memory/total?from={timestamp}&to={timestamp}

Path parameters

None.

Query parameters

Required parameters are highlighted.

Name	Type	Description
`from`	string	Start date. Format `YYYY-MM-DD`
`to`	string	End date. Format `YYYY-MM-DD`

Response body

Examples

Request:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/memory/total?from=2019-03-28&to=2019-06-28" -H  "accept: application/json" -H  "Authorization: JWT token"

Response:

{
  "1561536000000": 14663,
  "1561539600000": 14663,
  "1561543200000": 14663,
  "1561546800000": 14663,
  "1561550400000": 14663,
  "1561554000000": 14663,
  "1561557600000": 14663,
  "1561561200000": 14663,
  "1561564800000": 14663,
  "1561568400000": 14663,
  "1561572000000": 14663,
  "1561575600000": 14663,
  "1561579200000": 14663,
  "1561582800000": 14663,
  "1561586400000": 14663,
  "1561590000000": 14663,
  "1561593600000": 14663,
  "1561597200000": 14663,
  "1561600800000": 14663,
  "1561604400000": 14663,
  "1561608000000": 14663,
  "1561611600000": 14663,
  "1561615200000": 14663
}

/clusters/resources/memory/allocated

Gets a time series of memory allocations across all clusters.

Request

GET http://unravel-host:3000/api/v1/clusters/resources/memory/allocated?from={timestamp}&to={timestamp}

Path parameters

None.

Query parameters

Required parameters are highlighted.

Name	Type	Description
`from`	string	Start date. Format `YYYY-MM-DD`
`to`	string	End date. Format `YYYY-MM-DD`

Response body

Examples

Request:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/memory/allocated?from=2019-03-28&to=2019-06-28" -H  "accept: application/json" -H  "Authorization: JWT token"

Response body:

{
  "1561539600000": "4416",
  "1561543200000": "2304"
}

/clusters/resources/tagged/{query_type}?applicationType={value}

Gets total job count per status in a given time window (at one hour intervals).

Request

GET http://unravel-host:3000/api/v1/clusters/resources/tagged/{query_type}?applicationType={value}&from={timestamp}&to={timestamp}&groupBy={type}&interval={polling_interval}

Path parameters

Name	Type	Description
`query_type`	string	Query type. Valid values are: `cpu`: Returns vcores as count `memory`: Returns memory in bytes

Query parameters

Required parameters are highlighted.

Name	Type	Description
`from`	string	Start time in Unix epoch format.
`to`	string	End time in Unix epoch format.
`applicationType`	integer	Application type. Valid values are: `cascading`, `impala`, `hive`, `mr`, `pig`, `spark`, `tez`.
`interval`	integer	Polling interval. Valid values are: `1m`, `5m`, `10m`, `30m`, `1h`.

Response body

{
   epoch-timestamp : count
}

Examples

Get vcore usage for MapReduce:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/tagged/cpu?to=1536275883&groupBy=%7B%22type%22:%22applicationType%22,%22value%22:%5B%22MAPREDUCE%22%5D%7D&interval=1h&from=1536189483" -H  "accept: application/json" -H  "Authorization: JWT token"

Get memory usage for Spark:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/tagged/memory?to=1536275883&groupBy=%7B%22type%22:%22applicationType%22,%22value%22:%5B%22SPARK%22%5D%7D&interval=1h&from=1536189483" -H  "accept: application/json" -H  "Authorization: JWT token"

/clusters/resources/tagged/{query_type}?UserName={list}

Returns vcore/memory usage per user.

Request

GET http://unravel-host:3000/api/v1/clusters/resources/tagged/{query_type}?UserName={list}&from={timestamp}&to={timestamp}&groupBy={type}&interval={polling_interval}

Path Parameters

Name	Type	Description
`query_type`	string	Query type. Valid values are: `cpu`: Returns vcores as count `memory`: Returns memory in bytes

Query Parameters

Name	Type	Description
`from`	string	Start time in Unix epoch format.
`to`	string	End time in Unix epoch format.
`UserName`	integer	List of usernames separated by commas.
`interval`	integer	Polling interval. Valid values are: `1m`, `5m`, `10m`, `30m`, `1h`.

Response Fields

{
   epoch-timestamp : count
}

Examples

Get vcore usage for certain user(s):

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/tagged/cpu?to=1536276842&groupBy=%7B%22type%22:%22user%22,%22value%22:%5B%22root%22%5D%7D&interval=1h&from=1536190442" -H  "accept: application/json" -H  "Authorization: JWT token"

Get memory usage for certain user(s):

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/tagged/memory?to=1536276842&groupBy=%7B%22type%22:%22user%22,%22value%22:%5B%22root%22%5D%7D&interval=1h&from=1536190442" -H  "accept: application/json" -H  "Authorization: JWT token"

/clusters/resources/tagged/{query_type}?QueueName={list}

Returns vcore/memory usage per queue.

Request

GET http://unravel-host:3000/api/v1/clusters/resources/tagged/{query_type}?QueueName={list}&from={timestamp}&to={timestamp}&groupBy={type}&interval={polling_interval}

Path Parameters

Name	Type	Description
`query_type`	string	Query type. Valid values are: `cpu`: Returns vcores as count `memory`: Returns memory in bytes

Query Parameters

Name	Type	Description
`from`	string	Start time in Unix epoch format.
`to`	string	End time in Unix epoch format.
`QueueName`	integer	List of queue names separated by commas.
`interval`	integer	Polling interval. Valid values are: `1m`, `5m`, `10m`, `30m`, `1h`.

Response Fields

{
   epoch-timestamp : count
}

Examples

Get vcore usage for certain queue(s):

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/tagged/cpu?to=1536277362&groupBy=%7B%22type%22:%22queue%22,%22value%22:%5B%22root.users.root%22%5D%7D&interval=1h&from=1536190962" -H  "accept: application/json" -H  "Authorization: JWT token"

Get memory usage for certain queue(s):

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/tagged/memory?to=1536277362&groupBy=%7B%22type%22:%22queue%22,%22value%22:%5B%22root.users.root%22%5D%7D&interval=1h&from=1536190962" -H  "accept: application/json" -H  "Authorization: JWT token"

/clusters/resources/{resource_type}/{query_type}?to={time}&from={time}&interval={polling_interval}

Gets average number of jobs by type.

Request

GET http://unravel-host:3000/api/v1/clusters/resources/{resource_type}/{query_type}?to={time}&from={time}&interval={polling_interval}

Path parameters

Name

Type

Description

resource_type

string

Resource type.

Valid values:

cpu: vcore usage in bytes

memory: memory usage in bytes

query_type

string

Query type.

Valid values:

allocated:

total

Query parameters

Required parameters are highlighted.

Name	Type	Description
`interval` `interval`	string	Polling interval. Valid values are 1m, 5m, 10m, 30m, \|1hk
`from` `from`	string	Start time in Unix epoch format.
`to` `to`	string	End time in Unix epoch format.

Name

Type

Description

interval interval

string

Polling interval.

Valid values are 1m, 5m, 10m, 30m, |1hk

from from

string

Start time in Unix epoch format.

to to

string

End time in Unix epoch format.

Response body

{
   epoch-timestamp : count
}

Examples

Allocated vcores

Request:

curl -X GET "http://myserver.com:3000/api/v1/clusters/resources/cpu/allocated?to=1538666160&interval=30m&from=1536273311" -H "accept: application/json" -H  "Authorization: JWT token"

Response:

{
    "1535360400000": "12", 
    "1535364000000": "3.125", 
    "1535367600000": "5.0303030303"
}

Total memory

Request:

curl -X GET "http://localhost:3000/api/v1/clusters/resources/memory/total?to=1536273841&interval=1h&from=1536187441" -H "accept: application/json" -H  "Authorization: JWT token"

Response:

{
    "1535364000000": 47460,
    "1535367600000": 45087,
    "1535371200000": 37968
}

In this section:

Home

Cluster reports

/clusters

/clusters/{cluster_name}/nodes

/clusters/{cluster_name}/resources/cpu/total

/clusters/{cluster_name}/resources/cpu/allocated

/clusters/{cluster_name}/resources/memory/total

/clusters/{cluster_name}/resources/memory/allocated

/clusters/nodes

/clusters/resources/cpu/allocated

/clusters/resources/cpu/total

/clusters/resources/memory/total

/clusters/resources/memory/allocated

/clusters/resources/tagged/{query_type}?applicationType={value}

/clusters/resources/{resource_type}/{query_type}?to={time}&from={time}&interval={polling_interval}

Search results