Home

Cluster reports

These endpoints provide detailed data on various ongoing cluster activities, memory and vCore usage by cluster, queue, application, and user, job status, and recent alerts.

/clusters

Gets a list of clusters.

Request
GET http://unravel-host:3000/api/v1/clusters
Path parameters

None.

Query parameters

None.

Response body

Name

Type

Description

cluster-name

string

Cluster name.

Examples

Request:

curl -X GET "http://myserver.com:3000/api/v1/clusters" -H "accept: application/json" -H  "Authorization: JWT token"

Response:

[
  "QACDH601A"
]
/clusters/{cluster_name}/nodes

Gets a time series of all the nodes of given cluster grouped by node state.

Request
GET http://unravel-host:3000/api/v1/clusters/{cluster_name}/nodes?from={timestamp}&to={timestamp}
Path parameters

Name

Description

cluster_name

Cluster name.

Query parameters

Required parameters are highlighted.

Name

Type

Description

from

string

Start date. Format YYYY-MM-DD

to

string

End date. Format YYYY-MM-DD

Response body
{
  "total": [
    {
      "ts": 0
    }
  ],
  "active": [
    {
      "ts": 0
    }
  ],
  "unhealthy": [
    {
      "ts": 0
    }
  ],
  "lost": [
    {
      "ts": 0
    }
  ],
  "rebooted": [
    {
      "ts": 0
    }
  ],
  "decommissioned": [
    {
      "ts": 0
    }
  ]
}
Examples

Request:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/QACDH601A/nodes?from=2019-03-28&to=2019-06-28" -H  "accept: application/json" -H  "Authorization: JWT token"

Response body:

{
  "date": [
    1561536000000,
    1561539600000,
    1561543200000,
    1561546800000,
    1561550400000,
    1561554000000,
    1561557600000,
    1561561200000,
    1561564800000,
    1561568400000,
    1561572000000,
    1561575600000,
    1561579200000,
    1561582800000,
    1561586400000,
    1561590000000,
    1561593600000,
    1561597200000,
    1561600800000,
    1561604400000,
    1561608000000,
    1561611600000,
    1561615200000
  ],
  "total": {
    "1561536000000": 1,
    "1561539600000": 1,
    "1561543200000": 1,
    "1561546800000": 1,
    "1561550400000": 1,
    "1561554000000": 1,
    "1561557600000": 1,
    "1561561200000": 1,
    "1561564800000": 1,
    "1561568400000": 1,
    "1561572000000": 1,
    "1561575600000": 1,
    "1561579200000": 1,
    "1561582800000": 1,
    "1561586400000": 1,
    "1561590000000": 1,
    "1561593600000": 1,
    "1561597200000": 1,
    "1561600800000": 1,
    "1561604400000": 1,
    "1561608000000": 1,
    "1561611600000": 1,
    "1561615200000": 1
  },
  "active": {
    "1561536000000": 1,
    "1561539600000": 1,
    "1561543200000": 1,
    "1561546800000": 1,
    "1561550400000": 1,
    "1561554000000": 1,
    "1561557600000": 1,
    "1561561200000": 1,
    "1561564800000": 1,
    "1561568400000": 1,
    "1561572000000": 1,
    "1561575600000": 1,
    "1561579200000": 1,
    "1561582800000": 1,
    "1561586400000": 1,
    "1561590000000": 1,
    "1561593600000": 1,
    "1561597200000": 1,
    "1561600800000": 1,
    "1561604400000": 1,
    "1561608000000": 1,
    "1561611600000": 1,
    "1561615200000": 1
  },
  "lost": {
    "1561536000000": 0,
    "1561539600000": 0,
    "1561543200000": 0,
    "1561546800000": 0,
    "1561550400000": 0,
    "1561554000000": 0,
    "1561557600000": 0,
    "1561561200000": 0,
    "1561564800000": 0,
    "1561568400000": 0,
    "1561572000000": 0,
    "1561575600000": 0,
    "1561579200000": 0,
    "1561582800000": 0,
    "1561586400000": 0,
    "1561590000000": 0,
    "1561593600000": 0,
    "1561597200000": 0,
    "1561600800000": 0,
    "1561604400000": 0,
    "1561608000000": 0,
    "1561611600000": 0,
    "1561615200000": 0
  },
  "unhealthy": {
    "1561536000000": 0,
    "1561539600000": 0,
    "1561543200000": 0,
    "1561546800000": 0,
    "1561550400000": 0,
    "1561554000000": 0,
    "1561557600000": 0,
    "1561561200000": 0,
    "1561564800000": 0,
    "1561568400000": 0,
    "1561572000000": 0,
    "1561575600000": 0,
    "1561579200000": 0,
    "1561582800000": 0,
    "1561586400000": 0,
    "1561590000000": 0,
    "1561593600000": 0,
    "1561597200000": 0,
    "1561600800000": 0,
    "1561604400000": 0,
    "1561608000000": 0,
    "1561611600000": 0,
    "1561615200000": 0
  },
  "decommissioned": {
    "1561536000000": 0,
    "1561539600000": 0,
    "1561543200000": 0,
    "1561546800000": 0,
    "1561550400000": 0,
    "1561554000000": 0,
    "1561557600000": 0,
    "1561561200000": 0,
    "1561564800000": 0,
    "1561568400000": 0,
    "1561572000000": 0,
    "1561575600000": 0,
    "1561579200000": 0,
    "1561582800000": 0,
    "1561586400000": 0,
    "1561590000000": 0,
    "1561593600000": 0,
    "1561597200000": 0,
    "1561600800000": 0,
    "1561604400000": 0,
    "1561608000000": 0,
    "1561611600000": 0,
    "1561615200000": 0
  },
  "rebooted": {
    "1561536000000": 0,
    "1561539600000": 0,
    "1561543200000": 0,
    "1561546800000": 0,
    "1561550400000": 0,
    "1561554000000": 0,
    "1561557600000": 0,
    "1561561200000": 0,
    "1561564800000": 0,
    "1561568400000": 0,
    "1561572000000": 0,
    "1561575600000": 0,
    "1561579200000": 0,
    "1561582800000": 0,
    "1561586400000": 0,
    "1561590000000": 0,
    "1561593600000": 0,
    "1561597200000": 0,
    "1561600800000": 0,
    "1561604400000": 0,
    "1561608000000": 0,
    "1561611600000": 0,
    "1561615200000": 0
  }
}
/clusters/{cluster_name}/resources/cpu/total

Gets a time series for total CPU usage in a given cluster.

Request
GET http://unravel-host:3000/api/v1/clusters/{cluster_name}/resources/cpu/total?from={timestamp}&to={timestamp}
Path parameters

Name

Description

cluster_name

Cluster name.

Query parameters

Required parameters are highlighted.

Name

Type

Description

from

string

Start date. Format YYYY-MM-DD

to

string

End date. Format YYYY-MM-DD

Response body
Examples

Request:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/QACDH601A/resources/cpu/total?from=2019-03-28&to=2019-06-28" -H  "accept: application/json" -H  "Authorization: JWT token"

Response body:

{
  "1561536000000": 24,
  "1561539600000": 24,
  "1561543200000": 24,
  "1561546800000": 24,
  "1561550400000": 24,
  "1561554000000": 24,
  "1561557600000": 24,
  "1561561200000": 24,
  "1561564800000": 24,
  "1561568400000": 24,
  "1561572000000": 24,
  "1561575600000": 24,
  "1561579200000": 24,
  "1561582800000": 24,
  "1561586400000": 24,
  "1561590000000": 24,
  "1561593600000": 24,
  "1561597200000": 24,
  "1561600800000": 24,
  "1561604400000": 24,
  "1561608000000": 24,
  "1561611600000": 24,
  "1561615200000": 24
}
/clusters/{cluster_name}/resources/cpu/allocated

Gets a time series for allocated CPU in a given cluster.

Request
GET http://unravel-host:3000/api/v1/clusters/{cluster_name}/resources/cpu/allocated?from={timestamp}&to={timestamp}

Name

Description

cluster_name

Cluster name.

Query parameters

Required parameters are highlighted.

Name

Type

Description

from

string

Start date. Format YYYY-MM-DD

to

string

End date. Format YYYY-MM-DD

Response body

The response contains time series of CPU usage in the given cluster.

Examples

Request:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/QACDH601A/resources/cpu/allocated?from=2019-03-28&to=2019-06-28" -H  "accept: application/json" -H  "Authorization: JWT token"

Response body:

{
  "1561539600000": "2.875",
  "1561543200000": "1.5"
}
/clusters/{cluster_name}/resources/memory/total

Gets a time series for total memory usage in a given cluster.

Request
GET http://unravel-host:3000/api/v1/clusters/{cluster_name}/resources/memory/total?from={timestamp}&to={timestamp}
Path parameters

Name

Description

cluster_name

Cluster name.

Query parameters

Required parameters are highlighted.

Name

Type

Description

from

string

Start date. Format YYYY-MM-DD

to

string

End date. Format YYYY-MM-DD

Response body
Examples

Request:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/QACDH601A/resources/memory/total?from=2019-03-28&to=2019-06-28" -H  "accept: application/json" -H  "Authorization: JWT token"

Response body:

{
  "1561536000000": 14663,
  "1561539600000": 14663,
  "1561543200000": 14663,
  "1561546800000": 14663,
  "1561550400000": 14663,
  "1561554000000": 14663,
  "1561557600000": 14663,
  "1561561200000": 14663,
  "1561564800000": 14663,
  "1561568400000": 14663,
  "1561572000000": 14663,
  "1561575600000": 14663,
  "1561579200000": 14663,
  "1561582800000": 14663,
  "1561586400000": 14663,
  "1561590000000": 14663,
  "1561593600000": 14663,
  "1561597200000": 14663,
  "1561600800000": 14663,
  "1561604400000": 14663,
  "1561608000000": 14663,
  "1561611600000": 14663,
  "1561615200000": 14663
}
/clusters/{cluster_name}/resources/memory/allocated

Gets a time series for allocated memory in a given cluster.

Request
GET http://unravel-host:3000/api/v1/clusters/{cluster_name}/resources/memory/allocated?from={timestamp}&to={timestamp}
Path parameters

Name

Description

cluster_name

Cluster name.

Query parameters

Required parameters are highlighted.

Name

Type

Description

from

string

Start date. Format YYYY-MM-DD

to

string

End date. Format YYYY-MM-DD

Response body
Examples

Request:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/QACDH601A/resources/memory/allocated?from=2019-03-28&to=2019-06-28" -H  "accept: application/json" -H  "Authorization: JWT token"

Response body:

{
  "1561539600000": "4416",
  "1561543200000": "2304"
}
/clusters/nodes

Gets a summary of cluster nodes.

Request
GET http://unravel-host:3000/api/v1/clusters/nodes/start_time=(date}&end_time={date}
Path parameters

None.

Query parameters

Name

Type

Description

start_time

string

Start date for report.

Format: YYYY-MM-DD

end_time

string

End date for report.

Format: YYYY-MM-DD

Response body
{
  "total": [
    {
      "ts": 0
    }
  ],
  "active": [
    {
      "ts": 0
    }
  ],
  "unhealthy": [
    {
      "ts": 0
    }
  ],
  "lost": [
    {
      "ts": 0
    }
  ],
  "rebooted": [
    {
      "ts": 0
    }
  ],
  "decommissioned": [
    {
      "ts": 0
    }
  ]
}
Examples

Request:

curl -X GET "http://myserver.com:3000/api/v1/clusters/nodes" -H "accept: application/json" -H  "Authorization: JWT token"

Response:

{
  "date": [
    1561611600000,
    1561615200000
  ],
  "total": {
    "1561611600000": 1,
    "1561615200000": 1
  },
  "active": {
    "1561611600000": 1,
    "1561615200000": 1
  },
  "lost": {
    "1561611600000": 0,
    "1561615200000": 0
  },
  "unhealthy": {
    "1561611600000": 0,
    "1561615200000": 0
  },
  "decommissioned": {
    "1561611600000": 0,
    "1561615200000": 0
  },
  "rebooted": {
    "1561611600000": 0,
    "1561615200000": 0
  }
}
/clusters/resources/cpu/allocated

Gets a time series of CPU allocations across all clusters.

Request
GET http://unravel-host:3000/api/v1/clusters/resources/cpu/allocated?from={timestamp}&to={timestamp}
Path parameters

None.

Query parameters

Required parameters are highlighted.

Name

Type

Description

from

string

Start date. Format YYYY-MM-DD

to

string

End date. Format YYYY-MM-DD

Response body
Examples

Request:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/cpu/allocated?from=2019-03-28&to=2019-06-28" -H  "accept: application/json" -H  "Authorization: JWT token"

Response:

{
  "1561539600000": "2.875",
  "1561543200000": "1.5"
}
/clusters/resources/cpu/total

Gets total CPU usage across all clusters.

Request
GET http://unravel-host:3000/api/v1/clusters/resources/cpu/total?from={timestamp}&to={timestamp}
Path parameters

None.

Query parameters

Required parameters are highlighted.

Name

Type

Description

from

string

Start date. Format YYYY-MM-DD

to

string

End date. Format YYYY-MM-DD

Response body
Examples

Request:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/cpu/total?from=2019-03-28&to=2019-06-28" -H  "accept: application/json" -H  "Authorization: JWT token"

Response body:

{
  "1561536000000": 24,
  "1561539600000": 24,
  "1561543200000": 24,
  "1561546800000": 24,
  "1561550400000": 24,
  "1561554000000": 24,
  "1561557600000": 24,
  "1561561200000": 24,
  "1561564800000": 24,
  "1561568400000": 24,
  "1561572000000": 24,
  "1561575600000": 24,
  "1561579200000": 24,
  "1561582800000": 24,
  "1561586400000": 24,
  "1561590000000": 24,
  "1561593600000": 24,
  "1561597200000": 24,
  "1561600800000": 24,
  "1561604400000": 24,
  "1561608000000": 24,
  "1561611600000": 24,
  "1561615200000": 24
}
/clusters/resources/memory/total

Gets total memory usage across all clusters.

Request
GET http://unravel-host:3000/api/v1/clusters/resources/memory/total?from={timestamp}&to={timestamp}
Path parameters

None.

Query parameters

Required parameters are highlighted.

Name

Type

Description

from

string

Start date. Format YYYY-MM-DD

to

string

End date. Format YYYY-MM-DD

Response body
Examples

Request:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/memory/total?from=2019-03-28&to=2019-06-28" -H  "accept: application/json" -H  "Authorization: JWT token"

Response:

{
  "1561536000000": 14663,
  "1561539600000": 14663,
  "1561543200000": 14663,
  "1561546800000": 14663,
  "1561550400000": 14663,
  "1561554000000": 14663,
  "1561557600000": 14663,
  "1561561200000": 14663,
  "1561564800000": 14663,
  "1561568400000": 14663,
  "1561572000000": 14663,
  "1561575600000": 14663,
  "1561579200000": 14663,
  "1561582800000": 14663,
  "1561586400000": 14663,
  "1561590000000": 14663,
  "1561593600000": 14663,
  "1561597200000": 14663,
  "1561600800000": 14663,
  "1561604400000": 14663,
  "1561608000000": 14663,
  "1561611600000": 14663,
  "1561615200000": 14663
}
/clusters/resources/memory/allocated

Gets a time series of memory allocations across all clusters.

Request
GET http://unravel-host:3000/api/v1/clusters/resources/memory/allocated?from={timestamp}&to={timestamp}
Path parameters

None.

Query parameters

Required parameters are highlighted.

Name

Type

Description

from

string

Start date. Format YYYY-MM-DD

to

string

End date. Format YYYY-MM-DD

Response body
Examples

Request:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/memory/allocated?from=2019-03-28&to=2019-06-28" -H  "accept: application/json" -H  "Authorization: JWT token"

Response body:

{
  "1561539600000": "4416",
  "1561543200000": "2304"
}
/clusters/resources/tagged/{query_type}?applicationType={value}
/clusters/resources/tagged/{query_type}?applicationType={value}

Gets total job count per status in a given time window (at one hour intervals).

Request
GET http://unravel-host:3000/api/v1/clusters/resources/tagged/{query_type}?applicationType={value}&from={timestamp}&to={timestamp}&groupBy={type}&interval={polling_interval}
Path parameters

Name

Type

Description

query_type

string

Query type. Valid values are:

  • cpu: Returns vcores as count

  • memory: Returns memory in bytes

Query parameters

Required parameters are highlighted.

Name

Type

Description

from

string

Start time in Unix epoch format.

to

string

End time in Unix epoch format.

applicationType

integer

Application type.

Valid values are: cascading, impala, hive, mr, pig, spark, tez.

interval

integer

Polling interval.

Valid values are: 1m, 5m, 10m, 30m, 1h.

Response body
{
   epoch-timestamp : count
}
Examples

Get vcore usage for MapReduce:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/tagged/cpu?to=1536275883&groupBy=%7B%22type%22:%22applicationType%22,%22value%22:%5B%22MAPREDUCE%22%5D%7D&interval=1h&from=1536189483" -H  "accept: application/json" -H  "Authorization: JWT token"

Get memory usage for Spark:

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/tagged/memory?to=1536275883&groupBy=%7B%22type%22:%22applicationType%22,%22value%22:%5B%22SPARK%22%5D%7D&interval=1h&from=1536189483" -H  "accept: application/json" -H  "Authorization: JWT token"
/clusters/resources/tagged/{query_type}?UserName={list}

Returns vcore/memory usage per user.

Request
GET http://unravel-host:3000/api/v1/clusters/resources/tagged/{query_type}?UserName={list}&from={timestamp}&to={timestamp}&groupBy={type}&interval={polling_interval}
Path Parameters

Name

Type

Description

query_type

string

Query type. Valid values are:

  • cpu: Returns vcores as count

  • memory: Returns memory in bytes

Query Parameters

Name

Type

Description

from

string

Start time in Unix epoch format.

to

string

End time in Unix epoch format.

UserName

integer

List of usernames separated by commas.

interval

integer

Polling interval.

Valid values are: 1m, 5m, 10m, 30m, 1h.

Response Fields
{
   epoch-timestamp : count
}
Examples

Get vcore usage for certain user(s):

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/tagged/cpu?to=1536276842&groupBy=%7B%22type%22:%22user%22,%22value%22:%5B%22root%22%5D%7D&interval=1h&from=1536190442" -H  "accept: application/json" -H  "Authorization: JWT token"

Get memory usage for certain user(s):

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/tagged/memory?to=1536276842&groupBy=%7B%22type%22:%22user%22,%22value%22:%5B%22root%22%5D%7D&interval=1h&from=1536190442" -H  "accept: application/json" -H  "Authorization: JWT token"
/clusters/resources/tagged/{query_type}?QueueName={list}

Returns vcore/memory usage per queue.

Request
GET http://unravel-host:3000/api/v1/clusters/resources/tagged/{query_type}?QueueName={list}&from={timestamp}&to={timestamp}&groupBy={type}&interval={polling_interval}
Path Parameters

Name

Type

Description

query_type

string

Query type. Valid values are:

  • cpu: Returns vcores as count

  • memory: Returns memory in bytes

Query Parameters

Name

Type

Description

from

string

Start time in Unix epoch format.

to

string

End time in Unix epoch format.

QueueName

integer

List of queue names separated by commas.

interval

integer

Polling interval.

Valid values are: 1m, 5m, 10m, 30m, 1h.

Response Fields
{
   epoch-timestamp : count
}
Examples

Get vcore usage for certain queue(s):

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/tagged/cpu?to=1536277362&groupBy=%7B%22type%22:%22queue%22,%22value%22:%5B%22root.users.root%22%5D%7D&interval=1h&from=1536190962" -H  "accept: application/json" -H  "Authorization: JWT token"

Get memory usage for certain queue(s):

curl -X GET "http://http://myserver.com:3000/api/v1/clusters/resources/tagged/memory?to=1536277362&groupBy=%7B%22type%22:%22queue%22,%22value%22:%5B%22root.users.root%22%5D%7D&interval=1h&from=1536190962" -H  "accept: application/json" -H  "Authorization: JWT token"
/clusters/resources/{resource_type}/{query_type}?to={time}&from={time}&interval={polling_interval}

Gets average number of jobs by type.

Request
GET http://unravel-host:3000/api/v1/clusters/resources/{resource_type}/{query_type}?to={time}&from={time}&interval={polling_interval}
Path parameters

Name

Type

Description

resource_type

string

Resource type.

Valid values:

cpu: vcore usage in bytes
memory: memory usage in bytes

query_type

string

Query type.

Valid values:

allocated:
total
Query parameters

Required parameters are highlighted.

Name

Type

Description

interval interval

string

Polling interval.

Valid values are 1m, 5m, 10m, 30m, |1hk

from from

string

Start time in Unix epoch format.

to to

string

End time in Unix epoch format.

Response body
{
   epoch-timestamp : count
}
Examples
Allocated vcores

Request:

curl -X GET "http://myserver.com:3000/api/v1/clusters/resources/cpu/allocated?to=1538666160&interval=30m&from=1536273311" -H "accept: application/json" -H  "Authorization: JWT token"

Response:

{
    "1535360400000": "12", 
    "1535364000000": "3.125", 
    "1535367600000": "5.0303030303"
}
Total memory

Request:

curl -X GET "http://localhost:3000/api/v1/clusters/resources/memory/total?to=1536273841&interval=1h&from=1536187441" -H "accept: application/json" -H  "Authorization: JWT token"

Response:

{
    "1535364000000": 47460,
    "1535367600000": 45087,
    "1535371200000": 37968
}