Report endpoints
These endpoints return key performance indicators (KPIs) for clusters, the small files report, chargeback reports (by user, queue, app), and cluster workload reports.
Cluster reports
Chargeback reports
/reports/archives/get_report_archives
Under construction.
/reports/celeryreports
Under construction.
/reports/celeryreports/metadata
Under construction.
/reports/clusteroptimization/metadata
Under construction.
/reports/data/diskusage/capacity_task_details
Under construction.
/reports/data/diskusage/get_latest_reports
Gets the latest small file report.
GET http://unravel-host:3000/api/v1/reports/data/diskusage/get_latest_reports?task_name=small_files_reportNone.
Name | Type | Description |
|---|---|---|
| string | Task name. Set this to |
{
"date": date created EPOCH_timestamp,
"isSuccess": report generation sucess/failure,
// report parameters
"avg_size_threshold": small file size in bytes,
"num_files_threshold": minimum number of small files,
"top_n_small_files": # of directories to show,
"report_id": report name",
"root": [
// array[top_n_small files] directory
{
"MaxFilesize": maximum size file in directory,
"MinFilesize": minimum size file in directory,
"NumFiles": # of small files in the directory,
"DirPath": directory path,
"TotalFilesize": sum of file size in directory,
"AvgFilesize": average file size
}
]
}Request:
curl -X GET "http://myserver.com:3000/api/v1/reports/data/diskusage/get_latest_reports?task_name=small_files_report" -H "accept: application/json" -H "Authorization: JWT token"Response:
{
"date": 1536899695,
"isSuccess": true,
"avg_size_threshold": 100000,
"num_files_threshold": 100,
"top_n_small_files": 5,
"report_id": "small_files_1536927781_5674",
"root": [
{
"MaxFilesize": "0",
"MinFilesize": "0",
"NumFiles": "21523359",
"DirPath": "\/97ovg",
"TotalFilesize": "0",
"AvgFilesize": "0"
},
{
"MaxFilesize": "0",
"MinFilesize": "0",
"NumFiles": "11302185",
"DirPath": "\/98esm",
"TotalFilesize": "0",
"AvgFilesize": "0"
},
{
"MaxFilesize": "0",
"MinFilesize": "0",
"NumFiles": "7174452",
"DirPath": "\/99acx",
"TotalFilesize": "0",
"AvgFilesize": "0"
},
{
"MaxFilesize": "0",
"MinFilesize": "0",
"NumFiles": "7174452",
"DirPath": "\/97ovg\/97zqu",
"TotalFilesize": "0",
"AvgFilesize": "0"
},
{
"MaxFilesize": "0",
"MinFilesize": "0",
"NumFiles": "7174452",
"DirPath": "\/97ovg\/99kgg",
"TotalFilesize": "0",
"AvgFilesize": "0"
}
]
}/reports/data/diskusage/get_latest_success_reports
Under construction.
/reports/data/kpis
Generates a Key Performance Indicator (KPI) report.
GET http://unravel-host:3000/api/v1/reports/data/kpis?numDays=num-days
None.
Required parameters are highlighted.
Name | Type | Description |
|---|---|---|
| integer | Number of days to include. |
Name | Type | Description |
|---|---|---|
| integer | Start time in Unix epoch format. |
| integer | End time in Unix epoch format. |
| integer | Number of tables accessed. |
| integer | Number of partitions accessed. |
| integer | Number of queries accessing the table. |
| integer | Total Read I/O due to accessing the tables. |
| integer | Number of tables created. |
| integer | Number of partitions created. |
| integer | Size of tables created. |
| integer | Size of partitions created. |
| integer | Total number of tables in the system. |
| integer | Total number of partitions in the system. |
| integer | |
| integer | |
| integer | |
| integer | |
| integer | |
| integer | |
| integer | |
| integer | |
| integer | |
| array | Comma-separated list of users. |
Request:
curl -X GET "http://myserver.com:3000/api/v1/reports/data/kpis?numDays=1" -H "accept: application/json" -H "Authorization: JWT token"Response:
[
{
"st": 1536795705,
"et": 1536882105,
"nlaTb": 15,
"nlaPr": 0,
"nlaQr": 57,
"nlaRi": 101489411821,
"nlcTb": 7,
"nlcPr": 0,
"nlcTz": 304768890,
"nlcPz": 0,
"ntoTb": 378,
"ntoPr": 30061,
"nhtTb": 19,
"nwaTb": 0,
"ncoTb": 359,
"nhtPr": 1823,
"nwaPr": 0,
"ncoPr": 28238,
"rp": 30061,
"rs": 92544293666,
"fs": 92544293666,
"users": [
"root",
"hdfs"
]
}
]/reports/data/small_file_report_details
Gets details about the small file report.
GET http://unravel-host:3000/api/v1/reports/data/small_file_report_detailsNone.
None.
Name | Type | Description |
|---|---|---|
[no name] | array | JSON structure containing details about the small file report. |
Request:
curl -X GET "http://myserver.com:3000/api/v1/reports/data/small_file_report_details" -H "accept: application/json" -H "Authorization: JWT token"/reports/files/reports/latest
Under construction.
/reports/operational/clusterstats?st={timestamp}&et={timestamp}&mode={summary_type}
Gets a summary of the cluster's users, apps, or queues.
GET http://unravel-host:3000/api/v1/reports/operational/clusterstats?st={timestamp}&et={timestamp}&mode={summary_type}None.
Required parameters are highlighted.
Name | Type | Description |
|---|---|---|
| string | Start time, in Unix epoch format. |
| string | End time, in Unix epoch format. |
| string | Summary type. Valid values are |
JSON schema when mode is user or queue:
{
"userStats": [
// array of users | queues currently on cluster
{
"root": {
"running": {
"min": minimum applications running,,
"max": minimum applications running,
"mean": average appication number running,
"stddev": standard deviation
},
"memory": { // see running above },
"pending": { // see running above },
"vcores": { // see running above }
}
}
]
}Request with mode=user:
curl -X GET "http://myserver.com:3000/api/v1/reports/operational/clusterstats?st=1536658189000&et=1536744589000&mode=user" -H "accept: application/json" -H "Authorization: JWT token"Response for mode=user:
{
"userStats": [
{
"root": {
"running": {
"min": 1,
"max": 1,
"mean": 1,
"stddev": 0
},
"memory": {
"min": 1024,
"max": 16384,
"mean": 7040,
"stddev": 7099.0974074174
},
"pending": {
"min": 0,
"max": 0,
"mean": 0,
"stddev": 0
},
"vcores": {
"min": 1,
"max": 3,
"mean": 2,
"stddev": 0.81649658092773
}
}
}
]
}Request with mode=queue:
curl -X GET "http://myserver.com:3000/api/v1/reports/operational/clusterstats?st=1536658189000&et=1536744589000&mode=queue" -H "accept: application/json" -H "Authorization: JWT token"Response for mode=queue:
{
"queueStats": [
{
"root.users.root": {
"running": {
"min": 1,
"max": 1,
"mean": 1,
"stddev": 0
},
"memory": {
"min": 1024,
"max": 16384,
"mean": 7040,
"stddev": 7099.0974074174
},
"pending": {
"min": 0,
"max": 0,
"mean": 0,
"stddev": 0
},
"vcores": {
"min": 1,
"max": 3,
"mean": 2,
"stddev": 0.81649658092773
}
}
}
]
}Response for mode=app:
{
[
appID : application id,
type : memorySeconds/vcoreSeconds
vcore : vcore value
memory : memory value
]
}/reports/operational/clusterworkload?gte={timestamp}<e={timestamp}&reportBy={interval}
Gets a summary of the cluster's workloads.
GET http://unravel-host:3000/api/v1/reports/operational/clusterworkload?gte={timestamp}<e={timestamp}&reportBy={interval}None.
Required parameters are highlighted.
Name | Type | Description |
|---|---|---|
| string | Start time, in Unix epoch format. |
| string | End time, in Unix epoch format. |
| string | Report interval. Valid values are |
// work load by month one of more months with the application count for month
// minimum of one month
{ timestamp: appcount[,timestamp: appcount] }
// work load by hour array of 25 hours
[
{ timestamp: appcount }, ... { timestamp: appcount }
]
// work load by day - array for each days contained with the time period (Mon - Sun)
// minimum of one day
[
{ timestamp: appcount }
]
// work load by hour/day:array for each day (Mon - Sun) by hour
// minimum of 24 hours for one day
[
{ timestamp: appcount }
]
Request:
curl -X GET "http://myserver.com:3000/api/v1/reports/operational/clusterworkload?gte=1536777000Z<e=1536863400Z&reportBy=month" -H "accept: application/json" -H "Authorization: JWT token"Response:
{"1536777000":96,"1536863400":4}
[
{"1536813000000":0},{"1536816600000":0},{"1536820200000":0},{"1536823800000":10},{"1536827400000":7},{"1536831000000":13},{"1536834600000":10},{"1536838200000":31},{"1536841800000":0},{"1536845400000":0},{"1536849000000":4},{"1536852600000":2},{"1536856200000":0},{"1536859800000":0},{"1536863400000":0},{"1536867000000":0},{"1536870600000":0},{"1536874200000":0},{"1536877800000":0},{"1536881400000":0},{"1536885000000":0},{"1536888600000":0},{"1536892200000":0},{"1536895800000":6},{"1536899400000":2}]
[
{"1536777000000":96},{"1536863400000":7}
]
[{"1536813000000":0},{"1536816600000":0},{"1536820200000":0},{"1536823800000":10},{"1536827400000":7},{"1536831000000":13},{"1536834600000":10},{"1536838200000":31},{"1536841800000":0},{"1536845400000":0},{"1536849000000":4},{"1536852600000":2},{"1536856200000":0},{"1536859800000":0},{"1536863400000":0},{"1536867000000":0},{"1536870600000":0},{"1536874200000":0},{"1536877800000":0},{"1536881400000":0},{"1536885000000":0},{"1536888600000":0},{"1536892200000":0},{"1536895800000":6},{"1536899400000":2}]
/reports/queueanalysis/get_latest_report_by_queue_name
Under construction.
/reports/schedules
Under construction.