These endpoints return key performance indicators (KPIs) for clusters, the small files report, chargeback reports (by user, queue, app), and cluster workload reports.
Under construction.
Under construction.
Under construction.
Under construction.
Under construction.
Gets the latest small file report.
GET http://unravel-host
:3000/api/v1/reports/data/diskusage/get_latest_reports?task_name=small_files_report
None.
Name | Type | Description |
---|---|---|
| string | Task name. Set this to |
{ "date": date created EPOCH_timestamp, "isSuccess": report generation sucess/failure, // report parameters "avg_size_threshold": small file size in bytes, "num_files_threshold": minimum number of small files, "top_n_small_files": # of directories to show, "report_id": report name", "root": [ // array[top_n_small files] directory { "MaxFilesize": maximum size file in directory, "MinFilesize": minimum size file in directory, "NumFiles": # of small files in the directory, "DirPath": directory path, "TotalFilesize": sum of file size in directory, "AvgFilesize": average file size } ] }
Request:
curl -X GET "http://myserver.com:3000/api/v1/reports/data/diskusage/get_latest_reports?task_name=small_files_report" -H "accept: application/json" -H "Authorization: JWT token
"
Response:
{ "date": 1536899695, "isSuccess": true, "avg_size_threshold": 100000, "num_files_threshold": 100, "top_n_small_files": 5, "report_id": "small_files_1536927781_5674", "root": [ { "MaxFilesize": "0", "MinFilesize": "0", "NumFiles": "21523359", "DirPath": "\/97ovg", "TotalFilesize": "0", "AvgFilesize": "0" }, { "MaxFilesize": "0", "MinFilesize": "0", "NumFiles": "11302185", "DirPath": "\/98esm", "TotalFilesize": "0", "AvgFilesize": "0" }, { "MaxFilesize": "0", "MinFilesize": "0", "NumFiles": "7174452", "DirPath": "\/99acx", "TotalFilesize": "0", "AvgFilesize": "0" }, { "MaxFilesize": "0", "MinFilesize": "0", "NumFiles": "7174452", "DirPath": "\/97ovg\/97zqu", "TotalFilesize": "0", "AvgFilesize": "0" }, { "MaxFilesize": "0", "MinFilesize": "0", "NumFiles": "7174452", "DirPath": "\/97ovg\/99kgg", "TotalFilesize": "0", "AvgFilesize": "0" } ] }
Under construction.
Generates a Key Performance Indicator (KPI) report.
GET http://unravel-host
:3000/api/v1/reports/data/kpis?numDays=num-days
None.
Required parameters are highlighted
.
Name | Type | Description |
---|---|---|
| integer | Number of days to include. |
Name | Type | Description |
---|---|---|
| integer | Start time in Unix epoch format. |
| integer | End time in Unix epoch format. |
| integer | Number of tables accessed. |
| integer | Number of partitions accessed. |
| integer | Number of queries accessing the table. |
| integer | Total Read I/O due to accessing the tables. |
| integer | Number of tables created. |
| integer | Number of partitions created. |
| integer | Size of tables created. |
| integer | Size of partitions created. |
| integer | Total number of tables in the system. |
| integer | Total number of partitions in the system. |
| integer | |
| integer | |
| integer | |
| integer | |
| integer | |
| integer | |
| integer | |
| integer | |
| integer | |
| array | Comma-separated list of users. |
Request:
curl -X GET "http://myserver.com:3000/api/v1/reports/data/kpis?numDays=1" -H "accept: application/json" -H "Authorization: JWT token
"
Response:
[ { "st": 1536795705, "et": 1536882105, "nlaTb": 15, "nlaPr": 0, "nlaQr": 57, "nlaRi": 101489411821, "nlcTb": 7, "nlcPr": 0, "nlcTz": 304768890, "nlcPz": 0, "ntoTb": 378, "ntoPr": 30061, "nhtTb": 19, "nwaTb": 0, "ncoTb": 359, "nhtPr": 1823, "nwaPr": 0, "ncoPr": 28238, "rp": 30061, "rs": 92544293666, "fs": 92544293666, "users": [ "root", "hdfs" ] } ]
Gets details about the small file report.
GET http://unravel-host
:3000/api/v1/reports/data/small_file_report_details
None.
None.
Name | Type | Description |
---|---|---|
[no name] | array | JSON structure containing details about the small file report. |
Request:
curl -X GET "http://myserver.com:3000/api/v1/reports/data/small_file_report_details" -H "accept: application/json" -H "Authorization: JWT token
"
Under construction.
Gets a summary of the cluster's users, apps, or queues.
GET http://unravel-host
:3000/api/v1/reports/operational/clusterstats?st={timestamp}&et={timestamp}&mode={summary_type}
None.
Required parameters are highlighted
.
Name | Type | Description |
---|---|---|
| string | Start time, in Unix epoch format. |
| string | End time, in Unix epoch format. |
| string | Summary type. Valid values are |
JSON schema when mode
is user
or queue
:
{ "userStats": [ // array of users | queues currently on cluster { "root": { "running": { "min": minimum applications running,, "max": minimum applications running, "mean": average appication number running, "stddev": standard deviation }, "memory": { // see running above }, "pending": { // see running above }, "vcores": { // see running above } } } ] }
Request with mode=user
:
curl -X GET "http://myserver.com:3000/api/v1/reports/operational/clusterstats?st=1536658189000&et=1536744589000&mode=user" -H "accept: application/json" -H "Authorization: JWT token
"
Response for mode=user
:
{ "userStats": [ { "root": { "running": { "min": 1, "max": 1, "mean": 1, "stddev": 0 }, "memory": { "min": 1024, "max": 16384, "mean": 7040, "stddev": 7099.0974074174 }, "pending": { "min": 0, "max": 0, "mean": 0, "stddev": 0 }, "vcores": { "min": 1, "max": 3, "mean": 2, "stddev": 0.81649658092773 } } } ] }
Request with mode=queue
:
curl -X GET "http://myserver.com:3000/api/v1/reports/operational/clusterstats?st=1536658189000&et=1536744589000&mode=queue" -H "accept: application/json" -H "Authorization: JWT token
"
Response for mode=queue
:
{ "queueStats": [ { "root.users.root": { "running": { "min": 1, "max": 1, "mean": 1, "stddev": 0 }, "memory": { "min": 1024, "max": 16384, "mean": 7040, "stddev": 7099.0974074174 }, "pending": { "min": 0, "max": 0, "mean": 0, "stddev": 0 }, "vcores": { "min": 1, "max": 3, "mean": 2, "stddev": 0.81649658092773 } } } ] }
Response for mode=app
:
{ [ appID : application id, type : memorySeconds/vcoreSeconds vcore : vcore value memory : memory value ] }
Gets a summary of the cluster's workloads.
GET http://unravel-host
:3000/api/v1/reports/operational/clusterworkload?gte={timestamp}<e={timestamp}&reportBy={interval}
None.
Required parameters are highlighted
.
Name | Type | Description |
---|---|---|
| string | Start time, in Unix epoch format. |
| string | End time, in Unix epoch format. |
| string | Report interval. Valid values are |
// work load by month one of more months with the application count for month // minimum of one month { timestamp: appcount[,timestamp: appcount] } // work load by hour array of 25 hours [ { timestamp: appcount }, ... { timestamp: appcount } ] // work load by day - array for each days contained with the time period (Mon - Sun) // minimum of one day [ { timestamp: appcount } ] // work load by hour/day:array for each day (Mon - Sun) by hour // minimum of 24 hours for one day [ { timestamp: appcount } ]
Request:
curl -X GET "http://myserver.com:3000/api/v1/reports/operational/clusterworkload?gte=1536777000Z<e=1536863400Z&reportBy=month" -H "accept: application/json" -H "Authorization: JWT token
"
Response:
{"1536777000":96,"1536863400":4} [ {"1536813000000":0},{"1536816600000":0},{"1536820200000":0},{"1536823800000":10},{"1536827400000":7},{"1536831000000":13},{"1536834600000":10},{"1536838200000":31},{"1536841800000":0},{"1536845400000":0},{"1536849000000":4},{"1536852600000":2},{"1536856200000":0},{"1536859800000":0},{"1536863400000":0},{"1536867000000":0},{"1536870600000":0},{"1536874200000":0},{"1536877800000":0},{"1536881400000":0},{"1536885000000":0},{"1536888600000":0},{"1536892200000":0},{"1536895800000":6},{"1536899400000":2}] [ {"1536777000000":96},{"1536863400000":7} ] [{"1536813000000":0},{"1536816600000":0},{"1536820200000":0},{"1536823800000":10},{"1536827400000":7},{"1536831000000":13},{"1536834600000":10},{"1536838200000":31},{"1536841800000":0},{"1536845400000":0},{"1536849000000":4},{"1536852600000":2},{"1536856200000":0},{"1536859800000":0},{"1536863400000":0},{"1536867000000":0},{"1536870600000":0},{"1536874200000":0},{"1536877800000":0},{"1536881400000":0},{"1536885000000":0},{"1536888600000":0},{"1536892200000":0},{"1536895800000":6},{"1536899400000":2}]
Under construction.
Under construction.
Under construction.