A report to analyze the spending and wastage on AWS - Elastic Map Reduce (EMR). You can create this report either from the UI or the command line. You can analyze various costs incurred in running EMR clusters on AWS.
Note
The AWS EMR Cost and Wastage report is compatible only with Unravel 4.7.8.0 version.
Click Generate Reports > New button.
In the New Report dialog box, enter the following details.
Click OK. The generated reports will be listed under Reports on the App UI.
Select the generated report and then click Run. After the report is successfully run, the details of the report runs are listed in the Run box on the right.
Click the following:
HTML files link to view the report details.
Input parameters link to view the parameters you chose to run the report.
Log file link to view the logs of the report.
From the terminal, change directories to
unity-one/src
directory.cd /opt/unravel/data/apps/unity-one/src
Use a screen editor, such as vi editor, to edit the
job.json
file and copy-paste the following text into the file.{ "enabled": false, "retention_days": 30, "profile_memory": false, "report_type": "aws_emr_cost", "params": { "start_date": null, "end_date": null, "days": 30, "topk": 20, "cpu_idle_threshold": 30, "memory_idle_threshold": 30, "tags_cost_threshold": 10, "cluster_idle_time_threshold": 50, "all_filters": { "Pillar": null, "username": null } }, "notifications": {} }
Parameters
Description
start_date
Can be null or a valid start date.
end_date
Can be null or a valid start date.
days
The number of days to look back while downloading data.
topk
The number of applications to be shown in the report.
cpu_idle_threshold
The threshold in percentage for the idle utilization of the CPU.
memory_idle_threshold
The threshold in percentage for the idle utilization of memory.
tags_cost_threshold
The cost threshold for the tags in USD.
cluster_idle_time_threshold
The threshold in percentage for the idle time utilization of clusters.
all_filters
Filter options.
Save and exit the screen editor.
Run the following command to generate the report:
./unity-cli.sh report generate job.json awsEmrCost true
The AWS EMR Cost and Wastage report display the following sections:
This section provides detailed insights into the AWS EMR cost and wastage.

The total cluster cost and wastage in terms of expenditure and duration are plotted on the trend graph for the specified time period.
Insights are provided for cost spikes and wastages in the clusters. Click a link in the summary column to view the specifics of the insight alongwith a detailed trend graph, which plots the cost spikes and wastages. You can also download the CSV of the insights summary.
In this section, the following items are shown:
A line graph that plots two variables i.e. the total expenditure versus the wastage amount for each day in the specified period range for all the clusters. Click the variable names on the right to toggle between the variables.
A pie chart with sectors representing the total cost.


This section lists the top number of clusters by cost in a detailed table. The top clusters by cost are also represented node-wise in a bar graph

