How to cost-optimize an EMR job
Unravel provides Chargeback details for the EMR clusters. From a single interface, you can view all the information about any number of EMR clusters. A detailed cost breakdown is also provided for EMR, EC2, and EBS volume for each of the configured AWS account. Based on the insights, cost-saving recommendations and suggestions are provided. To get a complete view into the EMR cluster costs, do the following:
On the Unravel UI, click Clusters tab > Chargeback.
Click the date picker on the right and select a period range.
Click the Group By box to select a tag. The trends and chargeback details for the selected tag are displayed in donut charts and tables.
Trends in Donut chart
In the donut chart, the results are displayed for the following trends:
Cluster Count
The cluster count trends, in percentage, for each group by tags are displayed in color-coded slices.
Avg. Cost/Cluster
The trending average cost per cluster for each group by tags is displayed in color-coded slices.
Total Cost
The total cost trends, in percentage, for each of the group by tags are displayed in color-coded slices.
Note
Null is listed against those clusters that do not have any associated tags or which have tags but with empty tag values.
You can click any of the group by options on the right of the chart and toggle the display on the donut chart. Hover over any slice of the donut chart to view the details.
Chargeback details in table
The chargeback report is also shown in a table:
Columns
Description
Purpose
Selected Group by option.
Cluster count
Count of clusters in each of the group by option.
EC2 Cost
The cost incurred in USD for clusters on EC2.
EMR Cost
The cost incurred in USD for clusters computing on EMR.
EBS Cost
The cost incurred in USD for storage on EBS.
Avg Costs/Cluster
Average cost per cluster within each group by option.
% Avg Costs/Cluster
Percentage of average cost per cluster within each group by option.
Total Cost
The total cost incurred for each group by option.
% Total Cost
Percentage of the total cost incurred for each group by option.
Go To
Click Insights
to view the EMR Insights page for the specific group by option where recommendations and suggestions are provided to reduce the costs for running jobs.Click any row in this table, and the details of the clusters in the selected group by option are displayed.
Column
Description
Cluster ID
Unique ID of the cluster.
Cluster Name
Name of the cluster.
Created
Date when the cluster was created.
State
Status of the cluster.
Cost type
Type of cost
EC2 Cost
The total cost incurred for EC2 instances in a cluster.
EMR Cost
The total cost of EMR instances in a cluster.
EBS Cost
The total cost of EBS volumes in a cluster.
Total Cost
Total cost.
Go To
Takes you to the Applications page, where the details of all the applications in a cluster are shown.
EMR Insights page
The EMR insights page displays the EMR cluster insights for a specific group by option in the selected period. The cost per cluster details is also provided. Recommendations, efficiency, and tuning suggestions are given, which can be followed for efficient cost management of EMR clusters.
(
): Suggest areas to examine to improve efficiency. For example, inefficient usage of resources. While some are only configuration property suggestions, others call your attention to areas you should examine—for example, inefficient storage, problems with a query, etc.(
): Suggestions to update a property or configuration parameter. Click to view the current and recommended value.
