Home

How to cost-optimize an EMR job

Unravel provides Chargeback details for the EMR clusters. From a single interface, you can view all the information about any number of EMR clusters. A detailed cost breakdown is also provided for EMR, EC2, and EBS volume for each of the configured AWS account. Based on the insights, cost-saving recommendations and suggestions are provided. To get a complete view into the EMR cluster costs, do the following:

  1. On the Unravel UI, click Reports > Operational Insights > Chargeback EMR.

  2. Click the date picker on the right and select a period range.

  3. Click a Group By box and select a tag option. The trends for the selected tags are displayed in donut charts and tables. For this walkthrough, click More Tags > Purpose. The chargeback report is generated, and the trends and chargeback details are displayed in the donut charts and tables.

    • Trends in Donut chart

      saas-emr-chargeback-donut.png

      In the donut chart, the results are displayed for the following trends:

      • Cluster Count

        The cluster count trends, in percentage, for each group by tags are displayed in color-coded slices.

      • Avg. Cost/Cluster

        The trending average cost per cluster for each group by tags is displayed in color-coded slices.

      • Total Cost

        The total cost trends, in percentage, for each of the group by tags are displayed in color-coded slices.

      Note

      Null is listed against those clusters that do not have any associated tags or which have tags but with empty tag values.

      You can click any of the group by options on the right of the chart and toggle the display on the donut chart. Hover over any slice of the donut chart to view the details.

      saas-emr-chargeback-donut-slice-legend.png
    • Chargeback details in table

      The chargeback report is also shown in a table:

      saass-chargeback-table.png

      Columns

      Description

      Purpose

      Selected Group by option.

      Cluster count

      Count of clusters in each of the group by option.

      EC2 Cost

      Cost incurred in USD for clusters on EC2.

      EMR Cost

      Cost incurred in USD for clusters computing on EMR.

      EBS Cost

      Cost incurred in USD for storage on EBS.

      Avg Costs/Cluster

      Average cost per cluster within each group by option.

      % Avg Costs/Cluster

      Percentage of average cost per cluster within each group by option.

      Total Cost

      Total cost incurred for each group by option.

      % Total Cost

      Percentage of total cost incurred for each group by option.

      Go To

      Click Insights View.png to view the EMR Insights page for the specific group by option where recommendations and suggestions are provided to reduce the costs for running jobs.

      Click any row in this table, and the details of the clusters in the selected group by option are displayed.

      chargeback_table1.png

      Column

      Description

      Cluster ID

      Unique ID of the cluster.

      Cluster Name

      Name of the cluster.

      Created

      Date when the cluster was created.

      State

      Status of the cluster.

      Cost type

      Type of cost

      EC2 Cost

      The total cost incurred for EC2 instances in a cluster.

      EMR Cost

      The total cost of EMR instances in a cluster.

      EBS Cost

      The total cost of EBS volumes in a cluster.

      Total Cost

      Total cost.

      Go To

      Takes you to the Applications page, where the details of all the applications in a cluster are shown.

EMR Insights page

The EMR insights page displays the EMR cluster insights for a specific group by option in the selected period. The cost per cluster details are also provided. Recommendations, efficiency, and tuning suggestions are given, which can be followed for efficient cost management of EMR clusters.

  • (OrangeAlerts.png): Suggest areas to examine to improve efficiency. For example, inefficient usage of resources. While some are only configuration property suggestions, others call your attention to areas you should examine—for example, inefficient storage, problems with a query, etc.

  • (insights.png): Suggestions to update a property or configuration parameter. Click to view the current and recommended value.

saas-emr-insights.png