Skip to main content

Home

User and Usage report

Using this report, you can analyze users and usages. You can understand the usage of the cluster by joining cluster-level resource information with business-level metadata. This report is relevant for analyzing MR, Hive, Spark, and Impala applications.

  1. Click Generate Reports > New button.

  2. In the New Report dialog box, enter the following details.

    Items

    Description

    General

    Name

    Name of the report.

    Report type

    Type of report. Select User and Usage report.

    Schedule

    Select the checkbox to schedule the report to run daily, hourly, weekly, or monthly. You can also set the schedule can using a cron expression. You can expand the Example drop-down and select the corresponding options. The next four sample run times are displayed for reference.

    Retention

    The number of days to retain the report files. All the reports are stored in the apps-insights-reports/src/jobs directory. After completion of the retention period, the report files are automatically purged.

    Parameters

    Application Kind

    Select the type of application. Options are MR, Hive, Spark, and Impala.

    Look Back

    The period range in days when applications can be selected for report generation. A notification is displayed above this option that informs about the duration when data is available for the report.

    Resource Metric

    Specify the resource metric you want to be included in the report. This can be either memory or CPU.

    Use External User Details

    Check this option to specify if you want to include external users in the report. Further, you must provide the following details:

    • Path to user details file (csv)

      Specify the path to the CSV file with user details.

    • User Id Column Name

      Specify the column name of the User ID.

    • Aggregation Columns Name (Separated by comma)

      Specify the column names that you want to aggregate separated by a comma.

    • Country Column Name

      Specify the column name that lists the country name of the external users.

    Notifications

    Email to

    Email ID to send the notification when the report is generated. You can specify multiple email IDs with comma separation. You can also select the Attach Files to Email checkbox to receive the reports as an attachment.

    Advance Options

    Profile Memory

  3. Click OK. The generated reports will be listed under Reports on the App UI.

  4. Select the generated report and then click Run. After the report is successfully run, the details of the report runs are listed in the Run box on the right.

  5. Click the following:

    • HTML files link to view the report details.

    • Input parameters link to view the parameters you chose to run the report.

    • Log file link to view the logs of the report.

The following reports are generated:

  • Growth story - Overall summary report (Userbase, Query, and HDFS storage)

    Contains bar charts for users, queries, and HDFS storage.

    user-usage-growth-1.png
    user-usage-growth-2.png
    user-usage-growth-3.png
  • Usage distribution detailed report

    The report contains a resource usage table for each user.

    user-usage-detailed-1.png
  • Usage distribution summary report

    The summary report of all users’ resource usage.

    user-usage-summary-1.png
Viewing metrics and insights
  1. Open the report and click any app link from the Signature column.

    The Metrics tab displays the trend charts for the Duration, IO, MemorySeconds, and CPU metrics. The dots plotted on the trend charts are the app runs. The app info is provided in the right panel.

  2. You can select any of the following features to view the metric trends:

    • status

    • cluster id

    • cluster UID

  3. Click any dot on any of the Metric trend charts. This becomes the app target run. The details of the app target run appear in the App Run info panel on the right.

  4. In the App Run info panel, click the ID link. This is the ID link of the app target run. The Metric tab is shown with the app target run you have chosen in a different color. The details of the app target run are displayed in the App Target Run panel on the right. Click the View in Unravel link to view the app's details in Unravel.

    targetapprun-highlighted.png
  5. Select any dot (other than the dot for the app target run) on any of the Metric trend charts. This becomes the app baseline run. This is highlighted in a different color. The details of the app target run are displayed in the App Baseline Run panel on the right. Click the View in Unravel link to view the app's details in Unravel.

    baseapprun-highlighted.png
  6. In the App Baseline Run panel, click Compare.

    1. Select an app from the App Type options:

      • Hive on Tez

      • Spark

      • Impala

    2. From the Cluster, select a cluster.

    3. Specify the app runs that you want to compare.

      • Baseline App Run: Specify the ID of the baseline app.

      • Target App Run: Specify the ID of the target app with which you want to compare the baseline app.

    4. Click Compare. The following app details are given for the baseline and target app run:

      • App details

      • Cluster resource usage when these apps were running

      • App statistics comparison details

      • App config comparison details

      • Query string comparison details

      • Query plan comparison details

      unity-app-app-comparator-results.png
  7. In the App Baseline Run panel, click Refresh Insights. The insights are provided in the following sections.

    • Baseline comparison

      The results of the comparison are displayed.

    • Concurrent heavy hitter apps

      This section will show up only when the information is available.

      Select this option to view the top five applications in a specific category.

      1. Click the Concurrent heavy hitter apps option.

      2. Select any option from the Top-5 Applications option.

        troubleshooting-tuning-concurrent-heavy-hitter.png

        The corresponding details are displayed.

        troubleshooting-tuning-concurrent-heavy-hitter1.png
unity-usage.png