v4.2.3 Release notes
Software version
Date: 11/1/2017
On-premise RPM | unravel-4.2.3.x86_64.rpm |
EMR RPM |
How to download
curl -v http://preview.unraveldata.com/img/unravel.x86_64.rpm -o unravel-4.2.3.x86_64.rpm
Tip
MD5 for unravel.x86_64.rpm: 3701dca005208365b3490f3b0390b5ec
Tested platforms
CDH: On-premise CDH (up to version v5.12) with Kerberos enabled.
HDP: On-premise HDP (up to version v2.5) with Kerberos enabled.
EMR: testing is in progress.
Qubole: testing is in progress.
Unravel sensor upgrade
Optional
New features
Tez support
This release supports the monitoring of applications run within the Tez framework. This includes:
Hive on Tez.
Pig on Tez.
Cascading on Tez.
The Tez application performance manager (APM) in Unravel Web UI provides a detailed view into the behavior of "Tez framework" queries as a directed acyclic graph (DAG). DAG details include query text, DAG graphs, and information on vertices, tasks, and attempts. For more information, see the User Guide.
Spark pipeline
This release includes an improved pipeline for collecting data from Spark applications. The pipeline provides:
Additional metadata.
Stage level updates: Previously, the Spark APM in Unravel Web UI updated the application's page only once, when the application was finished. With this release, as soon as a stage completes, the Spark APM in Unravel Web UI shows stage level information. This improvement allows you to interact with the Spark APM more often.
Smaller memory footprint.
Smarter events: The event generation algorithm has been updated with additional triggers. Events are generated only if an application's original settings are significantly different from Unravel's suggested settings.
Workflow tagging: Unravel Web UI now displays tagged workflows identically, irrespective of the mode in which the Spark applications were loaded. Supported loading modes remain the same: OPS mode (loading through Unravel Sensor for Spark), and BATCH mode (loaded outside of Unravel Sensor).
Applications are tagged even when the event log files are unavailable.
Consistent display of information about each Spark application, even if the event log file is not available. At a minimum, a Spark application's page displays the application metadata, taken from the resource manager.
Multi-Cluster Kafka support
This release supports data collection for multi-cluster Kafka topics, and Kafka application performance management through Unravel Web UI. Kafka support includes:
Multi cluster support
Multi cluster metrics monitoring.
Multi cluster consumer offset/lag monitoring.
Consumer groups
Single view for CG status/stats across topics it consumes.
Additional metrics added for brokers
Insights
Tunable sliding window algorithm.
Consumer group lagging/stalled.
Improvements and bug fixes
Unravel Sensor has been upgraded to improve performance related to DNS issues. This upgrade is optional. For help with upgrading Unravel Sensor, contact Unravel Support.
Impala improvements and insights.
Impala OPs support
Daemon memory consumption graph.
Number of queries running graph.
Impala queries insights improvements
Improved time breakdown event.
Improved suggestions on hash joins.
Fixed the issue “Unravel not loading eventlog.inprogress file”.
Fixed the issue “Impala queries not coming up in UI”.
Fixed the issue “Page refresh changes the sorting order”.
Fixed the issue “High latency in loading MR jobs”.JCS2 now uses “hdfs ls” as opposed to “du”.
Fixed the issue “Issue with unravel_us_1 demon”.
ES migration script to migrate ES mappings for Impala (runs during upgrade of 4.1 or older version to 4.2).
Known issues
Data collection related to Impala queries causes the number of TCP connections to increase over time.
There are issues with multi-host installations of Unravel Server. For help, contact Unravel Support.