Home

v4.3.1.2 Release notes

Software version

Release Date: 05/09/2018 for 4.3.1.2

For details on downloading updates see Unravel 4.3.1.2.

Certified platforms

  • CDH: On-premise CDH (up to v5.13) with Kerberos.

  • HDP: On-premise HDP (up to v2.6) with Kerberos + spnego enabled.

Unravel sensor upgrade

  • Optional

New features

  • When sorting Kafka topics the sort works across pages not just current page, i.e., the complete topic data is sorted. (UIX-3005)

Improvements and bug fixes
Improvements

  • In Hive queries, if Unravel detects skew and has recommendation for resource configuration, it will highlight skew and direct users to address skew first before applying new resource configuration.

  • In Hive event for tuning the number of map tasks, the property mapreduce.input.fileinputformat.split.minsize will only be listed if the new value is different from the old value, as opposed to always showing for informational purpose.

  • In Hive events for tuning the number of map and reduce tasks, the recommended values could sometimes result in apps using too few tasks and having a longer duration instead. The new version will not sacrifice duration while improving resource utilization.

  • Security relevant HTTP headers added on various HTTP services. (UNRAVEL_3408 & UNRAVEL-3419)

  • You can select which columns to display in the Applications | Applications Tab.

Bug fixes

  • Improved stability of unravel_es service. (SENSOR-28)

  • Unravel cleanups temporary folder at startup so no longer needs to be deleted manually. (PLATFORM-203)

  • Cut and paste of job name now works correctly. (CUSTOMER-19)

  • Resolved bytes out unit display in Kafka display. (UIX-360)

  • Spark header now links to parent workflow. (UIX-359 & UIX-377)

  • unravel_hdi_bootstrap script no longer overwrites existing HDP configuration. (CUSTOMER-3)

  • Fixed segv/sigbus error due to dlopen() error on libjvm.so in NGUI. (CUSTOMER-12)

  • Label Distribution graph is now displayed on Reports | Data Insights page. (UIX-285)

  • Workflow view is a now disabled for non-admin users. (UIX-396)

  • On Data Page - Details the sorting now works properly. (UIX-382)

  • On Applications | Applications it's now notes if there are no applications. (UIX-382)

  • Hive MetaStore connections in Data page decreased. (CUSTOMER-1)

  • All Operators now shown in operators and query plan. (IMPALA-29)

  • "IOException: java.io.FileNotFoundException: /usr/local/unravel/etc/apptag.pys "(No such file or directory)" incorrectly displaying during install has been resolved. (CUSTOMER-25)

  • KafkaStringProduce and KafkaStringConsumer are now reading their property files. (PLATFORM-25)

Known issues

  • Reports downloaded in XLS format are currently unsupported on MAC operating systems. Hence the end user will be unable to view reports downloaded for VCores and Cluster MemoryMB from the UI on a MAC operating system.MAC users will be able to download and open reports in PDF, JPEG, PNG, SVG and CSV formats. Windows users will be able to download and open the reports in XLS fo

New configuration properties

  • None

Software upgrade support
Upgrading 4.2.6 or later to 4.3.x

Warning

Upgrade to 4.2.6 or later before upgrade to 4.3.x
Upgrading 4.2.x to 4.3.x, one-time potential must-do steps
Run-as

When upgrading from 4.2.x to 4.3.x, you need to be cognizant of the run-as user for Unravel server daemons. Unravel Server 4.2.x and earlier runs daemons with 2 different local users (usually 'unravel' and 'hdfs' or 'mapr'). In 4.3.x, Unravel server simplifies this to run-as the single user 'unravel' by default. During an upgrade from 4.2.x to 4.3.x, all the daemons are converted over to user 'unravel'. Other changes that occur because of this: /srv/unravel/log_hdfs/* logs are moved to /usr/local/unravel/logs.

  • /srv/unravel/log_hdfs/ directory is removed.

  • /srv/unravel/tmp_hdfs/ is no longer needed, so it is removed (/srv/unravel/tmp/ is used instead).

  • env vars HDFS_KEYTAB_PATH and HDFS_KERBEROS_PRINCIPAL in /usr/local/unravel/etc/unravel.ext.sh no longer used.

  • For kerberos, com.unraveldata.kerberos.principal and com.unraveldata.kerberos.keytab.path are used instead.

After the first 4.3.x RPM upgrade, you MUST evaluate whether the switch_to_user.sh must be run in order for Unravel Server to load logs via HDFS and to access kerberos-spnego protected REST endpoints. If MapReduce or Spark jobs stop loading after upgrade, that is a strong indication that you need to switch users. This script need only be run once. See Run Unravel Daemons with Custom User for more details. Subsequent upgrades 4.3.x to 4.3.y will not need to run switch_to_user.sh script.

Upgrade process

An RPM upgrade will trigger a temporary background process so changes can take effect. You can monitor or wait for these with /usr/local/unravel/install_bin/await_fixups.sh which prints "Done" when it finishes. The script waits and prints ps of the background process. It might take 2-5 min. depending on your data size.

The background fixups will produce log file output in:

  • /tmp/unravel_post_upgrade_fixups.*.out

  • /tmp/unravel_schema.*.out

  • /usr/local/unravel/logs/unravel_schema.log

  • /usr/local/unravel/logs/es_migrate_4.3.1.0_from42.*

  • /usr/local/unravel/logs/unravel_esm.log

For further support, contact Unravel Support.

In this section: