Skip to main content

Home

v4.3.1.7 Release notes

Software version

Release Date: 09/14/2018 for 4.3.1.7

For details on downloading updates see Downloads.

Certified platforms

  • HDP: On-premise (up to v2.6.3) with Kerberos + spnego enabled.

  • MapR: 6.0.0 with MapR Expansion Packs 4.1.1

  • CDH: On-premise 5.13 & 5.14(including Hive 2.3.2 and Spark 2.3.0) with Kerberos

  • HDI: 3.6 Hadoop 2.7.3 , HDI 3.6 spark 2.1,2.2, 2.3, HDI 3.6 kafka 0.10.0

Unravel sensor upgrade

  • No

New features

  • None

Improvements and bug fixes
Improvements

  • New configuration property to retain timezone of history server. (PLATFORM-699)

  • Added option to setup scripts to uninstall unravel instrumentation and sensor. (INSTALL-60)

Bug fixes

  • Backend

    • Reduced unravel_ngui daemon startup time. (CUSTOMER-244)

  • UIX

    • Failed jobs are only shown if Application ID is available. (CUSTOMER-192)

    • Unravel recommendations in Event Panel better formatted and more understandable. (CUSTOMER-204)

    • Cluster Workload Report populates correctly. (CUSTOMER-209, CUSTOMER-211)

    • Added ability to search by app name field on Applications and Workflow page tabs. (CUSTOMER-263)

    • Removed unnecessary fields from MapReduce Timeline view. (CUSTOMER-279)

    • Auto Refresh updates page correctly. (UIX-1229,UIX-1227)

    • Oozie Spark Tagged Workflow: time stamps, links and tags now populating correctly. (UIX-359)

    • Infrastructure page no longer hanging. (UIX-1228)

    • Metrics data now populates for Apps which are part of Workflows. (UIX-1225)

    • Date Range end time now automatically refreshes. (UIX-1167)

    • Sorting now works for Impala "Frag Id". (UIX-1051)

    • Pending count populates correctly on both Operations and Application pages. (UIX-709)

  • Platform

    • Hive-on-MR/Tez apps that are correctly labeled status unknown when in unknown state. (PLATFORM-664)

  • Tez

    • Various improvements in Tez pipeline efficiency. (TEZLLAP-114)

    • Tez tagged workflows are now working correctly. (CUSTOMER-272, CUSTOMER-282)

  • Report

    • App count is consistent and correct across different views in Cluster Workload. (REPORT-94)

Known issues

  • Missing WFI icon within Application tab for MR job associated with workflow. (CUSTOMER-274)

  • Application tab filter for Pending Status makes no sense it should be Accepted instead. (CUSTOMER-278)

  • Add workflow pop up hangs for 2000+ workflows. (REPORT-116)

  • Cluster Summary and Cluster compare hangs for large dataset. (REPORT-117)

  • Date - time picker is missing for reports. (UIX-1201)

  • RBAC: Issue with view by details in infrastructure. (UIX-942)

  • HDI : Logs for Spark jobs are not shown in Cluster running Spark2.3. (USPARK-182)

  • HDI : Hive app is shown only in "RUNNING" state on Hive APM page.

Configuration properties
New

  • com.unraveldata.hdfs.timezone: timezone of HDFS.

Deprecated

  • None

Software upgrade support

  • Support for upgrade from 4.2.6 (4.2.6-1128) or 4.2.7 (4.2.7-1154)

  • Support for upgrade from 4.3.0.X to 4.3.1.7

    Upgrading from 4.2.6 or 4.2.7 with this RPM causes a background process to run for about 15 minutes. During which data is migrated in the unravel_s_1 daemon. To see status of this migration, see /usr/local/unravel/install_bin/es_migrate_4.3.1.0_from42.out and /usr/local/unravel/logs/es_ migrate_4.3.1.0_from42.log ; this migration can be re-run using/usr/local/unravel/install_bin/es_migrate_4.3.1.0_from42.sh in the background with nohup. When the migration finishes successfully, disk space can be saved by running /usr/local/unravel/install_bin/es_migrate_4.3.1.0_from42_cleanup.sh.

    An RPM upgrade triggers a temporary background process so changes can take effect. You can monitor it or wait for these with /usr/local/unravel/install_bin/await_fixups.sh which prints DONE when it finishes. Tt takes anywhere from 2-15 minutes depending on your data size.

    After upgrade:

    Run /usr/local/unravel/install_bin/await_fixups.shand wait for it to finish. It can take up to 15 minutes. If shell is terminated, it can be run again at any time.

    There should be 47 tables. Check the table count manually with db_access.sh and 'show tables;if fewer tables are present, run: sudo /usr/local/unravel/dbin/db_schema_upgrade.sh.

Warning

Upgrade to 4.2.6 or later before upgrade to 4.3.x
Upgrading 4.2.x to 4.3.x, one-time potential must-so steps
Run-as

When upgrading from 4.2.x to 4.3.x, you need to be cognizant of the run-as user for Unravel server daemons. Unravel Server 4.2.x and earlier runs daemons with 2 different local users (usually 'unravel' and 'hdfs' or 'mapr'). In 4.3.x, Unravel server simplifies this to run-as the single user 'unravel' by default. During an upgrade from 4.2.x to 4.3.x, all the daemons are converted over to user 'unravel'. Other changes that occur because of this:/srv/unravel/log_hdfs/* logs are moved to /usr/local/unravel/logs

/srv/unravel/log_hdfs/ directory is removed

/srv/unravel/tmp_hdfs/ is no longer needed, so it is removed (/srv/unravel/tmp/ is used instead)

env vars HDFS_KEYTAB_PATH and HDFS_KERBEROS_PRINCIPAL in /usr/local/unravel/etc/unravel.ext.sh no longer used

For kerberos, com.unraveldata.kerberos.principal and com.unraveldata.kerberos.keytab.path are used instead.

After the first 4.3.x RPM upgrade, you MUST evaluate whether the switch_to_user.sh must be run in order for Unravel Server to load logs via HDFS and to access kerberos-spnego protected REST endpoints. If MapReduce or Spark jobs stop loading after upgrade, that is a strong indication that you need to switch users. This script need only be run once. See Run Unravel Daemons with Custom User for more details. Subsequent upgrades 4.3.x to 4.3.y will not need to run switch_to_user.sh script.

Upgrade process

An RPM upgrade will trigger a temporary background process so changes can take effect. You can monitor or wait for these with /usr/local/unravel/install_bin/await_fixups.sh which prints "Done" when it finishes. The script waits and prints ps of the background process. It might take 2-5 min. depending on your data size.

The background fixups will produce log file output in:

/tmp/unravel_post_upgrade_fixups.*.out

/tmp/unravel_schema.*.out

/usr/local/unravel/logs/unravel_schema.log

/usr/local/unravel/logs/es_migrate_4.3.1.0_from42.*

/usr/local/unravel/logs/unravel_esm.log

For further support, contact Unravel Support.

In this section: