Skip to main content

Home

Delta Tables

Delta tables are supported in Spark, which can be integrated into Unravel for monitoring. You can configure Unravel to fetch the metadata of the Delta tables and monitor them from Datapage. Data

Usually, Unravel fetches table metadata details from hive metastore. However, for Delta tables, some of the fields are missed while fetching metadata over hive metastore. Therefore, you must use the Delta Tables extractor utility, run a job, and fetch the metadata details in a JSON file. This file must be uploaded to a location on Unravel node.

After the file is uploaded to Unravel node, you must run the delta_file_handoff.sh script using the manager utility. You can then monitor Delta lake tables from Datapage.Data

See Configure Delta table for metadata processing