- Home
- Release Notes
- v4.7.x Release Notes
- v4.7.x - Updates to Unravel properties
v4.7.x - Updates to Unravel properties
Note
This page lists Unravel configuration properties added, renamed, removed, and deprecated in 4.7.x.
Property | Description | Type | Default Value |
|---|---|---|---|
com.unraveldata.aws.bill.import.dataexports.name | Specifies the name of the AWS billing data export file. | String | |
com.unraveldata.aws.bill.import.dataexports.prefix | Defines the prefix applied to the exported AWS billing data files. | string | |
com.unraveldata.aws.bill.import.enabled | Enables or disables AWS billing data import in Unravel. | string | true |
com.unraveldata.aws.bill.import.lookback.days | Specifies the number of past days of AWS billing data to import. | integer | 30 |
com.unraveldata.aws.bill.import.region | Defines the AWS region where the billing data is located. | string | |
com.unraveldata.aws.bill.import.s3.bucket.name | Specifies the name of the S3 bucket containing the AWS billing data files. | string | |
com.unraveldata.aws.bill.import.user.access.key | Specifies the AWS access key ID for accessing billing data. | string | |
com.unraveldata.aws.bill.import.user.secret.key | Specifies the AWS secret access key for accessing billing data. | string |
Property | Description | Sample Values |
|---|---|---|
com.unraveldata.unitycatalog.accessor.enabled | Enables Databricks system table accessor in Unravel. | true |
com.unraveldata.unitycatalog.accessor.jdbc.url | Specifies the JDBC URL for connecting to the Databricks workspace. | jdbc:databricks://dbc-f874t197-a823.cloud.databricks.com:443/default;transportMode=http;ssl=1;AuthMech=3;httpPath=/sql/1.0/warehouses/d7v51a813c7gf941; |
com.unraveldata.unitycatalog.accessor.lookback.days | Sets the number of days to look back for Databricks system table data. | 30 |
com.unraveldata.unitycatalog.accessor.workspace.id | Specifies the Workspace ID used to integrate Unravel with Databricks System tables | 4083330060312460 |
Property | Description | Default |
|---|---|---|
com.unraveldata.databricks.sql.enabled | Enables SQL pipelines. True to enable and false to disable. | True |
com.unraveldata.databricks.enabled.connectors | Enables or disables dbx_worker connectors like: CLUSTER_CONNECTOR, JOB_CONNECTOR, WAREHOUSE_CONNECTOR, QUERY_HISTORY_CONNECTOR. | CLUSTER_CONNECTOR, JOB_CONNECTOR, WAREHOUSE_CONNECTOR, QUERY_HISTORY_CONNECTOR. |
com.unraveldata.databricks.sql.warehouse.cost.timeout | Determines the timeout for warehouse cost calculation. | 600000 (10 minutes) |
com.unraveldata.databricks.sql.consumers | Determines the number of workers processing Databricks SQL data. | 8 |
com.unraveldata.databricks.sql.queryhistory.lookback.days | Determines the days to look back for query history after a fresh installation or downtime. | 15 |
NA
NA
NA
NA
NA
NA
NA
NA
NA
NA
NA
Property/Description | Set by user | Unit | Unit |
|---|---|---|---|
com.unraveldata.databricks.insights.supported.clusters Defines a comma-separated list of clusters for which insights generation is supported for Databricks. | Optional | String | INTERACTIVE,AUTOMATED,AUTOMATEDLIGHTINTERACTIVE,AUTOMATED,AUTOMATEDLIGHT |
Property/Description | Set by user | Unit | Default |
|---|---|---|---|
com.unraveldata.databricks.cost.import.local.dir Local directory to import cost files from. Each month’s aggregated cost data should be in a single CSV file within a directory named with the month (yyyymm). | Optional | NA | Path |
Property/Description | Set by user | Unit | Default |
|---|---|---|---|
com.unraveldata.databricks.cm.timeseries.enabled Enables the generation of Cluster Metrics data and Resource Manager data using Databricks worker when set to true. | Optional | Boolean | True |
com.unraveldata.databricks.consumers Sets the quantity of Kafka consumer threads responsible for processing data from the Databricks Kafka topic. Ensure this value does not exceed the number of Kafka partitions allocated for the Databricks topic. | Optional | Integer | 8 |
com.unraveldata.databricks.reader.pool.size Sets the number of concurrent reader threads for polling the Databricks API. | Optional | Integer | 40 |
com.unraveldata.databricks.api.polling.interval Sets the interval, in seconds, between consecutive REST API requests for polling a Databricks workspace. | Optional | Seconds | 30 |
com.unraveldata.databricks.api.lookback.seconds Sets the lookback period for polling the Databricks API to retrieve historical data on every worker restart. | Optional | Seconds | 60 |
com.unraveldata.databricks.cache.size.max Sets the maximum number of items to be cached in memory. | Optional | Integer | 1000 |
com.unraveldata.databricks.cache.entry.ttl.secs Sets the timeout, in seconds, for cached items in the Databricks worker cache. | Optional | Seconds | 3600 |
com.unraveldata.include.unknown.clusters.chargeback Determines whether UNKNOWN clusters should be included on Trends, Drill down, and Unravel billing pages. Set to true to include, false to exclude. | Optional | Boolean | True |
com.unraveldata.databricks.azure.allpurpose.photon.dbu.factor Sets the multiplication factor for the cost of non-Photon node types when Databricks Units (DBUs) are unavailable under all-purpose Photon. Enables dynamic cost management in such instances. | Optional | Double | 2 |
com.unraveldata.databricks.aws.allpurpose.photon.dbu.factor Sets the multiplication factor for the cost of non-Photon node types when Databricks Units (DBUs) are unavailable under all-purpose Photon. Enables dynamic cost management in such instances. | Optional | Double | 2 |
com.unraveldata.databricks.aws.jobcompute.photon.dbu.factor Sets the multiplication factor for the cost of non-Photon node types when Databricks Units (DBUs) are unavailable under job compute Photon. Enables dynamic cost management in such instances. | Mandatory | Double | 2.9 NoteSet the value to 2.9 for accurate cost computation on AWS. |
com.unraveldata.databricks.azure.jobcompute.photon.dbu.factor Sets the multiplication factor for the cost of non-Photon node types when Databricks Units (DBUs) are unavailable under job compute Photon. Enables dynamic cost management in such instances. | Optional | Double | 2.5 |
NA
NA
NA
NA
NA
NA
NA
NA
NA
NA
NA
Property/Description | Set by user | Unit | Unit |
|---|---|---|---|
com.unraveldata.databricks.insights.supported.clusters Defines a comma-separated list of clusters for which insights generation is supported for Databricks. | Optional | String | INTERACTIVE,AUTOMATED,AUTOMATEDLIGHTINTERACTIVE,AUTOMATED,AUTOMATEDLIGHT |
Property/Description | Set by user | Unit | Default |
|---|---|---|---|
com.unraveldata.databricks.cm.timeseries.enabled Enables the generation of Cluster Metrics data and Resource Manager data using Databricks worker when set to true. | Optional | Boolean | True |
com.unraveldata.databricks.consumers Sets the quantity of Kafka consumer threads responsible for processing data from the Databricks Kafka topic. Ensure this value does not exceed the number of Kafka partitions allocated for the Databricks topic. | Optional | Integer | 8 |
com.unraveldata.databricks.reader.pool.size Sets the number of concurrent reader threads for polling the Databricks API. | Optional | Integer | 40 |
com.unraveldata.databricks.api.polling.interval Sets the interval, in seconds, between consecutive REST API requests for polling a Databricks workspace. | Optional | Seconds | 30 |
com.unraveldata.databricks.api.lookback.seconds Sets the lookback period for polling the Databricks API to retrieve historical data on every worker restart. | Optional | Seconds | 60 |
com.unraveldata.databricks.cache.size.max Sets the maximum number of items to be cached in memory. | Optional | Integer | 1000 |
com.unraveldata.databricks.cache.entry.ttl.secs Sets the timeout, in seconds, for cached items in the Databricks worker cache. | Optional | Seconds | 3600 |
com.unraveldata.include.unknown.clusters.chargeback Determines whether UNKNOWN clusters should be included on Trends, Drill down, and Unravel billing pages. Set to true to include, false to exclude. | Optional | Boolean | True |
com.unraveldata.databricks.azure.allpurpose.photon.dbu.factor Sets the multiplication factor for the cost of non-Photon node types when Databricks Units (DBUs) are unavailable under all-purpose Photon. Enables dynamic cost management in such instances. | Optional | Double | 2 |
com.unraveldata.databricks.aws.allpurpose.photon.dbu.factor Sets the multiplication factor for the cost of non-Photon node types when Databricks Units (DBUs) are unavailable under all-purpose Photon. Enables dynamic cost management in such instances. | Optional | Double | 2 |
com.unraveldata.databricks.aws.jobcompute.photon.dbu.factor Sets the multiplication factor for the cost of non-Photon node types when Databricks Units (DBUs) are unavailable under job compute Photon. Enables dynamic cost management in such instances. | Mandatory | Double | 2.9 NoteSet the value to 2.9 for accurate cost computation on AWS. |
com.unraveldata.databricks.azure.jobcompute.photon.dbu.factor Sets the multiplication factor for the cost of non-Photon node types when Databricks Units (DBUs) are unavailable under job compute Photon. Enables dynamic cost management in such instances. | Optional | Double | 2.5 |
NA
NA
NA
NA
NA
Property/Description | Set by user | Unit | Default |
|---|---|---|---|
com.unraveldata.unityapp.enabled Enables access to the Unity app from Unravel UI. | Yes | boolean | false |
com.unraveldata.unityapp.link Link to access the Unity app from Unravel UI. | Yes | URL | - |
Property | Default |
|---|---|
com.unraveldata.spark.delay.db.flush Flushes all the DB updates only after processing individual kafka message when set to true. For example, if there are five database updates generated while processing a single kafka message with this flag enabled all these updates are cached and pushed in batch once the message is completely processed | false |
Property/Description | Set by user | Unit | Default |
|---|---|---|---|
com.unraveldata.cache.enabled Enables caching for the database. When the value is false, the cache will not be enabled. | Boolean | false | |
com.unraveldata.cache.type Available options are none, embedded, ucache, redis. | Text | none | |
com.unraveldata.cache.serialization.type Available options are Kryo and Java. | Text | Kryo | |
com.unraveldata.cache.entry.ttl.secs The time-to-live (TTL) value in seconds for cached items. | Integer | 3600 | |
com.unraveldata.cache.size.max The maximum number of entries to be retained in the cache per client. | Integer | 10000 | |
Unravel cache worker specific properties | |||
com.unraveldata.cache.worker.enabled Enables cache workers. When the value is false, worker is not enabled. | Boolean | false | |
com.unraveldata.cache.unravel.host The Unravel cache host. | Text | 127.0.0.1 | |
com.unraveldata.cache.unravel.port The Unravel cache port. | Integer | 4310 | |
com.unraveldata.cache.deadline.long.seconds The time limit in seconds for longer duration gRPC calls in seconds. | Integer | 10 | |
com.unraveldata.cache.deadline.short.millis The time limit in seconds for short-duration gRPC calls in milliseconds. | Integer | 1000 | |
Redis specific properties | |||
com.unraveldata.cache.redis.host The Redis host. | Text | ||
com.unraveldata.cache.redis.port The Redis port. | Integer | ||
com.unraveldata.cache.redis.connection.timeout.millis The Redis connection timeout in milliseconds. | Integer | 10000 | |
com.unraveldata.cache.redis.read.timeout.millis The Redis read timeout in milliseconds. | Integer | 10000 | |
com.unraveldata.cache.redis.password The Redis password. | Text | null | |
com.unraveldata.cache.redis.database The Redis database. | Integer | 0 | |
com.unraveldata.cache.redis.client.name The Redis client name. | Text | null | |
com.unraveldata.cache.redis.ssl.enabled Enables the SSL for Redis. | Boolean | false | |
com.unraveldata.cache.redis.ssl.socket.factory The Redis SSL socket factory. | Text | null | |
com.unraveldata.cache.redis.ssl.params The Redis SSL parameters. | Text | null | |
com.unraveldata.cache.redis.host.name.verifier The Redis host name verifier. | Text | null | |
NA
NA
NA
NA
NA
4.7.9.1
Property/Description | Default |
|---|---|
com.unraveldata.metrics.factor Property to skip metrics updates. If this property is specified, lesser messages are processed by the insights worker. Specify any of the following to skip metrics updates:
| 1 |
unravel.python.fsimage.java.home Specify the JAVA home for FSImage processing. | - |
com.unraveldata.table.polling.limit Modify the batch size for polling the table metadata based on the number of tables available in your Hive metastore. | 1000000 |
com.unraveldata.partition.polling.limit Modify the batch size for polling partitions metadata based on the number of partitions available in your Hive metastore. | 1000000 |
NA
NA
NA
NA
NA
4.7.9.0
Property/Description | Default |
|---|---|
logreceiver.verticles_count To increase the capacity of LR, you can vertically scale it by adding more verticles. Use the logreceiver.verticles_count property to set the number of verticles. |
The value specifies that the number of LR verticles equals MIN (CPU cores, 12). |
unravel.appstore.apps.gunicorn.timeout Set the timeout for apps in the appstore. | 300 seconds |
com.unraveldata.config.redaction.regex Use this common unravel property to redact configuration values. |
NA
NA
NA
Property Name | Description |
|---|---|
| Removed the property from the Hive hook to remove HDFS dependency. If encountering difficulties accessing the Log Receiver (LR), logging the information is adequate instead of storing the data in HDFS. |
4.7.8.0
Property/Description | Default |
|---|---|
AutoActions support for EMR Cluster | |
com.unraveldata.auto.action.emr.cluster.enabled Enables ElasticSearch polling for collecting EMR cluster metrics. | false |
com.unraveldata.auto.action.emr.cluster.idle.lookback.millis Set the minimum look-back period for calculating the idleness of an EMR cluster. The EMR cluster idle duration is specific to the EMR Cluster Idle AutoAction policy. You must set this property if configured idle threshold for an EMR cluster is more than an hour. Ensure to specify the value in milliseconds. | 3600000 (1 hour) |
Insights consumers | |
com.unraveldata.insights.consumers Specify this property to set the number of threads in the insights topic worker. This property sets the number of insights consumers running within an insights worker to increase processing speed. | 2 |
Spark consumers | |
com.unraveldata.spark.consumers Specify this property to run multiple Kafka consumers in a single Spark worker. This property sets the number of spark consumers running within a spark worker. You must configure this property in every environment depending on the desired number of consumers. Additionally, set the number of Kafka partitions to the same number (or higher). | 1 |
NA
NA
NA
NA
4.7.7.0
Property/Description | Default |
|---|---|
License | |
com.unraveldata.license.file When you run the | - |
GCP - BigQuery | |
com.unraveldata.bigquery.ondemand.analysis.pricing.per.tb.usd.{region} Specify the on-demand analysis price per TB in USD based on the region to process queries. The The sample Examples:
You can find the possible values for | 5 ($5) |
com.unraveldata.bigquery.live.metrics.polling.interval.mins Determines the frequency at which the live metrics should be fetched for a long running query in minutes. | 5 mins |
com.unraveldata.bigquery.projects.polling.interval.secs The frequency of polling interval of audit log messages for the project in seconds. | 300 secs |
com.unraveldata.bigquery.pubsub.total.message.{projectId} Determines the maximum no of messages that must be fetched in each polling interval for a project. | 100 |
The default value for the spark.unravel.shutdown.delay.ms is changed from 300 ms to 1500 ms (1.5 seconds).
NA
NA
NA
4.7.6.0
Property/Description | Default |
|---|---|
Persistent API Token | |
com.unraveldata.api.access.tokens.store Provide the path to the JSON file that is created in the location accessible only to the Unravel user | - |
com.unraveldata.auto.action.use.cli.enabled Enables use of Command Line Interface (CLI) for actions associated with AutoActions. | true |
NA
NA
NA
NA
4.7.5.0
Property/Description | Default |
|---|---|
Log receiver | |
logreceiver.msg_retention.hours The retention period, in hours, for messages stored in the DocStorage by the log receiver. This value should align with the Kafka retention time for AWS, Spark, and JC topics. | 1 hour |
Spark | |
com.unraveldata.spark.live.skip.small.jobs When set to | false |
com.unraveldata.spark.live.min.jobs.stored.count Indicates the count of a minimum number of Spark jobs to persist for an application (irrespective of the duration) in a live pipeline. Considered only when com.unraveldata.spark.live.skip.small.jobs is set to | 500 |
com.unraveldata.spark.live.small.jobs.duration.threshold Indicates the minimum duration threshold of Spark jobs to skip insignificant jobs. Specify the duration in milliseconds. | 1000 |
Impala | |
com.unraveldata.impala.polling.look.back.secs A maximum number of seconds to look back when polling the Cloudera Manager API for subsequent polls. | 0 |
Workflow | |
com.unraveldata.workflow.fetch.size This property defines a maximum number of workflows to be retrieved from the cluster and displayed on the Unravel UI. | 100 |
Live Metrics
The default value of the com.unraveldata.metrics.merge.cache property has been changed from 6 to 30.
None
None
com.unraveldata.cloudera.manager.impala.poll.interval.millis
com.unraveldata.databricks.dbu.discount.percentage
4.7.4.0
Property/Description | Default |
|---|---|
Data page | |
com.unraveldata.bigquery.tableinfo.enabled Enable/ disable BigQuery for the Data page. By default, this is enabled. | TRUE |
com.unraveldata.datapage.cluster.names The Data page supported valid cluster names. If this property is not set then all clusters are shown. | - |
com.unraveldata.metastore.types Support metastore type for Data page. | Based on the cluster type:
|
com.unraveldata.datapage.workspace.names The Data page supported valid workspace names. If this property is not set then all workspaces are shown. | - |
Databricks | |
com.unraveldata.databricks.keys.toRedact Redacts spark property values if the property key is in the list. | password |
Migration | |
com.unraveldata.migrationplanning.clusterdiscovery.diagnostics Set to true to enable diagnostics for Cluster Discovery report. | false |
com.unraveldata.migrationplanning.cmph.diagnostics Set to true to enable diagnostics for Cloud Mapping Per Host report. | false |
com.unraveldata.migrationplanning.slicedice.diagnostics Set to true to enable diagnostics for Workload Fit > Cloud Topology report. | false |
com.unraveldata.migrationplanning.workload_fit_heatmap.diagnostics Set to true to enable diagnostics for Workload Fit > Heatmap report. | false |
com.unraveldata.migrationplanning.services_compatibility.diagnostics Set to true to enable diagnostics for Services and Versions Compatibility report. | false |
Spark | |
com.unraveldata.spark.query.size.max | 200000 |
Pipelines | |
General pipelines | |
com.unraveldata.min.failed.job.duration.for.attempt.log Minimum failed application duration to attempt yarn log loading for an application. | 0 |
com.unraveldata.min.failed.job.duration.for.attempt.log Minimum failed application duration to attempt yarn log loading for an application. | 0 |
com.unraveldata.logs.max.containers.persisted The maximum number of container logs to be persisted to the DB for an application. | 4 |
com.unraveldata.logs.max.containers.processed The maximum number of yarn containers to be processed from the yarn log of an application. | 100 |
Impala pipelines | |
com.unraveldata.impala.scheduled.thread.pool.size Minimum application duration to attempt yarn log loading for an application. | 5 |
com.unraveldata.impala.thread.initial.delay.secs Time to delay the first execution of the thread. | 5 |
com.unraveldata.impala.cm.queries.polling.interval.secs The period between successive executions for polling queries from CM. | 60 |
com.unraveldata.impala.cm.metrics.polling.interval.secs The period between successive executions for polling metrics from CM | 60 |
com.unraveldata.impala.kafka.retry.millis Delay (in milliseconds) between successive Impala Kafka message retries. | 1000 |
None
None
None
com.unraveldata.cloudera.manager.impala.poll.interval.millis
4.7.3.0
Property/Description | Default |
|---|---|
App Store | |
com.unravel.appstore.apps.enabled To enable/disable Appstore UI. Set to true to enable and false to disable. | true |
unravel.appstore.flask.port The port on which the Appstore’s flask process will listen. This can be any free port on Unravel’s core node. | 5001 |
unravel.appstore.apps.port.range.start Defines the start value of a range of ports on which Individual apps will run. This can be any valid port on Unravel's core node. | 8190 |
unravel.appstore.apps.port.range.end Defines end value of a range of ports on which Individual apps will run. This can be any valid port on Unravel's core node. This value should be greater than the value set for unravel.appstore.apps.port.range.start. | 8290 |
unravel.appstore.apps.gunicorn.worker.count The default number of gUnicorn workers to be started for each app. | Default is calculated based on the number of cores of Unravel core node. |
unravel.appstore.apps.<app_id>.gunicorn.worker.count The default number of gUnicorn workers to be started for a specific app. | Defaults to value set for unravel.appstore.apps.gunicorn.worker.count |
unravel.appstore.apps.gunicorn.thread.count The default number of gUnicorn threads to be started per gUnicorn worker for each app. | Default is based on the number of cores of the Unravel core node. |
unravel.appstore.apps.<app_id>.gunicorn.thread.count The default number of gUnicorn threads to be started per gUnicorn worker for a specific app. | Defaults to value set for unravel.appstore.apps.gunicorn.thread.count |
unravel.appstore.resource_limit.rss.gb Total RSS memory in GB that all running apps can consume. | The default value is set to half of the system memory of the Unravel core node. |
unravel.appstore.apps.resource_limit.process_count The default maximum limit on the number of processes that can be running for each app. | 5 x value set forunravel.appstore.apps.gunicorn.worker.count |
unravel.appstore.apps.<app_id>.resource_limit.process_count The default maximum limit on the number of processes that can be running for a specific app. | Same as set forunravel.appstore.apps.resource_limit.process_count |
unravel.appstore.apps.resource_limit.sum.thread_count Limit on the sum of the number of threads across processes belonging to an app. | No limit |
unravel.appstore.apps.<app_id>.resource_limit.sum.thread_count Limit on the sum of the number of threads across processes belonging to a specific app. | The same value set for unravel.appstore.apps.resource_limit.sum.thread_count If this property is not set, then the default is: 5 x (unravel.appstore.apps.<app_id>.gunicorn.worker.count x unravel.appstore.apps.<app_id>.gunicorn.thread.count) |
unravel.appstore.apps.resource_limit.sum.rss Limit on the sum of RSS in bytes across processes belonging to an app. | 1/10 x Total system memory on Unravel core node |
unravel.appstore.apps.<app_id>.resource_limit.sum.rss Limit on the sum of RSS in bytes across processes belonging to a specific app. | Same value set for unravel.appstore.apps.resource_limit.sum.rss |
unravel.appstore.apps.resource_limit.sum.vms Limit on the sum of VMS in bytes across processes belonging to an app. | No limit |
>>unravel.appstore.apps.<app_id}.resource_limit.sum.vms Limit on the sum of VMS in bytes across processes belonging to a specific app. | Same value set for unravel.appstore.apps.resource_limit.sum.vms |
unravel.appstore.apps.resource_limit.sum.memory_percent Limit on the sum of memory percentage across processes belonging to an app. | No limit |
unravel.appstore.apps.<app_id>.resource_limit.sum.memory_percent Limit on the sum of memory percentage across processes belonging to a specific app. | Same value set forunravel.appstore.apps.resource_limit.sum.memory_percent |
unravel.appstore.apps.resource_limit.sum.cpu_percent Limit on the sum of CPU percentage across processes belonging to an app. | No limit |
unravel.appstore.apps.<app_id>.resource_limit.sum.cpu_percent Limit on the sum of memory percentage across processes belonging to a specific app. | Same value set for unravel.appstore.apps.resource_limit.sum.cpu_percent |
unravel.appstore.apps.resource_limit.sum.cpu_time Limit on the sum of CPU time in seconds across processes belonging to an app. | No limit |
unravel.appstore.apps.{app_id}.resource_limit.sum.cpu_time Limit on the sum of cpu_time in seconds across processes belonging to a specific app. | Same value set for unravel.appstore.apps.resource_limit.sum.cpu_time |
unravel.appstore.apps.resource_limit.<aggregation>.<resource_name}> Property to set resource limits and aggregation methods for all the apps. | No limit |
unravel.appstore.apps.<app_id>.resource_limit.<aggregation>.<resource_name> Property to set resource limits and aggregation methods for a specific app. | No limit |
unravel.appstore.apps.healthcheck.interval.seconds The interval at which the Health check process is run for the apps. | 60 seconds |
unravel.appstore.apps.resource.usage.refresh.rate.seconds The interval at which resource usages will be updated. | 60 seconds |
BigQuery | |
com.unraveldata.bigquery.project.details.poll.delay.mins Sets the frequency to poll Project data such as project name, state, etc. By default, it is one day that is 1440 minutes. | 1440 mins |
com.unraveldata.bigquery.projects Specify project IDs for BigQuery monitoring. | |
com.unraveldata.bigquery.pubsub.subscription_id.<project-ID> Specify the subscription ID of a project for BigQuery monitoring. | |
com.unraveldata.bigquery.credentials.<project-id> Specify service key file for Google API authentication. | |
Multi-cluster Hadoop | |
unravel.hadoop_module.platform Set this property when you install the core node on a server where there is no Hadoop configuration. | unravel |
Impala | |
com.unraveldata.impala.http.conn.timeout.ms HTTP connection time out for impala connector. | 2000 ms |
com.unraveldata.impala.http.read.timeout.ms HTTP read time out for impala connector. | 6000 ms |
com.unraveldata.impala.http.backoff.ms HTTP client back-off time for impala connector. Time for which HTTP client sleeps before re-attempting after unsuccessful read attempt. | 1000 ms |
com.unraveldata.impala.query.max.retries Specifies the number of retries that will be done to fetch the profile tree for any impala query. | 5 |
LDAP | |
com.unraveldata.ldap.domain Contains a real domain name. | - |
Live metrics | |
com.unraveldata.metrics.merge.cache Specify the count of messages to be stored in the cache or memory before storing in the database. For example, if com.unraveldata.metrics.merge.cache = 30 and unravel.metrics.factor = 5 (seconds), then 30 * 5 = 150 seconds. The data is pushed to the database every 150 seconds. For 150 seconds, the data is kept in memory. | 6 |
com.unraveldata.metrics.sample.count.max Specify the maximum count of data points for sampling purposes. | 16 |
com.unraveldata.metrics.data.points The number of data points to be stored in a single database row. | 44 |
Role-based Access Control (RBAC) | |
com.unraveldata.rbac.roles A comma-separated list of custom roles. | - |
com.unraveldata.rbac.role.<role>.users A comma-separated list of users that can be assigned to a role. The role can be admin, readonlyAdmin, or custom_role, which is set using com.unraveldata.rbac.roles property. | |
com.unraveldata.rbac.role.<role>.groups A comma-separated list of groups that can be assigned to a role. The role can be admin, readonlyAdmin, or custom_role, which is set using com.unraveldata.rbac.roles property. | |
com.unraveldata.rbac.role.<role>.data.user.filter User data filter, if enabled data is filtered on the username. The role can be admin or custom_role, which is set using com.unraveldata.rbac.roles property. | FALSE |
com.unraveldata.rbac.role.<role>.data.tags.filter Tags data filter, if enabled data is filtered on the user tags. The role can be admin or custom_role, which is set using com.unraveldata.rbac.roles property. | FALSE |
com.unraveldata.rbac.role.<role>.data.es.query Query data filter, if enabled data is filtered on the given valid ElasticSearch query. The role can be admin or custom_role, which is set using com.unraveldata.rbac.roles property. | |
com.unraveldata.rbac.role.<role>.data.field.<field> Fields data filter, if enabled data is filtered on the supported ElasticSearch fields for the given values. The <field> should be in the supported field list. The role can be user or custom_role, which is set using com.unraveldata.rbac.roles property. | |
com.unraveldata.rbac.role.<role>.views The views that are shown for a specific role. This is configured based on the View ID list. The role can be user or custom_role, which is set using com.unraveldata.rbac.roles property. | clusters.resources, clusters.chargeback, compute, jobs |
The property com.unraveldata.ldap.customLDAPQuery is changed to a new format. This can be specified in two formats as follows:
Generic format (Same value across all LDAP servers): com.unraveldata.ldap.customLDAPQuery:
LDAP specific format (Different value for a specific LDAP server): com.unraveldata.ldap.<ldap_id>.custom.query.filter
None
Old property name | New property name |
|---|---|
Role-based Access Control (RBAC) | |
com.unraveldata.saml.groupFilter Restricts access to a few selected SAML groups. | com.unraveldata.login.groupFilter |
com.unraveldata.rbac.default Determines how an end-user's views are filtered when no specific tags are set for them. | com.unraveldata.rbac.user.field |
com.unraveldata.rbac.ldap.tags / com.unraveldata.rbac.saml.tags A comma-separated list of the tags. For example, proj, dept. | com.unraveldata.login.tags |
com.unraveldata.rbac.ldap.tag.<TAG_KEY>.regex.find / com.unraveldata.rbac.saml.tag.<TAG_KEY>.regex.find Defines regular expression used to extract a tag_value for a given tag_key. | com.unraveldata.login.tag.<tag_name>.regex.find |
com.unraveldata.login.admins Defines admin users. | com.unraveldata.rbac.role.admin.users |
com.unraveldata.login.admins.readonly Defines read-only admin. users | com.unraveldata.rbac.role.readonlyAdmin.users |
com.unraveldata.login.admins.groups / com.unraveldata.login.admins.ldap.groups / com.unraveldata.login.admins.saml.groups Defines admin groups. | com.unraveldata.rbac.role.admin.groups |
com.unraveldata.login.admins.readonly.groups / com.unraveldata.login.admins.readonly.groups / com.unraveldata.login.admins.readonly.saml.groups Defines read-only admin groups. | com.unraveldata.rbac.role.readonlyAdmin.groups |
com.unraveldata.ldap.verbose
com.unraveldata.login.hbase.groups
com.unraveldata.login.clusters
com.unraveldata.login.clusters.<clusterid>.groups
com.unraveldata.ngui.user.mode
4.7.2.2
Property/Description | Default |
|---|---|
LDAP | |
com.unraveldata.ldap.ids A colon-separated list of LDAP servers (domains). In case this property is not defined then the default value is considered. | default_ldap_id |
com.unraveldata.ldap.default.id The default LDAP server from the com.unraveldata.ldap.ids list. If this property is not defined, the first LDAP server from com.unraveldata.ldap.ids is used | - |
The following existing properties are updated for consistency and replaced. However backward compatibility is provided for all the existing properties.
LDAP
Existing properties | New properties | |
|---|---|---|
Generic usage | LDAP specific usage | |
com.unraveldata.ldap.baseDN | com.unraveldata.ldap.base.dn | com.unraveldata.ldap.<ldap_id>.base.dn |
com.unraveldata.ldap.bind_dn | com.unraveldata.ldap.bind.dn | com.unraveldata.ldap.<ldap_id>.bind.dn |
com.unraveldata.ldap.bind_pw | com.unraveldata.ldap.bind.pw | com.unraveldata.ldap.<ldap_id>.bind.pw |
com.unraveldata.ldap.groupClassKey | com.unraveldata.ldap.group.class | com.unraveldata.ldap.<ldap_id>.group.class |
com.unraveldata.ldap.groupDNPattern | com.unraveldata.ldap.group.dn.pattern | com.unraveldata.ldap.<ldap_id>.group.dn.pattern |
com.unraveldata.ldap.groupMembershipKey | com.unraveldata.ldap.group.member.attr | com.unraveldata.ldap.<ldap_id>.group.member.attr |
com.unraveldata.ldap.groupSearchMethods | com.unraveldata.ldap.group.search.methods | com.unraveldata.ldap.<ldap_id>.group.search.methods |
com.unraveldata.ldap.guidKey | com.unraveldata.ldap.uid.attr | com.unraveldata.ldap.<ldap_id>.uid.attr |
com.unraveldata.ldap.mailAttribute | com.unraveldata.ldap.mail.attr | com.unraveldata.ldap.<ldap_id>.mail.attr |
com.unraveldata.ldap.realUserAttribute | com.unraveldata.ldap.real.uid.attr | com.unraveldata.ldap.<ldap_id>.real.uid.attr |
com.unraveldata.ldap.userDNPattern | com.unraveldata.ldap.user.dn.pattern | com.unraveldata.ldap.<ldap_id>.user.dn.pattern |
com.unraveldata.ldap.groupFilter | com.unraveldata.login.groupFilter | NA |
com.unraveldata.ldap.userFilter | com.unraveldata.login.userFilter | NA |
com.unraveldata.ldap.SAMAccountNameFilter | com.unraveldata.ldap.sAMAccountName.enabled | com.unraveldata.ldap.<ldap_id>.sAMAccountName.enabled |
com.unraveldata.ldap.groupQueryFilter | com.unraveldata.ldap.group.query.filter | com.unraveldata.ldap.<ldap_id>.group.query.filter |
None
None
com.unraveldata.ldap.Domain
This property is deprecated, however, backward compatibility is provided. com.unraveldata.ldap.default.id is used in place of the deprecated property.
4.7.2.0
Property/Description | Default |
|---|---|
Azure Active Directory (AAD) Properties to be updated in | |
tenantID ID of the AAD directory. | - |
Migration | |
com.unraveldata.migrationplanning.vmprices.<cloud provider> Set the cloud provider and the path to the CSV file which contains the custom prices for instances. The following values can be used for the cloud provider:
| - |
Insights Overview | |
com.unraveldata.valuedashboard.enabled Enables the Insights Overview tab on Unravel UI. | false |
Databricks | |
com.unraveldata.agent.metrics.ganglia_enabled Enables Ganglia metrics, which is supported by the Btrace agent. To enable Ganglia metrics, Dcom.unraveldata.agent.metrics.ganglia_enabled is set to true in extraJavaOptions. | |
Datastore | |
com.unraveldata.datastore.rest.api.timeout.millis Specifies the timeout period for different APIs used by UI to fetch data. | 120000 |
AAD Integration
unravel_mapping property includes the following:
username: Any username attribute to use.
group: Group name.
For example:
"unravel_mapping": { "username": "upn", "group": "name" }For more details refer to AAD Authentication.
None
None
None
4.7.1.0
Property/Description | Default |
|---|---|
Migration | |
com.unraveldata.migrationplanning.supported.cloudproviders Controls the cloud providers that must be displayed for selection for the migration reports. Values are as follows:
You can provide a comma-separated list of cloud providers when you specify this property. For example: com.unraveldata.migrationplanning.supported.cloudproviders EMR,EC2,HDI | - |
com.unraveldata.migrationplanning.blacklist Provides the path to a CSV or TXT file that can be used to blacklist specific VM instances for all cloud providers. The CSV or TXT file contains a comma-separated list of VM instances that must not be displayed for the migration reports. For example: com.unraveldata.migrationplanning.blacklist /opt/unravel/data/migration/blacklist.txt. | - |
com.unraveldata.migrationplanning.blacklist. Provides the path to a CSV or TXT file that can be used to blacklist specific VM instances for specific cloud providers. The CSV or TXT file contains a comma-separated list of VM instances that must not be displayed for the migration reports. The values for
For example: com.unraveldata.migrationplanning.blacklist.EMR /opt/unravel/data/migration/blacklist.txt. | - |
Azure Active Directory (AAD) integration Properties to be updated in | |
clientID Client ID from Overview > Essentials > Application (client) ID. | - |
clientSecret Client secret. | - |
callbackURL Redirect URL to consume login response from AAD. | - |
logoutCallbackURL Redirect URL to consume logout response from AAD. | - |
logoutEnabled Enable/disable logout on AAD when the user logs out from Unravel. | - |
unravel_mapping Maps the AAD attributes to Unravel attributes. | - |
Yarn log processing for Tez apps | |
com.unraveldata.yarn.log.delayForRetry Delay in milliseconds for consecutive retries when loading the yarn log files. | 2000 ms |
com.unraveldata.yarn.log.maxAttempts A maximum number of attempts for loading the yarn log files. | 3 |
None
None
None
com.unraveldata.migrationplanning.workloadfit.max.apps
Configures the limit on the maximum number of apps shown in a single slice of workload fit report and heatmap generation.