Kafka
Property/Definition | Set by user | Unit | Default |
|---|---|---|---|
com.unraveldata.ext.kafka.clusters Cluster list. These user-defined names are used to clearly identify the Kafka cluster in the Unravel UI. Use a comma-separated list for multiple clusters. | Required | CSL | - |
com.unraveldata.ext.kafka. List of brokers that are used to retrieve initial information about the Kafka cluster. For each cluster in com.unraveldata.ext.kafka.clusters you must define the associated brokers. Use a comma-separated list for multiple brokers. For example, com.unraveldata.ext.kafka.East.bootstrap_servers=tnode1.unraveldata.com:9092,tnode2.unraveldata.com:9093. | Required | CSL | - |
com.unraveldata.ext.kafka. Aliases for each Kafka nodes in the clusters with JMX ports exposed. You must assign aliases to the cluster nodes. Use a comma-separated list for multiple nodes. For example, unraveldata.ext.kafka.East.jmx_servers=kNode-1, kNode-2/. | Required | CSL | - |
com.unraveldata.ext.kafka. The host for a node in the cluster. You must define a host for each node in each cluster. For example, com.unraveldata.ext.kafka.East.kNode1=tnode1.unraveldata.com com.unraveldata.ext.kafka.East.kNode2=tnode2.unraveldata.com. | Required | - | |
com.unraveldata.ext.kafka. For each node in each cluster, you must assign a port. For example, com.unraveldata.ext.kafka.East.jmx.kNode1.port=5005. | Required | number | - |
Tip
To locate Kafka and JMX ports:
Cloudera Manager. Navigate to: Clusters → Kafka → Configuration → Ports and Addresses.
Alternatively, you may lookup up the information in the broker nodes of Zookeeper CLI.
HDP: For Protocol and broker port navigate to: Kafka → Configs → Kafka Broker.
JMX port navigate to: Kafka → Configs → Advanced kafka-env → kafka-env template.
Property/Definition | Set by user | Unit | Default |
|---|---|---|---|
com.unraveldata.ext.kafka.aggs.broker | integer | 100 | |
com.unraveldata.ext.kafka.aggs.broker_metrics | integer | 1000 | |
com.unraveldata.ext.kafka.aggs.consumer_group | integer | 1000 | |
com.unraveldata.ext.kafka.insight.interval_min | min | 15 | |
com.unraveldata.ext.kafka.insight.lag_threshold | integer | 100 | |
com.unraveldata.ext.kafka.insight.num_ignored_intervals | integer | 2 | |
com.unraveldata.ext.kafka.insight.sw_size | integer | 30 | |
com.unraveldata.ext.kafka.servers Use a comma separated list for multiple servers. | Required | CSL | kafka-test1,kafka-test2 |
com.unraveldata.kafka.broker_list | 127.0.0.1:4091 | ||
com.unraveldata.monitoring.kafka.check.interval How often Kafka metrics should be queried. Set 0 for disabling Kafka monitoring. | sec | 30 | |
com.unraveldata.monitoring.kafka.history.size How many data sets consider in computing average values. | integer | 5 | |
com.unraveldata.monitoring.kafka.ignore.topics Topics to ignore. Use a comma separated list for multiple topics. |
| CSL | __consumer_offsets, connect-configs, connect-offsets, connect-status |