Elasticsearch storage requirements on the Unravel Node
When you are using HBase you must ensure you have enough disk space to accommodate the Elasticsearch Index on the Unravel node. The volume (size) of metrics that Unravel collects is dependent on the following:
Number of
Master servers
Region servers
Tables
Table regions
com.unraveldata.hbase.metric.poll.interval
The total disk size you need for one week's metrics is equal to
(total size per poll) * (number of polls per week)
where
total size per poll = (total documents per poll) * (avg doc size)
total documents per poll = (12 * #_of_Master_Servers) + 3 + (19 * #_of_Region_Servers)
avg doc size = ±100 bytes
number of polls per week = (7 * 24 * 60) / com.unraveldata.hbase.metric.poll.interval
Important
avg doc size is a rough estimate and is an average across master, region server, table, table region, and OS metrics. It varies depending on the content, for example, namespace, host name, table name, etc.
The polling interval determines both the granularity of the collected data and the storage size needed. The more frequent the collection the larger the storage space required. For instance, if you need 100 GB using a five-minute polling, you need 50 GB for a 10-minute interval.