- Home
- Unravel 4.5 Documentation
- Advanced topics
- Unravel properties
- Small files and file reports properties
Small files and file reports properties
Note
You must restart the unravel_ondemand daemon for any changes to take effect
Property/Description | Set by user | Unit | Default |
|---|---|---|---|
com.unraveldata.ngui.sfhivetable.schedule.interval Frequency, in days, in which to trigger FSimage extraction, for example, every 3 days The scheduler schedules extraction relative to the 1st of the month and then sets each extraction such that it 1st, 1st + X days, 1st + 2X days until 1st + nX days crosses into the next month, at which point the schedule resets to the 1st. See below for an example. Format: | day | 1d | |
com.unraveldata.ngui.sfhivetable.schedule.time Specify the time to download in hours (using 24 hour time) the FSimage. Format: two digits between 00 and 23. | two digits (member of set) | 00 | |
unravel.python.reporting.files.external_fsimage_dir Directory for fsimage when skip_fetch_fsimage=true. The fsimage externally fetched is expected to be in this directory. Unravel uses the latest file in this directory which starts with " fsimage_". This directory must be different than the Unravel's internal directory, i.e., /srv/unravel/tmp/reports/fsimage. | string | - | |
unravel.python.reporting.files.skip_fetch_fsimage If hdfs admin privileges can not be granted, set this to true to allow Unravel's Ondemand process to use an externally fetched FSimage.
| boolean | false |
Note
In order for the configuration changes to take effect, unravel_ondemand and unravel_ngui daemons need to be restarted.
/etc/init.d/unravel_ngui restart /etc/init.d/unravel_ondemand restart
Property/Description | Set by user | Unit | Default |
|---|---|---|---|
unravel.python.reporting.files.disable Enables or disables Unravel ability to generate Small Files and File Reports. Note
| boolean | false | |
unravel.python.reporting.files.hive_database Hive Database where Ondemand creates five Hive tables (four temporary, one permanent) for Small Files/File Reports. When not set, tables are created in the default Hive Database. In addition, the Hive queries used for this feature run against default MR queue. It must point to a valid Hive database. | string | default database | |
unravel.python.reporting.files.hive_mr_queue The Hcive queries ran by Ondemand process run against this MR queue. It must point to a valid MR queue. | string | default |
Note
These are global properties which apply to both Small Files Report and File Reports. They each have equivalent "local" properties. The "local" property takes precedence over the equivalent global property.
Should you unset/delete any of the below properties and their equivalent properties in Small Files Report or Files Report, Unravel has hard-coded values to ensure your reports are generated.
Property/Description | Set by user | Unit | Default |
|---|---|---|---|
unravel.python.reporting.files.files_use_avg_file_size_flag
| Optional | boolean | - |
unravel.python.reporting.files.min_parent_dir_depth Directory depth to end the search at. For instance, if depth=2, search begins below two levels. Give given HDFS_root/one/two the search starts in the directory two | Optional | count | - |
unravel.python.reporting.files.max_parent_dir_depth Directory depth to end search at. Maximum is 50. For instance, if depth=5 given HDFS_root/one/two/three/four/five/six/seven the search ends at five. | Optional | count | - |
unravel.python.reporting.drill_down_subdirs_flag When true a file is accounted for (listed) in all of its ancestors.
For example given the directory structure is /one/two false: / - lists files in /. /one - lists files in one. /one/two - lists files in /one/two. true: / - lists files in /, /one, and /one/two. /one - lists files in /one, and /one/two. /one/two - lists files in /one/two. | Optional | boolean | false |