v4.4.1 - New configuration properties
New report
Files reports & Small Files report
Property | Description | Unit | Default |
---|---|---|---|
NOTE: You must restart the unravel_ondemand and unravel_ngui daemons for changes to take effect. | |||
unravel.python.reporting.files.disable | Enables or disables Unravel ability to generate Small File and File Reports. True: disables the functionality in the Backend and UI. False: enables the ability to generate the report. | boolean | false |
File report
Property | Description | Unit | Default |
---|---|---|---|
NOTE: You must restart the unravel_ondemand and unravel_ngui_daemons for any changes to take effect. prepend: unravel.python.reporting.files. | |||
huge_files_threshold_size | file size >= threshold_size | bytes | 100GB |
huge_files_min_files | number of files in directory >= min_files | count | 1 |
huge_files_top_n_dirs | maximum number of directories to display | count | 10 |
medium_files_max_threshold_size | file size <= max_threshold_size | bytes | 10GB |
medium_files_min_threshold_size | file size >= min_threshold_size | bytes | 5GB |
medium_files_min_files | number of files in directory >= min_files | count | 5 |
medium_files_top_n_dirs | maximum number of directories to display | count | 20 |
tiny_files_threshold_size | file size <= threshold_size | bytes | 100KB |
tiny_files_min_files | maximum number of directories to display | count | 10 |
tiny_files_top_n_dirs | maximum number of directories to display | count | 30 |
empty_files_min_files | number of files in directory >= min_files | count | 10 |
empty_files_top_n_dirs | maximum number of directories to display | count | 30 |
Workflow
Property | Description | Unit | Default |
---|---|---|---|
prepend: com.unraveldata.workflow. | |||
num.past.instances | count | 1 | |
num.future.instances | count | 10 |
Spark
Property | Description | Unit | Default |
---|---|---|---|
prepend: com.unraveldata | |||
spark.master | This property is used if | string | yarn |
Yarn
Property | Description | Unit | Default |
---|---|---|---|
prepend: com.unraveldata | |||
attempt.log.max.containers | If number of containers of an YARN app > this value, log is not loaded and and getSizeOfPartition API is not called to check size of log directory. | count | 500 |
min.failed.job.duration.for.attempt.log | Load YARN log for failed/killed apps only when duration in seconds > this property, similar to com.unraveldata.min.job.duration.for.attempt.log for successful apps | count | 0 |
Deprecated
None