grain.experimental.DatasetOptions#
- class grain.experimental.DatasetOptions(*, filter_warn_threshold_ratio=_Default(value=0.9), filter_raise_threshold_ratio=_Default(value=None), execution_tracking_mode=_Default(value=<ExecutionTrackingMode.DISABLED: 1>), min_shm_size=_Default(value=0))#
Holds options used by dataset transformations.
- Parameters:
filter_warn_threshold_ratio (float | None | _Default[float])
filter_raise_threshold_ratio (float | None | _Default[None])
execution_tracking_mode (ExecutionTrackingMode | _Default[ExecutionTrackingMode])
min_shm_size (int | _Default[int])
- filter_warn_threshold_ratio#
If the ratio of filtered out elements is above these thresholds, a warning will be issued. Value None disables the check. The ratio is calculated on non-overlapping windows of 1000 elements. For instance, with filter_warn_threshold_ratio=0.9 and 901 elements out of the first 1000 (or elements 1000…2000) filtered out, a warning will be issued.
- Type:
float | None | grain._src.python.dataset.base._Default[float]
- filter_raise_threshold_ratio#
If the ratio of filtered out elements is above these thresholds, an exception will be issued. Value None disables the check.
- Type:
float | None | grain._src.python.dataset.base._Default[None]
- execution_tracking_mode#
The collection of execution statistics like total processing time taken by each transformation, number of elements produced etc. can be managed through various modes. If DISABLED, no statistics are collected.If STAGE_TIMING, the time it takes to process each transormation is collected. See ExecutionTrackingMode for more details.
- Type:
grain._src.python.dataset.base.ExecutionTrackingMode | grain._src.python.dataset.base._Default[grain._src.python.dataset.base.ExecutionTrackingMode]
- min_shm_size#
The minimum size below which numpy arrays will copied between processes rather than passed via shared memory. For smaller arrays, the overhead of using shared memory can be higher than the cost of copying.
- Type:
int | grain._src.python.dataset.base._Default[int]
- __init__(*, filter_warn_threshold_ratio=_Default(value=0.9), filter_raise_threshold_ratio=_Default(value=None), execution_tracking_mode=_Default(value=<ExecutionTrackingMode.DISABLED: 1>), min_shm_size=_Default(value=0))#
- Parameters:
filter_warn_threshold_ratio (float | None | _Default[float])
filter_raise_threshold_ratio (float | None | _Default[None])
execution_tracking_mode (ExecutionTrackingMode | _Default[ExecutionTrackingMode])
min_shm_size (int | _Default[int])
- Return type:
None
Methods
__init__(*[, filter_warn_threshold_ratio, ...])merge(other)Merges these options with the other.
Attributes