grain.experimental module

Experimental Grain APIs.

List of Members

FlatMapTransform()	Abstract base class for splitting operations of individual elements.
DatasetOptions(*[, ...])	Holds options used by dataset transformations.
ExecutionTrackingMode(*values)	Represents different modes for tracking execution statistics.
apply_transformations(ds, transformations)	Applies transformations to a dataset.
ElasticIterator(ds, global_batch_size, ...)	Iterator supporting recovery from a checkpoint after changes in sharding.
WithOptionsIterDataset(parent, options)	Applies options to transformations in the pipeline.
ParquetIterDataset(path, **read_kwargs)	An IterDataset for a parquet format file.
FlatMapMapDataset(parent, transform)	Flat map for one-to-many split.
FlatMapIterDataset(parent, transform)	Flat map for one-to-many split.
InterleaveIterDataset(datasets, *, cycle_length)	Interleaves the given sequence of datasets.
LimitIterDataset(parent, count)	Limits the number of elements in the dataset.
RngPool(seed)	RNG pool.
FirstFitPackIterDataset(parent, *, ...[, ...])	Implements first-fit packing of sequences.
BOSHandling(*values)	The BOS handling done inside a packing algorithm.
ConcatThenSplitIterDataset(parent, *, ...[, ...])	Implements concat-then-split packing for sequence features.
ThreadPrefetchIterDataset(parent, *, ...)	Iterable dataset that uses a synchronized queue for prefetching.
ThreadPrefetchDatasetIterator(parent, ...)	Iterator that performs prefetching using a synchronized queue.
WindowShuffleMapDataset(parent, *, ...)	Shuffles the parent dataset within a given window.
WindowShuffleIterDataset(parent, *, ...)	Shuffles the parent dataset within a given window.
ZipMapDataset(parents)	Combines MapDatasets of the same length to return a tuple of items.
ZipIterDataset(parents, *[, strict])	Combines IterDatasets of the same length to return a tuple of items.
PackAndBatchOperation(length_struct, batch_size)	PyGrain pack-and-batch operation - see module docstring.
assert_equal_output_after_checkpoint(ds)	Tests restoring an iterator to various checkpointed states.
device_put(ds, device, *[, cpu_buffer_size, ...])	Moves the data to the given devices with prefetching.
batch_and_pad(values, *, batch_size[, pad_value])	Batches the given values and, if needed, pads the batch to the given size.