nlpstack.common.iterutil module#
- class nlpstack.common.iterutil.SizedIterator(iterator, size)[source]#
Bases:
Generic[T]A wrapper for an iterator that knows its size.
- Parameters:
iterator (
Iterator[TypeVar(T)]) – The iterator.size (
int) – The size of the iterator.
- nlpstack.common.iterutil.batched(iterable, batch_size, drop_last=False)[source]#
Batch an iterable into lists of the given size.
- Parameters:
iterable (
Iterable[TypeVar(T)]) – The iterable.batch_size (
int) – The size of each batch.drop_last (
bool) – Whether to drop the last batch if it is smaller than the given size.
- Return type:
Iterator[List[TypeVar(T)]]- Returns:
An iterator over batches.
- nlpstack.common.iterutil.batched_iterator(iterable, batch_size)[source]#
Batch an iterable into iterators of the given size.
- Parameters:
iterable (
Iterable[TypeVar(T)]) – The iterable.batch_size (
int) – The size of each batch.
- Return type:
Iterator[Iterator[TypeVar(T)]]- Returns:
An iterator over batches.
- nlpstack.common.iterutil.iter_with_callback(iterable, callback)[source]#
Iterate over an iterable and call a callback for each item.
- Parameters:
iterable (
Iterable[TypeVar(T)]) – The iterable.callback (
Callable[[TypeVar(T)],Any]) – The callback to call for each item.
- Return type:
Iterator[TypeVar(T)]- Returns:
An iterator over the iterable.
- nlpstack.common.iterutil.wrap_iterator(wrapper, iterable)[source]#
Wrap an iterator with a function.
Note
This function assume that the wrapped iterator is of the same size as the input iterator.
- Parameters:
wrapper (
Callable[[Iterable[TypeVar(T)]],Iterator[TypeVar(T)]]) – The function to wrap the iterator.iterable (Iterable[T]) –
- Return type:
Iterator- Returns:
An iterator wrapped with the function.