doc stuff

Author: virgesmith

README.md

@@ -1,68 +1,115 @@
-# itr - a powerful `Iterable` wrapper
+# itr - A Powerful Iterable Wrapper
 
-Heavily inspired by rust's [Iterator trait](https://doc.rust-lang.org/std/iter/trait.Iterator.html),
-with added pythonicness (such as `starmap`)
+`itr` is a powerful Python library that wraps iterators, iterables, and generators, providing a Rust-inspired `Iterator` trait experience with added Pythonic conveniences. It enables developers to build complex data processing pipelines with a fluent, chainable, and lazy API. In most cases, it simply wraps itertools in syntactic sugar.
 
+Heavily inspired by Rust's [Iterator trait](https://doc.rust-lang.org/std/iter/trait.Iterator.html), `itr` offers a familiar and robust pattern for sequence manipulation.
 
-### Functionality
+## Key Features & Why `Itr`?
 
-`Itr` wraps python Iterators, Iterables and Generators providing the ability to arbitrarily chain (and lazily evaluate) methods, e.g.
+*   **Fluent & Chainable API:** Write expressive, left-to-right sequences of operations on your data.
+*   **Lazy Evaluation:** Operations are only performed when their results are needed, improving efficiency for large or infinite sequences.
+*   **Rust-Inspired Patterns:** Bring the power and clarity of Rust's `Iterator` design to your Python projects.
+*   **Pythonic Additions:** Includes convenient methods like `starmap` that integrate smoothly with Python's ecosystem.
+*   **Simplifies Complex Logic:** Often provides a more readable and concise alternative to nested `itertools` or built-in functions.
+
+## Quick Start & Core Functionality
+
+`Itr` wraps Python Iterators, Iterables, and Generators, allowing you to arbitrarily chain methods for data transformation.
 
 ```py
 >>> from itr import Itr
+>>> # Process a range of numbers with a concise, chainable syntax
 >>> Itr(range(100)).rev().step_by(4).skip(10).map(lambda x: x * x).filter(lambda x: x % 3 == 0).for_each(print)
-2304
-1296
-576
-144
-0
->>>
+2601
+1521
+729
+225
+9
+
 ```
 
-For reference, the equivalent expression using builtins and `itertools`:
+For reference, the equivalent expression using built-ins and `itertools` can be significantly less readable:
 
-```py
-from iterools import islice
+```python
+from itertools import islice
 
 for item in filter(
     lambda x: x % 3 == 0,
     map(lambda x: x * x, islice(islice(reversed(range(100)), None, None, 4), 10, None)),
 ):
     print(item)
+
 ```
 
+## How `Itr` Works: Lazy vs. Eager
+
+Most `Itr` methods are **lazy transformations**, meaning they return a new `Itr` instance without immediately processing any data. This allows for arbitrary chaining and efficient memory usage, as items are only processed as they are requested. In many cases, `Itr` acts as a convenient wrapper around `itertools`, enabling this left-to-right chaining syntax.
+
+However, some methods are **eager consumers**. These methods iterate over and consume the underlying data, returning concrete values, collections, or aggregates. Examples include:
+*   **Collection methods:** `collect`, `next`, `next_chunk`
+*   **Aggregation methods:** `count`, `reduce`, `max`, `min`, `all`, `any`, `last`, `find`, `fold`, `product`
 
-In many cases the `Itr` simply provides a wrapper around `functools` enabling chaining of operations in a left-to-right syntax.
+### Important Considerations
 
-Whilst many methods return either `self` or another instance/instances of `Itr` (thus allowing for chaining and lazy evaluation), some methods consume and return actual values (such as `collect`, `next`, `next_chunk`) or aggregates (such as `count`,`reduce`,`max`) from the iterable (i.e. eager evaluation). Note that:
+When working with `Itr`, keep these points in mind:
 
-- iterators can only be consumed once (use `copy`, `cycle` or `repeat` as necessary)
-- it's not possible to rewind, but the next value in the sequence can be "previewed" using the `peek` method. Doing this a lot is unlikely to be efficent though, as it copies the iterator.
-- be wary of open-ended iterators, eager evaluation can lead to infinite loops.
+*   **Single-Pass Iterators:** Like all Python iterators, `Itr` instances (and their underlying iterators) can generally only be consumed once. If you need to process the same sequence multiple times, use methods like `copy()`, `cycle()`, or `repeat()` as necessary.
+*   **No Rewinding:** It's not possible to rewind an `Itr` to an earlier state. You can "preview" the next value using the `peek()` method, but be aware that `peek()` often copies the iterator internally, which can be inefficient if used excessively.
+*   **Infinite Iterators:** Be cautious with open-ended iterators (e.g., those from `itertools.count()` or custom generators). Eager evaluation methods (like `collect()`, `count()`, `reduce()`) will attempt to consume the entire sequence, potentially leading to infinite loops or out-of-memory errors if applied to an infinite source.
 
-Methods implemented (see the docstrings for more details):
+## API Reference
+
+`Itr` provides a comprehensive set of methods for various iterable operations. For a complete list of methods and their detailed descriptions, please refer to the [full documentation](autodoc.md).
+
+*Note: The detailed list of methods in `autodoc.md` is automatically generated by [introspect.py], where `Itr` introspects itself.*
+
+```py introspect.py
+from operator import add
+
+from itr import Itr
+
+# use Itr to introspect itself for methods and their docstrings
+methods = (
+    Itr(dir(Itr))
+    .filter(lambda m: m in ("__init__", "__iter__", "__next__") or not m.startswith("_"))
+    .map(lambda m: (m, getattr(Itr, m).__doc__))
+    .collect(dict)
+)
 
-`__iter__`, `__next__`, `all`, `any`, `batched`, `chain`, `collect`, `copy`, `count`, `cycle`, `diff`, `enumerate`, `filter`, `find`, `flat_map`, `flatten`, `fold`, `for_each`, `groupby`, `interleave`, `intersperse`, `last`, `map`, `max`, `min`, `next`, `next_chunk`, `nth`, `pairwise`, `partition`, `peek`, `product`, `reduce`, `repeat`, `rev`, `rolling`, `skip`, `skip_while`, `starmap`, `step_by`, `take`, `take_while`, `unzip`, `zip`
+# use Itr to make some documentation
+method_template = """
+### `{method_name}`
 
-This list is auto-generated by `Itr` introspecting itself, which also generates [more detailed documentation](autodoc.md)
+{method_doc}
+"""
 
-The code is here: [introspect.py](introspect.py)
+with open("doc/autodoc.md", "w") as fd:
+    fd.write("# `Itr` class documentation\n")
+    fd.write(Itr.__doc__ or "")
+    fd.write("## Public methods\n")
+    fd.write(Itr(methods.items()).map(lambda m: method_template.format(method_name=m[0], method_doc=m[1])).reduce(add))
+```
 
 ## Examples
 
-### Inputs
+### Input Flexibility
 
-Accepts iterables (sequences and ranges), iterators or generators:
+`Itr` is designed to work seamlessly with various Python iterable types, including sequences, ranges, custom iterators, and generators.
 
 ```py
+from collections.abc import Iterator
+from typing import Self, Generator
 from itr import Itr
 
+# From a list (iterable)
 it1 = Itr([0, 1, 2, 3, 4])
 assert it1.collect() == (0, 1, 2, 3, 4)
 
+# From a range (iterable)
 it2 = Itr(range(5))
 assert it2.collect() == (0, 1, 2, 3, 4)
 
+# From a custom iterator (implementing __iter__ and __next__)
 class Fib(Iterator):
     def __init__(self) -> None:
         self.a = 0
@@ -79,6 +126,7 @@ class Fib(Iterator):
 it3 = Itr(Fib())
 assert it3.take(10).collect() == (0, 1, 1, 2, 3, 5, 8, 13, 21, 34)
 
+# From a generator function
 def fibgen() -> Generator[int, None, None]:
     a, b = 0, 1
     while True:
@@ -89,59 +137,78 @@ it4 = Itr(fibgen())
 assert it4.take(10).collect() == (0, 1, 1, 2, 3, 5, 8, 13, 21, 34)
 ```
 
-### Outputs
+### Output Options with `collect()`
 
-By default, the `collect()` method returns a `tuple`, but this can be changed to `list`, `set` or `dict`. Here's how to group a words by their length into a dictionary:
+While `Itr` methods typically return another `Itr` instance, the `collect()` method allows you to materialize the results into various Python collections: `tuple` (default), `list`, `set`, or `dict`.
 
-```py
+Here's how to group words by their length into a dictionary:
+
+```python
 >>> from itr import Itr
 >>> Itr(("apple", "banana", "carrot")).groupby(len).collect(dict)
 {5: ('apple',), 6: ('banana', 'carrot')}
->>>
+
 ```
 
+For reference, an equivalent using `itertools` directly:
+
 ```py
+>>> import itertools
 >>> {k: tuple(v) for k, v in itertools.groupby(("apple", "banana", "carrot"), key=len)}
+{5: ('apple',), 6: ('banana', 'carrot')}
+
 ```
 
+*Note: Using `collect(dict)` requires an iterable that produces 2-tuples (key-value pairs).*
 
-NB using `dict` requires an iterable that produces 2-tuples.
+### More examples
 
-### Composition
+More examples can be found [here](./doc/examples.md).
 
-Numerical differentiation and integration:
 
-```py
+
+
+
+Beyond basic transformations, `Itr` enables powerful composition of operations, making complex data processing more manageable. This example demonstrates numerical differentiation and integration using `Itr`.
+
+```python
 from itr import Itr
 import numpy as np
+from operator import add
 
 x = Itr(np.linspace(0, 1, 11))
-# need a copy as the original x is consumed in the zip method
+# Need a copy of 'x' as the original 'x' is consumed in the zip method.
 xy = x.copy().zip(x.map(lambda xi: xi * xi / 2))
 
-def diff1central(xy):
-    (x0, y0), (x1, _y1), (x2, y2) = xy
+# Numerical differentiation (central difference)
+def diff1central(xy_segment):
+    # Unpack the 3-element rolling window: (x0, y0), (x1, y1_ignored), (x2, y2)
+    (x0, y0), (x1, _y1), (x2, y2) = xy_segment
     return x1, (y2 - y0) / (x2 - x0)
 
+print("Numerical Derivatives:")
 print(
-    xy.copy() # need a copy as reusing xy below
+    xy.copy() # Need a copy as xy will be reused below for integration
     .rolling(3)
-    .map(diff1central)  # x, dy/dx pairs
+    .map(diff1central)  # Produces (x, dy/dx) pairs
     .collect()
 )
 
-# similarly, the definite integral x**3/6 -> using the trapezoidal rule over 0,1 = 1/6
-def trapz(sum, xy):
-    (x0, y0), (x1, y1) = xy
-    return sum + 0.5 * (y0 + y1) * (x1 - x0)
-
-print(xy.pairwise().fold(0, trapz))
+# Numerical integration (trapezoidal rule)
+# Integrates x**2 / 2, resulting in x**3 / 6. Over 0,1 this is 1/6.
+def trapz_accumulate(current_sum, xy_pair):
+    # Unpack (x0, y0) and (x1, y1) from the pairwise iteration
+    (x0, y0), (x1, y1) = xy_pair
+    return current_sum + 0.5 * (y0 + y1) * (x1 - x0)
 
-# ...or an alternative implementation using starmap and two levels of flattening
+print("\nNumerical Integral (Trapezoidal Rule - Fold):")
+print(xy.pairwise().fold(0, trapz_accumulate))
 
-def trapz2(x0, y0, x1, y1):  # type: ignore[no-untyped-def]  # noqa: D103
+# ...or an alternative integration implementation using starmap and flattening
+def trapz2(x0, y0, x1, y1):  # type: ignore[no-untyped-def]
+    """Calculates trapezoidal area for two points."""
     return (y0 + y1) * (x1 - x0) / 2
 
+print("\nNumerical Integral (Trapezoidal Rule - Starmap):")
 print(xy.copy().pairwise().flatten().flatten().batched(4).starmap(trapz2).reduce(add))
-
 ```

autodoc.md doc/autodoc.mdautodoc.md renamed to doc/autodoc.md

@@ -114,21 +114,12 @@ Example:
     >>> itr = Itr([1, 2, 3])
     >>> cycler = itr.cycle()
     >>> cycler.take(5).collect()
-    [1, 2, 3, 1, 2]
-
-
-### `diff`
-
-
-Convenience method for differencing a sequence
-T must have a "-" operator
-n must be != 0
-if n is negative, diff is reversed
+    (1, 2, 3, 1, 2)
 
 
 ### `enumerate`
 
-Yield pairs of (index, item) for each item in the iterator.
+Yield pairs of (index, item) for each item in the iterator, where index starts at 0 or the value provided
 
 Returns:
     Itr[tuple[int, T]]: An iterator of (index, item) pairs.