Merge pull request #51 from pepkit/dev

Dev to master

Author: nsheff

README.md

@@ -11,61 +11,74 @@ Documentation is written using [mkdocs](https://www.mkdocs.org/) and themed with
 
 Each tool gets a `nav` section in `mkdocs.yml`, which maps to its own section/tab in the rendered documentation. So to add a new page, change titles, or change structure, edit `mkdocs.yml`. To edit the documentation itself, edit the `.md` documentation files in the subfolders under `/docs`.
 
-### Prereqs
+### Prerequisites
 
+```bash
+pip install mkdocs-material mkdocstrings[python] mkdocs-jupyter
 ```
-pip install mkdocs-material
+
+You'll also need to install the packages being documented (peppy, looper, pipestat, pypiper, geofetch, eido, yacman) for the API documentation to build correctly:
+
+```bash
+pip install peppy looper pipestat pypiper geofetch eido yacman
 ```
 
 
 ### Building locally for development
 
 I recommend previewing your changes locally before deploying. You can get a hot-reload server going by cloning this repository, and then just running:
 
-```
+```bash
 mkdocs serve
 ```
 
 You can also use `mkdocs build` to build a portable local version of the docs.
 
+The documentation now uses **mkdocstrings** for Python API documentation and **mkdocs-jupyter** for Jupyter notebooks. These plugins automatically generate documentation from the source code and render notebooks, so the build process is now a single step.
+
 
 ### Publishing updates
 
 The documentation is published automatically upon commits to `master` using a GitHub Action, which runs `mkdocs gh-deploy`. This builds the docs, and pushes them to the `gh-pages` branch. This branch is then published with GitHub Pages. There's no need to do this locally, just let the action deploy the updates for you automatically.
 
 ## FAQ
 
+### Python API Documentation
 
-### Updating automatic documentation
-
-In the past, I had a plugin that would auto-document 2 things: 1. Python docs using lucidoc, and 2. Jupyter notebooks. This plugin was neat, but it caused me a lot of maintenance issues as well. So now, I've made it much simpler; now it's no longer a plugin, just a simple Python script. Update all the auto-generated docs (stored in `docs/autodoc_build`) by running the update script manually:
+Python API documentation is now automatically generated using **mkdocstrings** during the build process. No separate script is needed. The API docs are defined in markdown files (e.g., `docs/peppy/code/python-api.md`) using the `:::` syntax:
 
-```console
-python autodoc.py
+```markdown
+::: peppy.Project
+    options:
+      docstring_style: google
+      show_source: true
 ```
 
-#### Configuring lucidoc rendering
+This syntax tells mkdocstrings to extract and render the documentation for the specified class or function directly from the source code.
 
-Auto-generated Python documentation with `lucidoc` rendering is configured in the `lucidoc`  sections of `mkdocs.yml`.
+### Jupyter Notebooks
+
+Jupyter notebooks are now rendered automatically using the **mkdocs-jupyter** plugin. Configure which notebooks to include in the `plugins` section of `mkdocs.yml`:
 
 ```yaml
-lucidoc:
-  peppy: path/to/output.md
+plugins:
+- mkdocs-jupyter:
+    include:
+      - peppy/notebooks/*.ipynb
+      - looper/notebooks/*.ipynb
 ```
 
-#### Configuring jupyter rendering
+Notebooks are rendered directly from `.ipynb` files during the build - no conversion step is needed.
 
-Configure jupyter notebeeoks in the `jupyter` section, where you specify a list of `in` (for `.ipynb` files) and `out` (for `.md` files) locations.
+### CLI Usage Documentation
 
-```yaml
-jupyter:
-  - in: path/to/notebook_folder1
-    out: path/to/rendered_folder1
-  - in: path/to/notebook_folder2
-    out: path/to/rendered_folder2
-``` 
-
-There, you can specify which folders contain notebooks, and to where they should be rendered as markdown.
+CLI usage documentation for geofetch can be updated manually when needed using the helper script:
+
+```bash
+python scripts/generate_cli_usage_docs.py
+```
+
+This script reads the template at `docs/geofetch/usage-template.md.tpl` and runs `geofetch --help` to generate `docs/geofetch/code/usage.md`. This only needs to be run when the CLI interface changes.
 
 ### Can we version the docs?
 

_typos.toml

@@ -4,4 +4,6 @@ extend-exclude = ["*.ipynb", "*.svg"]
 [default.extend-words]
 opf = "opf"
 PN="PN"
-Sur="Sur"
+Sur="Sur"
+certifi = "certifi"
+Tru = "Tru"

autodoc.py

@@ -80,7 +80,7 @@ eido validate peppro_paper.yaml -s http://schema.databio.org/pep/2.0.0.yaml -e
     Validation successful
 
 
-Any PEP should validate against that schema, which describes generic PEP format. We can go one step further and validate it against the PEPPRO schema, which describes Proseq projects specfically for this pipeline:
+Any PEP should validate against that schema, which describes generic PEP format. We can go one step further and validate it against the PEPPRO schema, which describes Proseq projects specifically for this pipeline:
 
 
 ```bash
@@ -144,7 +144,7 @@ eido validate -h
 
 Let's use `eido convert` command to convert PEPs to a variety of different formats. `eido` supports a plugin system, which can be used by other tool developers to create Python plugin functions that save PEPs in a desired format. Please refer to the documentation for more details. For now let's focus on a couple of plugins that are built-in in `eido`.
 
-To see what plugins are currently avaialable in your Python environment call:
+To see what plugins are currently available in your Python environment call:
 
 
 ```bash

docs/eido/code/cli.md

@@ -129,7 +129,7 @@ required:
   - samples
 ```
 
-PEPs to succesfully validate against this schema will need to fulfill all the generic PEP2.0.0 schema requirements _and_ fulfill the new `my_numeric_attribute` requirement.
+PEPs to successfully validate against this schema will need to fulfill all the generic PEP2.0.0 schema requirements _and_ fulfill the new `my_numeric_attribute` requirement.
 
 ### How importing works
 
@@ -306,7 +306,7 @@ validate_project(project=p, schema="../tests/data/schemas/test_schema_invalid.ya
 
 ## Config validation
 
-Similarily, the config part of the PEP can be validated; the function inputs remain the same
+Similarly, the config part of the PEP can be validated; the function inputs remain the same
 
 
 ```python
@@ -326,7 +326,7 @@ validate_sample(
 
 ## Output details
 
-As depicted above the error raised by the `jsonschema` package is very detailed. That's because the entire validated PEP is printed out for the user reference. Since it can get overwhelming in case of the multi sample PEPs each of the `eido` functions presented above privide a way to limit the output to just the general information indicating the unmet schema requirements
+As depicted above the error raised by the `jsonschema` package is very detailed. That's because the entire validated PEP is printed out for the user reference. Since it can get overwhelming in case of the multi sample PEPs each of the `eido` functions presented above provide a way to limit the output to just the general information indicating the unmet schema requirements
 
 
 ```python

docs/eido/code/demo.md

@@ -1,95 +1,42 @@
-<script>
-document.addEventListener('DOMContentLoaded', (event) => {
-  document.querySelectorAll('h3 code').forEach((block) => {
-    hljs.highlightBlock(block);
-  });
-});
-</script>
+# Eido Built-in Filters API
 
-<style>
-h3 .content { 
-    padding-left: 22px;
-    text-indent: -15px;
- }
-h3 .hljs .content {
-    padding-left: 20px;
-    margin-left: 0px;
-    text-indent: -15px;
-    martin-bottom: 0px;
-}
-h4 .content, table .content, p .content, li .content { margin-left: 30px; }
-h4 .content { 
-    font-style: italic;
-    font-size: 1em;
-    margin-bottom: 0px;
-}
+## Overview
 
-</style>
+Eido provides built-in filter functions that can transform PEP projects into different output formats. These filters are useful for converting PEPs to various representations like YAML, CSV, or other formats.
 
+### Available Filters
 
-# Package `eido` Documentation
+Eido includes several built-in filters for converting and exporting PEP data:
 
+- **basic_pep_filter**: Returns the basic PEP representation
+- **yaml_pep_filter**: Converts PEP to YAML format
+- **csv_pep_filter**: Exports sample tables as CSV
+- **yaml_samples_pep_filter**: Exports only sample data as YAML
 
-Project configuration
+## API Reference
 
-```python
-def basic_pep_filter(p, **kwargs) -> Dict[str, str]
-```
+### Filter Functions
 
-Basic PEP filter, that does not convert the Project object.
+::: eido.basic_pep_filter
+    options:
+      docstring_style: google
+      show_source: true
+      show_signature: true
 
-This filter can save the PEP representation to file, if kwargs include `path`.
-#### Parameters:
+::: eido.yaml_pep_filter
+    options:
+      docstring_style: google
+      show_source: true
+      show_signature: true
 
-- `p` (`peppy.Project`):  a Project to run filter on
+::: eido.csv_pep_filter
+    options:
+      docstring_style: google
+      show_source: true
+      show_signature: true
 
-
-
-
-```python
-def yaml_pep_filter(p, **kwargs) -> Dict[str, str]
-```
-
-YAML PEP filter, that returns Project object representation.
-
-This filter can save the YAML to file, if kwargs include `path`.
-#### Parameters:
-
-- `p` (`peppy.Project`):  a Project to run filter on
-
-
-
-
-```python
-def csv_pep_filter(p, **kwargs) -> Dict[str, str]
-```
-
-CSV PEP filter, that returns Sample object representations
-
-This filter can save the CSVs to files, if kwargs include
-`sample_table_path` and/or `subsample_table_path`.
-#### Parameters:
-
-- `p` (`peppy.Project`):  a Project to run filter on
-
-
-
-
-```python
-def yaml_samples_pep_filter(p, **kwargs) -> Dict[str, str]
-```
-
-YAML samples PEP filter, that returns only Sample object representations.
-
-This filter can save the YAML to file, if kwargs include `path`.
-#### Parameters:
-
-- `p` (`peppy.Project`):  a Project to run filter on
-
-
-
-
-
-
-
-*Version Information: `eido` v0.2.2, generated by `lucidoc` v0.4.4*
+::: eido.yaml_samples_pep_filter
+    options:
+      docstring_style: google
+      show_source: true
+      show_signature: true