Skip to content

Commit 8832370

Browse files
gaudybGaudy Blanco
gaudyb
and
Gaudy Blanco
authored
Add notebook example support for each package (#2205)
* add notebook example support for each package * add notebook example support for each package * semversioner change * feedback implemented for notebooks * feedback implemented for notebooks * feedback implemented for notebooks --------- Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
1 parent e3278c0 commit 8832370

37 files changed

+3619
-2340
lines changed

‎.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,3 +63,7 @@ docsite/
6363

6464
# Root build assets
6565
packages/*/LICENSE
66+
67+
# Notebooks outputs
68+
packages/graphrag-*/example_notebooks/**/output
69+
packages/graphrag-*/example_notebooks/**/cache

‎.semversioner/next-release/patch-20260205172351041030.json‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"type": "patch",
3+
"description": "update notebooks"
4+
}

‎docs/examples_notebooks/api_overview.ipynb‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,10 +28,11 @@
2828
"from pathlib import Path\n",
2929
"from pprint import pprint\n",
3030
"\n",
31-
"import graphrag.api as api\n",
3231
"import pandas as pd\n",
3332
"from graphrag.config.load_config import load_config\n",
34-
"from graphrag.index.typing.pipeline_run_result import PipelineRunResult"
33+
"from graphrag.index.typing.pipeline_run_result import PipelineRunResult\n",
34+
"\n",
35+
"import graphrag.api as api"
3536
]
3637
},
3738
{

‎docs/examples_notebooks/input_documents.ipynb‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,10 +30,11 @@
3030
"from pathlib import Path\n",
3131
"from pprint import pprint\n",
3232
"\n",
33-
"import graphrag.api as api\n",
3433
"import pandas as pd\n",
3534
"from graphrag.config.load_config import load_config\n",
36-
"from graphrag.index.typing.pipeline_run_result import PipelineRunResult"
35+
"from graphrag.index.typing.pipeline_run_result import PipelineRunResult\n",
36+
"\n",
37+
"import graphrag.api as api"
3738
]
3839
},
3940
{

‎packages/graphrag-cache/README.md‎

Lines changed: 3 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -6,80 +6,13 @@ This package contains a collection of utilities to handle GraphRAG caching imple
66

77
This example shows how to create a JSON cache with file storage using the GraphRAG cache package's configuration system.
88

9-
```python
10-
import asyncio
11-
from graphrag_storage import StorageConfig, create_storage, StorageType
12-
from graphrag_cache import CacheConfig, create_cache, CacheType, create_cache_key
13-
14-
async def run():
15-
cache = create_cache()
16-
17-
# The above is equivalent to the following:
18-
cache = create_cache(
19-
CacheConfig(
20-
type=CacheType.Json,
21-
storage=StorageConfig(
22-
type=StorageType.File,
23-
base_dir="cache"
24-
)
25-
),
26-
)
27-
28-
await cache.set("my_key", {"some": "object to cache"})
29-
print(await cache.get("my_key"))
30-
31-
# create cache key from data dict.
32-
cache_key = create_cache_key({
33-
"some_arg": "some_value",
34-
"something_else": 5
35-
})
36-
await cache.set(cache_key, {"some": "object to cache"})
37-
print(await cache.get(cache_key))
38-
39-
if __name__ == "__main__":
40-
asyncio.run(run())
41-
```
9+
[Open the notebook to explore the basic example code](example-notebooks/basic_cache_example.ipynb)
4210

4311
### Custom Cache
4412

45-
This demonstrates how to create a custom cache implementation by extending the base Cache class and registering it with the GraphRAG cache system. Once registered, the custom cache can be instantiated through the factory pattern using either CacheConfig or directly via cache_factory, allowing for extensible caching solutions tailored to specific needs.
13+
This example demonstrates how to create a custom cache implementation by extending the base Cache class and registering it with the GraphRAG cache system. Once registered, the custom cache can be instantiated through the factory pattern using either CacheConfig or directly via cache_factory, allowing for extensible caching solutions tailored to specific needs.
4614

47-
```python
48-
import asyncio
49-
from typing import Any
50-
from graphrag_storage import Storage
51-
from graphrag_cache import Cache, CacheConfig, create_cache, register_cache
52-
53-
class MyCache(Cache):
54-
def __init__(self, some_setting: str, optional_setting: str = "default setting", **kwargs: Any):
55-
# Validate settings and initialize
56-
# View the JsonCache implementation to see how to create a cache that relies on a Storage provider.
57-
...
58-
59-
#Implement rest of interface
60-
...
61-
62-
register_cache("MyCache", MyCache)
63-
64-
async def run():
65-
cache = create_cache(
66-
CacheConfig(
67-
type="MyCache",
68-
some_setting="My Setting"
69-
)
70-
)
71-
72-
# Or use the factory directly to instantiate with a dict instead of using
73-
# CacheConfig + create_factory
74-
# from graphrag_cache.cache_factory import cache_factory
75-
# cache = cache_factory.create(strategy="MyCache", init_args={"some_setting": "My Setting"})
76-
77-
await cache.set("my_key", {"some": "object to cache"})
78-
print(await cache.get("my_key"))
79-
80-
if __name__ == "__main__":
81-
asyncio.run(run())
82-
```
15+
[Open the notebook to explore the basic custom example code](example-notebooks/custom_cache_example.ipynb)
8316

8417
#### Details
8518

‎packages/graphrag-cache/example_notebooks/basic_cache_example.ipynb‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "code",
5+
"execution_count": 1,
6+
"id": "fcb917cf",
7+
"metadata": {},
8+
"outputs": [],
9+
"source": [
10+
"# Copyright (c) 2026 Microsoft Corporation.\n",
11+
"# Licensed under the MIT License."
12+
]
13+
},
14+
{
15+
"cell_type": "markdown",
16+
"id": "33461307",
17+
"metadata": {},
18+
"source": [
19+
"## Basic cache example\n",
20+
"\n",
21+
"This example shows how to create a JSON cache with file storage using the GraphRAG cache package's configuration system. "
22+
]
23+
},
24+
{
25+
"cell_type": "code",
26+
"execution_count": 2,
27+
"id": "ee33abd6",
28+
"metadata": {},
29+
"outputs": [
30+
{
31+
"name": "stdout",
32+
"output_type": "stream",
33+
"text": [
34+
"Value stored in cache for 'my_key':\n",
35+
"{'k1': 'object to cache'}\n",
36+
"\n",
37+
"Value stored in cache for cache_key using data dict:\n",
38+
"{'k2': 'object to cache'}\n"
39+
]
40+
}
41+
],
42+
"source": [
43+
"from graphrag_cache import CacheConfig, CacheType, create_cache, create_cache_key\n",
44+
"from graphrag_storage import StorageConfig, StorageType\n",
45+
"\n",
46+
"\n",
47+
"async def run():\n",
48+
" \"\"\"Demonstrate basic cache usage with graphrag_cache.\"\"\"\n",
49+
" cache = create_cache()\n",
50+
"\n",
51+
" # The above is equivalent to the following:\n",
52+
" cache = create_cache(\n",
53+
" CacheConfig(\n",
54+
" type=CacheType.Json,\n",
55+
" storage=StorageConfig(type=StorageType.File, base_dir=\"cache\"),\n",
56+
" ),\n",
57+
" )\n",
58+
"\n",
59+
" await cache.set(\"my_key\", {\"k1\": \"object to cache\"})\n",
60+
" print(\"Value stored in cache for 'my_key':\")\n",
61+
" print(await cache.get(\"my_key\"))\n",
62+
"\n",
63+
" # create cache key from data dict.\n",
64+
" cache_key = create_cache_key({\"some_arg\": \"some_value\", \"something_else\": 5})\n",
65+
" await cache.set(cache_key, {\"k2\": \"object to cache\"})\n",
66+
" print(\"\\nValue stored in cache for cache_key using data dict:\")\n",
67+
" print(await cache.get(cache_key))\n",
68+
"\n",
69+
"\n",
70+
"if __name__ == \"__main__\":\n",
71+
" await run()"
72+
]
73+
}
74+
],
75+
"metadata": {
76+
"kernelspec": {
77+
"display_name": "Python 3",
78+
"language": "python",
79+
"name": "python3"
80+
},
81+
"language_info": {
82+
"codemirror_mode": {
83+
"name": "ipython",
84+
"version": 3
85+
},
86+
"file_extension": ".py",
87+
"mimetype": "text/x-python",
88+
"name": "python",
89+
"nbconvert_exporter": "python",
90+
"pygments_lexer": "ipython3",
91+
"version": "3.12.9"
92+
}
93+
},
94+
"nbformat": 4,
95+
"nbformat_minor": 5
96+
}

‎packages/graphrag-cache/example_notebooks/custom_cache_example.ipynb‎

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "code",
5+
"execution_count": 1,
6+
"id": "42524e97",
7+
"metadata": {},
8+
"outputs": [],
9+
"source": [
10+
"# Copyright (c) 2026 Microsoft Corporation.\n",
11+
"# Licensed under the MIT License."
12+
]
13+
},
14+
{
15+
"cell_type": "markdown",
16+
"id": "a7cd6743",
17+
"metadata": {},
18+
"source": [
19+
"## Custom cache example\n",
20+
"\n",
21+
"This example demonstrates how to create a custom cache implementation by extending the base Cache class and registering it with the GraphRAG cache system. Once registered, the custom cache can be instantiated through the factory pattern using either CacheConfig or directly via cache_factory, allowing for extensible caching solutions tailored to specific needs."
22+
]
23+
},
24+
{
25+
"cell_type": "code",
26+
"execution_count": 2,
27+
"id": "e7ec8dc1",
28+
"metadata": {},
29+
"outputs": [
30+
{
31+
"name": "stdout",
32+
"output_type": "stream",
33+
"text": [
34+
"Value stored in cache for 'my_key':\n",
35+
"{'k1': 'object to cache'}\n"
36+
]
37+
}
38+
],
39+
"source": [
40+
"from typing import Any\n",
41+
"\n",
42+
"from graphrag_cache import Cache, CacheConfig, create_cache, register_cache\n",
43+
"\n",
44+
"\n",
45+
"class MyCache(Cache):\n",
46+
" \"\"\"Custom cache implementation for storing and retrieving cached data.\"\"\"\n",
47+
"\n",
48+
" def __init__(\n",
49+
" self,\n",
50+
" some_setting: str,\n",
51+
" optional_setting: str = \"default setting\",\n",
52+
" **kwargs: Any,\n",
53+
" ):\n",
54+
" # Validate settings and initialize\n",
55+
" # View the JsonCache implementation to see how to create a cache that relies on a Storage provider.\n",
56+
" self.some_setting = some_setting\n",
57+
" self.optional_setting = optional_setting\n",
58+
" self._cache: dict[str, Any] = {}\n",
59+
" self._child: Cache = self\n",
60+
"\n",
61+
" async def get(self, key: str) -> Any:\n",
62+
" \"\"\"Retrieve a value from the cache by key.\"\"\"\n",
63+
" return self._cache.get(key)\n",
64+
"\n",
65+
" async def set(self, key: str, value: Any, debug_data: Any = None) -> None:\n",
66+
" \"\"\"Store a value in the cache with the specified key.\"\"\"\n",
67+
" self._cache[key] = value\n",
68+
"\n",
69+
" async def has(self, key: str) -> bool:\n",
70+
" \"\"\"Check if a key exists in the cache.\"\"\"\n",
71+
" return key in self._cache\n",
72+
"\n",
73+
" async def delete(self, key: str) -> None:\n",
74+
" \"\"\"Remove a key and its value from the cache.\"\"\"\n",
75+
" self._cache.pop(key, None)\n",
76+
"\n",
77+
" async def clear(self) -> None:\n",
78+
" \"\"\"Clear all items from the cache.\"\"\"\n",
79+
" self._cache.clear()\n",
80+
"\n",
81+
" def child(self, name: str) -> Cache:\n",
82+
" \"\"\"Create or access a child cache (not implemented in this example).\"\"\"\n",
83+
" return self._child\n",
84+
"\n",
85+
"\n",
86+
"register_cache(\"MyCache\", MyCache)\n",
87+
"\n",
88+
"\n",
89+
"async def run():\n",
90+
" \"\"\"Demonstrate usage of the custom cache implementation.\"\"\"\n",
91+
" cache = create_cache(\n",
92+
" CacheConfig(\n",
93+
" type=\"MyCache\",\n",
94+
" some_setting=\"important setting value\", # type: ignore\n",
95+
" )\n",
96+
" )\n",
97+
"\n",
98+
" await cache.set(\"my_key\", {\"k1\": \"object to cache\"})\n",
99+
" print(\"Value stored in cache for 'my_key':\")\n",
100+
" print(await cache.get(\"my_key\"))\n",
101+
"\n",
102+
"\n",
103+
"if __name__ == \"__main__\":\n",
104+
" await run()"
105+
]
106+
}
107+
],
108+
"metadata": {
109+
"kernelspec": {
110+
"display_name": "Python 3",
111+
"language": "python",
112+
"name": "python3"
113+
},
114+
"language_info": {
115+
"codemirror_mode": {
116+
"name": "ipython",
117+
"version": 3
118+
},
119+
"file_extension": ".py",
120+
"mimetype": "text/x-python",
121+
"name": "python",
122+
"nbconvert_exporter": "python",
123+
"pygments_lexer": "ipython3",
124+
"version": "3.12.9"
125+
}
126+
},
127+
"nbformat": 4,
128+
"nbformat_minor": 5
129+
}

‎packages/graphrag-chunking/README.md‎

Lines changed: 4 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -8,34 +8,17 @@ This package contains a collection of text chunkers, a core config model, and a
88

99
The SentenceChunker class splits text into individual sentences by identifying sentence boundaries. It takes input text and returns a list where each element is a separate sentence, making it easy to process text at the sentence level.
1010

11-
```python
12-
chunker = SentenceChunker()
13-
chunks = chunker.chunk("This is a test. Another sentence.")
14-
print(chunks) # ["This is a test.", "Another sentence."]
15-
```
11+
[Open the notebook to explore the basic sentence example code](example_notebooks/basic_sentence_example.ipynb)
1612

1713
### Token chunking
1814

1915
The TokenChunker splits text into fixed-size chunks based on token count rather than sentence boundaries. It uses a tokenizer to encode text into tokens, then creates chunks of a specified size with configurable overlap between chunks.
2016

21-
```python
22-
tokenizer = tiktoken.get_encoding("o200k_base")
23-
chunker = TokenChunker(size=3, overlap=0, encode=tokenizer.encode, decode=tokenizer.decode)
24-
chunks = chunker.chunk("This is a random test fragment of some text")
25-
print(chunks) # ["This is a", " random test fragment", " of some text"]
26-
```
17+
[Open the notebook to explore the token chunking example code](example_notebooks/token_chunking_example.ipynb)
18+
2719

2820
### Using the factory via helper util
2921

3022
The create_chunker factory function provides a configuration-driven approach to instantiate chunkers by accepting a ChunkingConfig object that specifies the chunking strategy and parameters. This allows for more flexible and maintainable code by separating chunker configuration from direct instantiation.
3123

32-
```python
33-
tokenizer = tiktoken.get_encoding("o200k_base")
34-
config = ChunkingConfig(
35-
strategy="tokens",
36-
size=3,
37-
overlap=0
38-
)
39-
chunker = create_chunker(config, tokenizer.encode, tokenizer.decode)
40-
...
41-
```
24+
[Open the notebook to explore the factory helper util example code](example_notebooks/factory_helper_util_example.ipynb)

0 commit comments

Comments
 (0)