Skip to content

Commit fc4cebf

Browse files
junhewkclaude
andcommitted
Add v3 data model with EntityType and relationship fields
- Add EntityType union type with 10 fixed entity types - Add entityType field to OntologyNode (replacing flexible labels) - Add relationship field to OntologyEdge (free-form verbs) - Add helper functions: getEntityTypeColor, getNodeEntityType, getEdgeRelationship - Add labelToEntityType for legacy v2 data migration - Update prompts for v3 extraction format - Update search to use new type helpers Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 52bf08f commit fc4cebf

6 files changed

Lines changed: 367 additions & 175 deletions

File tree

README.md

Lines changed: 42 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -6,17 +6,17 @@ This plugin builds a lightweight knowledge graph from users' Obsidian notes usin
66

77
## Why Lightweight Ontology?
88

9-
Traditional knowledge graphs often require complex schemas with dozens of relationship types, making them difficult to maintain and query. Simple Graph Builder takes a different approach:
9+
Traditional knowledge graphs often require complex schemas with dozens of entity and relationship types, making them difficult to maintain and query. Simple Graph Builder takes a different approach:
1010

11-
- **Flexible Node Labels**: The LLM determines the most appropriate label for each entity (Person, Concept, Tool, Project, etc.) - no predefined restrictions
12-
- **Fixed Relationship Types**: Only 5 universal relationship types that cover most knowledge connections
11+
- **10 Fixed Entity Types**: PERSON, ORGANIZATION, CONCEPT, PROJECT, TOOL, EVENT, PLACE, DOCUMENT, METHOD, TOPIC - covering all common knowledge domains
12+
- **Free-form Relationship Verbs**: Express relationships naturally with active verbs like "develops", "uses", "causes", "cites"
1313
- **Detail Property**: Each relationship includes a `detail` field for nuanced descriptions without schema explosion
1414

15-
This design provides **80% of the expressiveness with 20% of the complexity**, making it easy to build, query, and maintain your personal knowledge graph.
15+
This design provides **structured entity classification with expressive relationships**, making it easy to build, query, and maintain your personal knowledge graph.
1616

1717
## Features
1818

19-
- **Lightweight Ontology Model**: Simple but expressive - flexible node labels + 5 fixed relationship types with detail annotations
19+
- **Lightweight Ontology Model**: Simple but expressive - 10 fixed entity types + free-form relationship verbs with detail annotations
2020
- **Hybrid Entity Resolution**: Multi-stage deduplication pipeline combining fast lookups with embedding similarity and LLM verification (inspired by KGGen [3])
2121
- **Smart Search**: AI-powered natural language queries over your knowledge graph with multi-path exploration
2222
- **Entity Extraction**: Automatically extract entities from your notes using AI (configurable extraction depth)
@@ -62,25 +62,35 @@ This approach resolves most entities via fast hash lookups, reserving expensive
6262

6363
## Data Model
6464

65-
### Node Labels (Flexible)
66-
The LLM determines appropriate labels for each entity:
67-
- **Person, Organization, Team** - People and groups
68-
- **Concept, Theory, Method, Technique** - Ideas and approaches
69-
- **Project, Product, System** - Work items
70-
- **Tool, Library, Framework, Software** - Technical tools
71-
- **Event, Meeting, Conference** - Occurrences
72-
- **Document, Paper, Book** - Written works
73-
- **Place, Location** - Geography
74-
- Any other appropriate label
75-
76-
### Relationship Types (Fixed)
77-
| Type | Meaning | Example Details |
78-
|------|---------|-----------------|
79-
| `HAS_PART` | Parent/Child, Inclusion | "member of", "contains", "subtopic" |
80-
| `LEADS_TO` | Causality, Sequence, Dependency | "causes", "blocks", "enables" |
81-
| `ACTED_ON` | Creation, Modification, Usage | "created", "maintains", "uses" |
82-
| `CITES` | Reference, Source, Evidence | "references", "based on", "quotes" |
83-
| `RELATED_TO` | Loose association, Similarity | "similar to", "see also", "wikilink" |
65+
### Entity Types (10 Fixed Types)
66+
The LLM must classify each entity into one of these types:
67+
68+
| Type | Description | Examples |
69+
|------|-------------|----------|
70+
| `PERSON` | People, individuals | Authors, researchers, team members |
71+
| `ORGANIZATION` | Companies, institutions | Google, MIT, research labs |
72+
| `CONCEPT` | Ideas, theories, principles | Machine learning, API design |
73+
| `PROJECT` | Projects, products, initiatives | Obsidian, GraphRAG |
74+
| `TOOL` | Software, hardware, instruments | Python, VS Code, Docker |
75+
| `EVENT` | Meetings, conferences, milestones | NeurIPS 2024, sprint review |
76+
| `PLACE` | Locations, venues, geography | San Francisco, AWS us-east-1 |
77+
| `DOCUMENT` | Papers, books, articles, notes | "Attention Is All You Need" |
78+
| `METHOD` | Techniques, approaches, workflows | Agile, TDD, fine-tuning |
79+
| `TOPIC` | Subjects, themes, fields, domains | NLP, distributed systems |
80+
81+
### Relationships (Free-form Verbs)
82+
Relationships are expressed as active verbs describing how entities relate:
83+
84+
| Verb Examples | Meaning |
85+
|--------------|---------|
86+
| `develops`, `creates`, `builds` | Creation, authorship |
87+
| `uses`, `applies`, `implements` | Usage, application |
88+
| `causes`, `leads to`, `enables` | Causality, dependency |
89+
| `contains`, `includes`, `has` | Composition, membership |
90+
| `cites`, `references`, `based on` | Citation, source |
91+
| `relates to`, `similar to` | General association |
92+
93+
Each relationship also includes an optional `detail` field for additional context.
8494

8595
## UI Elements
8696

@@ -94,9 +104,9 @@ Shows real-time graph statistics with node counts by label.
94104

95105
### Note Neighborhood Panel
96106
A sidebar panel showing:
97-
- **Extracted Nodes**: Entities from the current note with label badges
98-
- **Connected Nodes**: Grouped by label (Person, Concept, Tool, etc.)
99-
- **Relationships**: Shows relationship type and detail for each connection
107+
- **Extracted Nodes**: Entities from the current note with entity type badges
108+
- **Connected Nodes**: Grouped by entity type (PERSON, CONCEPT, TOOL, etc.)
109+
- **Relationships**: Shows relationship verb and detail for each connection
100110
- Click nodes to see source notes and relationship details
101111

102112
### Graph View Context Menu
@@ -112,9 +122,9 @@ Right-click a node to:
112122

113123
### Analysis Settings
114124
- **Extraction Mode**: Control extraction depth
115-
- *Simple*: Max 15 entities, 20 relationships (fast, low cost)
116-
- *Advanced*: Max 30 entities, 50 relationships (balanced)
117-
- *Maximum*: No limits (thorough extraction)
125+
- *Standard*: Max 15 entities per chunk (fast, low cost)
126+
- *Thorough*: No limits per chunk (comprehensive extraction)
127+
- **Chunked Processing**: Long notes are automatically split into ~500 token chunks and processed in parallel (max 3 concurrent)
118128
- **Auto-analyze on save**: Automatically analyze notes when you save them (2-second debounce)
119129
- **Analyze entire vault**: Batch analyze all notes with progress tracking and cancellation support
120130

@@ -137,7 +147,7 @@ Enable embedding-based entity resolution for intelligent deduplication:
137147
- **Open graph in main window**: Toggle to open the graph visualization in a main tab instead of the right sidebar
138148

139149
### Data Management
140-
- View graph statistics (nodes by label, relationships by type)
150+
- View graph statistics (nodes by entity type, total relationships)
141151
- Clear all graph data
142152

143153
## Installation
@@ -175,7 +185,7 @@ Enable embedding-based entity resolution for intelligent deduplication:
175185
- **Scroll** to zoom in/out
176186
- **Drag** to pan around the graph
177187

178-
Node colors are determined by label (predefined colors for common labels, hash-based colors for others). Edge styles vary by relationship type.
188+
Node colors are determined by entity type (10 predefined colors). Edges use unified gray styling with relationship verbs shown on hover.
179189

180190
### Search
181191
Two search modes are available:

src/commands/analyze.ts

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@ import { Notice, MarkdownView, TFile } from 'obsidian';
22
import SimpleGraphBuilderPlugin from '../main';
33
import { loadHashes, saveHashes, computeHash, hasNoteChanged, updateNoteHash, removeNoteHash, clearHashes } from '../graph/hashes';
44
import { mergeExtractionIntoCache, mergeExtractionIntoCacheWithResolution, mergeInternalLinksIntoCache, removeNoteFromCache } from '../graph/merge';
5-
import { buildExtractionPrompt, truncateContent } from '../extraction/prompts';
6-
import { extractOntology, settingsToExtractionOptions, ExtractionError } from '../extraction/llm-client';
5+
import { truncateContent } from '../extraction/prompts';
6+
import { extractOntologyChunked, settingsToExtractionOptions, ExtractionError } from '../extraction/llm-client';
77

88
// Vault analysis state (encapsulated to avoid module-level mutable variables)
99
const vaultAnalysisState = {
@@ -54,11 +54,11 @@ export async function analyzeCurrentNote(plugin: SimpleGraphBuilderPlugin): Prom
5454
// Get existing node names for context (O(1) via cache)
5555
const existingNodeNames = plugin.graphCache.getExistingNodeNames();
5656

57-
// Build prompt and call LLM
57+
// Use chunked extraction for better handling of long notes
5858
const truncatedContent = truncateContent(content);
59-
const prompt = buildExtractionPrompt(truncatedContent, existingNodeNames, plugin.settings.extractionMode || 'simple');
6059
const options = settingsToExtractionOptions(plugin.settings);
61-
const result = await extractOntology(options, prompt);
60+
const mode = plugin.settings.extractionMode || 'standard';
61+
const { result, chunkCount } = await extractOntologyChunked(options, truncatedContent, existingNodeNames, mode);
6262

6363
// Hide loading notice
6464
loadingNotice.hide();
@@ -128,11 +128,12 @@ export async function analyzeCurrentNote(plugin: SimpleGraphBuilderPlugin): Prom
128128
// Also show total extracted (even if merged with existing)
129129
const totalNodes = result.nodes.length;
130130
const totalRels = result.relationships.length;
131+
const chunkInfo = chunkCount > 1 ? ` in ${chunkCount} chunks` : '';
131132

132133
if (parts.length > 0) {
133-
new Notice(`Added: ${parts.join(', ')}\n(Extracted: ${totalNodes} nodes, ${totalRels} relationships)${resolutionInfo}`);
134+
new Notice(`Added: ${parts.join(', ')}\n(Extracted: ${totalNodes} nodes, ${totalRels} relationships${chunkInfo})${resolutionInfo}`);
134135
} else if (totalNodes > 0 || totalRels > 0) {
135-
new Notice(`Extracted ${totalNodes} nodes, ${totalRels} relationships (all merged with existing)${resolutionInfo}`);
136+
new Notice(`Extracted ${totalNodes} nodes, ${totalRels} relationships${chunkInfo} (all merged with existing)${resolutionInfo}`);
136137
} else {
137138
new Notice('No entities or relationships found in this note');
138139
}
@@ -226,11 +227,11 @@ export async function analyzeFile(
226227
// Get existing node names for context
227228
const existingNodeNames = plugin.graphCache.getExistingNodeNames();
228229

229-
// Build prompt and call LLM
230+
// Use chunked extraction
230231
const truncatedContent = truncateContent(content);
231-
const prompt = buildExtractionPrompt(truncatedContent, existingNodeNames, plugin.settings.extractionMode || 'simple');
232232
const extractionOptions = settingsToExtractionOptions(plugin.settings);
233-
const result = await extractOntology(extractionOptions, prompt);
233+
const mode = plugin.settings.extractionMode || 'standard';
234+
const { result } = await extractOntologyChunked(extractionOptions, truncatedContent, existingNodeNames, mode);
234235

235236
// Merge results into graph cache with resolution if embeddings enabled
236237
let nodesAdded: number;

0 commit comments

Comments
 (0)