Commit 574c6de
feat: Add Snowflake Semantic Views support (#200)
* feat: Add Snowflake Semantic Views support
Add automatic detection and context injection for Snowflake Semantic Views
when a Snowflake connection is provided. This helps LLMs generate correct
queries using the SEMANTIC_VIEW() function with certified business metrics.
Changes:
- Add SnowflakeSource class (Python and R) that extends SQLAlchemySource/DBISource
- Discover semantic views via SHOW SEMANTIC VIEWS at initialization
- Retrieve DDL definitions via GET_DDL('SEMANTIC_VIEW', ...)
- Include semantic view context in schema output
- Add SEMANTIC_VIEW() syntax reference to system prompt
- Add Snowflake-specific SQL tips (QUALIFY, LATERAL FLATTEN, time travel)
- Graceful fallback when no semantic views exist or discovery fails
Based on the Snowflake skill from posit-dev/databot#278.
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* `air format` (GitHub Actions)
* fix: Address Snowflake Semantic Views code review concerns
- Fix SQL injection risk by escaping single quotes in view names
- Add discover_semantic_views parameter for lazy initialization
- Remove error swallowing, let errors propagate for debugging
- Add debug logging when no semantic views are found
- Move logger placement to top of file (Python)
- Add defensive dialect check in normalize_data_source (Python)
- Add comprehensive unit tests for both Python and R
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* `air format` (GitHub Actions)
* refactor: Extract Snowflake semantic views into adapter pattern
Introduces a Protocol-based adapter pattern for Snowflake semantic view
discovery that works with both SQLAlchemy and Ibis backends:
- Add _snowflake.py with RawSQLExecutor Protocol, executor implementations
(SQLAlchemyExecutor, IbisExecutor), and standalone discovery functions
- Add _snowflake_sources.py with SnowflakeSource and new SnowflakeIbisSource
- Update normalize_data_source() to route Ibis Snowflake backends
- Maintain backwards-compatible imports from _datasource.py
This enables semantic view support for Ibis connections to Snowflake,
not just SQLAlchemy connections.
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* chore: Remove backwards-compat re-exports from _datasource.py
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* refactor: Move Snowflake semantic view discovery into base classes
Instead of separate SnowflakeSource and SnowflakeIbisSource classes,
SQLAlchemySource and IbisSource now auto-detect Snowflake backends
and discover semantic views during initialization.
Changes:
- SQLAlchemySource/IbisSource check dialect/backend name for "snowflake"
- Discovery can be disabled via QUERYCHAT_DISABLE_SEMANTIC_VIEWS env var
- Removed _snowflake_sources.py (no longer needed)
- Simplified normalize_data_source() - no Snowflake-specific routing
- Updated tests to verify new architecture
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* refactor: Move SEMANTIC_VIEW_SYNTAX to shared prompt files
Extract inline syntax documentation to `prompts/semantic-view-syntax.md`
in both R and Python packages, making it language-agnostic and easier
to maintain.
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* `air format` (GitHub Actions)
* Update pkg-py/src/querychat/_querychat_base.py
* refactor: Extract Snowflake semantic views into adapter pattern
- Replace Protocol/class-based OOP with functional approach using
backend_type: Literal["sqlalchemy", "ibis"] discriminator
- Move env var check (QUERYCHAT_DISABLE_SEMANTIC_VIEWS) into
discover_semantic_views() for early exit
- Move semantic view discovery from __init__ to get_schema() for
lazy initialization
- Remove SemanticViewMixin in favor of direct function calls
- Update tests to verify new lazy discovery behavior
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* chore: Remove is_snowflake template variable
Snowflake SQL Tips section isn't necessary for semantic views support.
Removed the {{#is_snowflake}} block from prompt templates and the
is_snowflake variable from system prompt code in both R and Python.
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* refactor: Use isinstance() checks instead of type parameter
Replace backend_type: Literal["sqlalchemy", "ibis"] parameter with
isinstance(backend, sqlalchemy.Engine) checks for cleaner API.
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* chore: Remove _semantic_views class attribute declarations
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* refactor: Merge SnowflakeSource semantic view logic into DBISource
Move semantic view discovery from separate SnowflakeSource class into
DBISource to match Python implementation:
- Add semantic_views field and has_semantic_views() method to DBISource
- Move is_snowflake_connection(), discover_semantic_views_impl(),
get_semantic_view_ddl(), format_semantic_views_section() to DBISource.R
- Add QUERYCHAT_DISABLE_SEMANTIC_VIEWS env var support for R
- Delete SnowflakeSource.R
- Update normalize_data_source() to use DBISource for all DBI connections
- Update QueryChatSystemPrompt to check DBISource for semantic views
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* (local)
* chore: Minimize docstrings and remove redundant comments
- Use single-line docstrings for internal functions
- Remove redundant inline comments
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* `air format` (GitHub Actions)
* `devtools::document()` (GitHub Actions)
* refactor: Remove _semantic_views attribute and has_semantic_views property
- Simplify get_schema() to discover semantic views inline without storing
- Detect semantic views in system prompt by checking schema string
- Remove unnecessary state management
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* refactor: Consolidate semantic view information into single prompt section
Move semantic view DDLs out of <database_schema> into a dedicated
<semantic_views> section, placing all semantic view info in one cohesive
location in the prompt template.
Structure is now:
- General explanation (why semantic views matter)
- Query syntax documentation
- <semantic_views> tag with table-specific DDL definitions
Changes:
- Remove semantic view info from get_schema() output
- Add get_semantic_view_ddls() method to return just DDL content
- Add semantic_view_ddls template variable to prompt context
- Update prompt.md to include <semantic_views> tag after syntax docs
- Rename format_semantic_views_section to format_semantic_view_ddls
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* `air format` (GitHub Actions)
* `devtools::document()` (GitHub Actions)
* refactor: Add semantic view methods to DataSource base class
- Add has_semantic_views() and get_semantic_view_ddls() to DataSource
with default implementations returning False and empty string
- Remove hasattr check in _system_prompt.py since methods now exist
on base class
- Revert superfluous formatting changes from ruff
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* refactor: Move semantic view discovery into get_semantic_view_ddls
Extract discovery logic into _ensure_semantic_views_discovered() method
called from has_semantic_views() and get_semantic_view_ddls(). This:
- Removes discovery from get_schema() as requested
- Eliminates the type: ignore by ensuring _semantic_views is always
a list after discovery
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* refactor: Remove has_semantic_views(), improve prompt text
- Remove has_semantic_views() method from DataSource classes
- Use truthy check on get_semantic_view_ddls() instead
- Update prompt text with improved explanation and real-world example
that better illustrates why semantic views matter
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* `devtools::document()` (GitHub Actions)
* refactor: Simplify semantic views, adjust prompt structure
- Change "### Semantic Views" to "## Semantic Views" (top-level section)
- Move "use SEMANTIC_VIEW() instead of raw SQL" into prompt.md
- Adjust header levels in semantic-view-syntax.md accordingly
- Remove _semantic_views attribute from datasource classes - compute
directly in get_semantic_view_ddls() without caching
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* Apply suggestions from code review
* refactor: Restructure semantic views into dedicated directory
- Move semantic view prompts to prompts/semantic-views/ directory
- prompt.md: Contains IMPORTANT notice and real-world example
- syntax.md: Contains SEMANTIC_VIEW() query syntax reference
- Rename get_semantic_view_ddls() to get_semantic_views_section()
which now returns the complete assembled section
- Simplify main prompt.md to use single {{{semantic_views}}} placeholder
- Remove type ignore in _snowflake.py by using itertuples instead of
to_dict(orient="records")
- Update IMPORTANT paragraph to include SEMANTIC_VIEW() instruction
inline per PR feedback
- Update tests to reflect new method names and structure
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* `air format` (GitHub Actions)
* `devtools::document()` (GitHub Actions)
* style: Revert formatting changes in DBISource.R
Minimize diff by reverting air format changes that were not
part of the semantic views feature.
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* fix: Use raw_sql() for Ibis backends to support SHOW commands
The `backend.sql()` method in Ibis parses queries with sqlglot, which
doesn't support Snowflake commands like `SHOW SEMANTIC VIEWS`. Switch
to using `backend.raw_sql()` which executes queries without parsing.
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* fix: Address PR feedback and fix pyright error
- Remove comment about raw_sql() per PR feedback
- Add type ignore for raw_sql() to fix pyright error (method exists
on concrete backends but not typed on SQLBackend base class)
- Remove unused PROMPTS_DIR constant per PR feedback
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* Add brand.yml website dependency
* refactor: Rename semantic views methods for clarity
- Rename get_semantic_views_section() to get_semantic_views_description()
on DataSource classes (clearer intent)
- Rename get_semantic_views_section() to format_semantic_views() in
_snowflake.py / DBISource.R (matches other format_* functions)
- Update tests to use new method names
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* `devtools::document()` (GitHub Actions)
* style: Revert formatting changes and _shiny.py modification
Minimize diff by reverting auto-formatter changes that were not
part of the semantic views feature.
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* docs: Add Snowflake Semantic Views to CHANGELOG and NEWS
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* refactor: Update docstrings and revert formatting changes
- Update get_semantic_views_description() docstrings to clarify purpose
- Revert list comprehension formatting to match original style
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* `devtools::document()` (GitHub Actions)
* style: Revert list comprehension formatting to original style
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* style: Revert unnecessary R formatting changes
Minimize diff by reverting air format changes that were not
part of the semantic views feature.
Co-Authored-By: Claude Opus 4.5 <[email protected]>
* chore: Remove accidentally committed files
---------
Co-authored-by: Claude Opus 4.5 <[email protected]>
Co-authored-by: cpsievert <[email protected]>1 parent f3a4360 commit 574c6de
File tree
19 files changed
+1139
-9
lines changed- .github/workflows
- pkg-py
- src/querychat
- prompts
- semantic-views
- tests
- pkg-r
- R
- inst/prompts
- semantic-views
- man
- tests/testthat
19 files changed
+1139
-9
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
51 | | - | |
| 51 | + | |
52 | 52 | | |
53 | 53 | | |
54 | 54 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
8 | 14 | | |
9 | 15 | | |
10 | 16 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
14 | 18 | | |
15 | 19 | | |
16 | 20 | | |
| |||
179 | 183 | | |
180 | 184 | | |
181 | 185 | | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
182 | 190 | | |
183 | 191 | | |
184 | 192 | | |
| |||
489 | 497 | | |
490 | 498 | | |
491 | 499 | | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
492 | 507 | | |
493 | 508 | | |
494 | 509 | | |
| |||
895 | 910 | | |
896 | 911 | | |
897 | 912 | | |
898 | | - | |
899 | | - | |
| 913 | + | |
900 | 914 | | |
901 | 915 | | |
902 | 916 | | |
| |||
960 | 974 | | |
961 | 975 | | |
962 | 976 | | |
| 977 | + | |
| 978 | + | |
| 979 | + | |
| 980 | + | |
| 981 | + | |
| 982 | + | |
| 983 | + | |
963 | 984 | | |
964 | 985 | | |
965 | 986 | | |
| |||
1018 | 1039 | | |
1019 | 1040 | | |
1020 | 1041 | | |
1021 | | - | |
1022 | | - | |
| 1042 | + | |
1023 | 1043 | | |
1024 | 1044 | | |
1025 | 1045 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
65 | 65 | | |
66 | 66 | | |
67 | 67 | | |
68 | | - | |
| 68 | + | |
| 69 | + | |
69 | 70 | | |
70 | 71 | | |
71 | | - | |
| 72 | + | |
72 | 73 | | |
| 74 | + | |
73 | 75 | | |
74 | 76 | | |
75 | 77 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
71 | 71 | | |
72 | 72 | | |
73 | 73 | | |
| 74 | + | |
74 | 75 | | |
75 | 76 | | |
76 | 77 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
0 commit comments