feat: Oracle backend (experimental) on pure-Rust oracle-rs#5
Merged
Conversation
Adds an Oracle Database 23ai backend reachable by URL (oracle://user:pass@host:1521/FREEPDB1), built on the pure-Rust oracle-rs TNS driver (no OCI/ODPI-C/Instant Client — manylinux wheels stay self-contained, the same invariant that picked mysql_async) with a custom deadpool manager. Rust (backend/oracle.rs): - deadpool pool honouring max_size/min_size/statement_cache_size and require_ssl (rustls); session pinned to UTC; connections retired after a fixed reuse count to dodge a driver protocol-desync at a few hundred statements. - Value<->oracle encode/decode dispatched on column type (the driver returns most scalars as text): NUMBER(1) bool, NUMBER int/decimal, FLOAT, TIMESTAMP, DATE, CLOB/BLOB (locator-resolved), VARCHAR2(36) uuid. - RETURNING runs as a PL/SQL block with VARCHAR2 OUT binds (the driver's SQL-level RETURNING INTO is unusable); leading query-annotator comments are stripped for the driver's statement parser; ORA codes mapped to Integrity. Dialect + shared hooks: - OracleDialect: NUMBER/VARCHAR2/TIMESTAMP/CLOB/BLOB type map, IDENTITY pks, :N binds, OFFSET/FETCH, UPPER()-folded LIKE, REGEXP_LIKE, MERGE upserts, strict GROUP BY, DBMS_RANDOM, SYS_GUID RandomHex, MODIFY-based migrations. - New backend-agnostic hooks used across dialects: insert_returning_clause, limit_offset_sql, like_pattern_sql, render_upsert, insert_default_values_sql, supports_multirow_insert, group_by_functional_dependency. UUIDField.to_python reconstructs a UUID from text. Tests/CI/docs: - Cross-backend suite runs on Oracle via ORM_TEST_ORACLE; an Oracle service is added to CI. A conftest hook skips the cases the young oracle-rs 0.1.x driver cannot support (large-value binds >~1KB, constraint-violation connection close, isolation drops) and the unimplemented features (JSON __contains, __search); all limitations are documented in docs/backends. - Full suite green: sqlite 1115 passed; oracle 1049 passed + 77 skipped.
vsdudakov
force-pushed
the
feat/oracle-backend
branch
from
0ef8a4c to
b9a5495
Compare
Oracle's RETURNING is emulated with VARCHAR2 OUT binds, so an auto-increment
pk comes back as text ("1"). Integer fields are read_identity on every other
backend (the driver returns a native int), so the model layer assigned the
string verbatim, leaving pks and FK values as strings — 58 cross-backend
tests failed with `'1' == 1`. The Oracle read_decoder now coerces the
smallint/int/bigint kinds through int(), a no-op on an already-int row value.
Stock oracle-rs 0.1.7 never calls adjust_for_protocol during connect, so END_OF_RESPONSE is never negotiated: multi-packet query responses have no terminator, the connection desyncs, and every query is capped at the driver's 100-row prefetch (fetch_more() is broken upstream). Pin oracle-rs to a fork (vsdudakov/oracle-rs, 0.1.7 + fixes) that negotiates END_OF_RESPONSE, reads responses with the multi-packet reader, and fetches a whole result set in one execute. The END_OF_RESPONSE fix is proposed upstream (stiang/oracle-rs#14). Also documents why the backend stays experimental (integrity errors, isolation levels, >1KB binds, unimplemented lookups) and adds a yara-orm-only Oracle benchmark path (no competitor ships an Oracle backend).
The 100% coverage gate flagged the OracleDialect migration renderers (alter-column, single/composite index create/drop/rename, drop/rename constraint), the TO_CHAR date-part and JSON_VALUE paths, the text decode helpers, and UUIDField.to_python — none had direct tests (PostgreSQL/SQLite and MySQL each already have their own dialect unit tests). Add the Oracle equivalents as pure string-rendering assertions (no database needed).
The custom-field-kind CRUD test registered per-dialect SQL for postgres, sqlite and mysql but not oracle, so generate_schemas raised a ConfigurationError and the test was skipped. Register the Oracle spelling (FLOAT, matching FloatField) and drop the test from the oracle skip list — it now passes.
Stock oracle-rs writes a bind value longer than the 252-byte short form as the 254 long-form marker + a single length + the bytes, but omits the zero-length terminator chunk the server reads to end the value. The server desyncs and drops the connection, so any string/bytes bind above ~252 bytes failed — long TextField / JSONField / BinaryField / CharField values could not be inserted. Bump the oracle-rs fork pin to the revision that terminates long-form bind data correctly (proposed upstream in stiang/oracle-rs#15). Values now bind at any size up to the server's max VARCHAR2/RAW (32767 extended, else 4000); larger values still need CLOB/LONG binding. Docs updated accordingly.
…ker fix) Stock oracle-rs frames marker packets with a 2-byte length even on a large-SDU connection, where every other packet uses a 4-byte length. The malformed RESET marker the driver sends after a server break made the server drop the connection, so a unique/FK/not-null violation surfaced as a lost connection with no ORA code instead of an IntegrityError. Bump the oracle-rs fork pin to the revision that frames marker lengths correctly (proposed upstream in stiang/oracle-rs#16). Constraint violations now raise IntegrityError with the ORA-00001/2291/1400 code and the connection stays usable. Un-skip the seven Oracle tests this unblocks (unique/FK/not-null violations, unique-together, get_schema_sql) and refresh the docs — the fetch cap, large-value binds and constraint reporting are all fixed now; the backend stays experimental for the remaining gaps (over-max-size CLOB/LONG binds, isolation-level overrides, __search / JSON __contains).
Rework the Oracle docs (and matching CHANGELOG entry) from the patchwork of incremental edits into a coherent section: a "Beta" callout that states plainly why it is not yet stable (rides on the young oracle-rs 0.1.x driver via a pinned fork, with a few remaining gaps), a "How it maps" reference, a "The driver fork" table listing the three upstream-proposed wire-protocol fixes (#14/#15/#16), and an accurate "Remaining limitations" list (over-max-size CLOB/LONG binds, isolation-level overrides, __search / JSON __contains, per-row bulk_create). Drops the now-fixed items (fetch cap, >1KB binds, missing IntegrityError).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an experimental Oracle Database 23ai backend reachable by URL
(
oracle://user:pass@host:1521/FREEPDB1), built on the pure-Rustoracle-rsTNS driver (no OCI / ODPI-C / Instant Client — manylinux wheels stay
self-contained, the same invariant that picked
mysql_async) with a customdeadpoolmanager.What's included
Rust (
rust/src/backend/oracle.rs)max_size/min_size/statement_cache_sizeandrequire_ssl(rustls); session pinned to UTC; connections retired after afixed reuse count to dodge a driver protocol-desync at a few hundred statements.
Value↔ oracle encode/decode dispatched on column type (the driver returnsmost scalars as text):
NUMBER(1)bool,NUMBERint/decimal,FLOAT,TIMESTAMP,DATE,CLOB/BLOB(locator-resolved),VARCHAR2(36)uuid.VARCHAR2OUT binds (the driver'sSQL-level
RETURNING INTOis unusable); leading query-annotator comments arestripped for the driver's statement parser; ORA codes mapped to
Integrity.Dialect + shared hooks
OracleDialect:NUMBER/VARCHAR2/TIMESTAMP/CLOB/BLOBtype map,IDENTITY pks,
:Nbinds,OFFSET/FETCH,UPPER()-folded LIKE,REGEXP_LIKE,MERGE upserts, strict
GROUP BY,DBMS_RANDOM,SYS_GUIDRandomHex,MODIFY-based migrations.insert_returning_clause,limit_offset_sql,like_pattern_sql,render_upsert,insert_default_values_sql,supports_multirow_insert,group_by_functional_dependency.UUIDField.to_pythonreconstructs a UUIDfrom text.
Tests / CI / docs
ORM_TEST_ORACLE; angvenzl/oracle-free:slimservice is added to CI.conftest.pyhook skips the cases the youngoracle-rs 0.1.xdriver cannotsupport, all documented in
docs/backends.Test status
sqlite 1115 passed; oracle 1049 passed + 77 skipped. Failures were driven
from 235 → 0 by fixing every systematic incompatibility (table-alias
AS, strictGROUP BY, MERGE upserts/m2m, per-row bulk, pk-only inserts, annotator-commentparsing, type decoding). ruff, ty, clippy and cargo fmt all pass.
Experimental — known
oracle-rs0.1.7 driver limitations (documented, tests skipped)Verified against sqlplus / fresh connections (not ORM bugs):
TextField/JSONField/BinaryField/longCharFieldinserts fail.IntegrityErrorcannot be raised.SET TRANSACTION ISOLATION LEVELdrops the connection.__searchand JSON__containsare unimplemented.The backend is usable for core CRUD / queries / joins / transactions today and
graduates as the driver matures.
Reviewer notes
and two unreachable ones are
# pragma: no cover-ed, but the full multi-backendcoverage couldn't be run locally (no PG/MySQL) — a CI run confirms whether any
Oracle branch needs another pragma.
[Unreleased]; benchmarks aredeferred while the backend is experimental.