Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 18 additions & 12 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,35 +4,41 @@

**JSONPath syntax changes**

These breaking changes apply to Python JSONPath in its default configuration. We've also introduced a _strict mode_ where we follow the RFC 9535 specification exactly. See [optional dependencies](https://jg-rp.github.io/python-jsonpath/#optional-dependencies) and the [syntax guide](https://jg-rp.github.io/python-jsonpath/syntax/) for more information.

- Using bracket notation, unquoted property names are no longer interpreted as quoted property names. These paths used to be equivalent, `$[foo]`, `$['foo']` and `$["foo"]`. Now, names without quotes start a _singular query selector_. With an implicit _root identifier_, `$.a[b]` is equivalent to `$.a[$.b]`. See [Singular query selector](https://jg-rp.github.io/python-jsonpath/syntax/#singular-query-selector) in the syntax guide.
- In filter selector expressions, float literals now follow the specification. Previously `.1` and `1.` where allowed, now it must be `0.1` and `1.0`, with at least one digit either side of the decimal point.
- Slice selector indexes and step now follow the specification. Previously leading zeros and negative zero were allowed, now they raise a `JSONPathSyntaxError`.
- Whitespace is no longer allowed between a dot (`.` or `..`) and a name when using shorthand notation for the name selector. Whitespace before the dot oor double dot is OK.
These breaking changes affect the **default configuration** of Python JSONPath.
Version 2 also introduces a new _strict mode_, which enforces full compliance with [RFC 9535](https://datatracker.ietf.org/doc/html/rfc9535). See [optional dependencies](https://jg-rp.github.io/python-jsonpath/#optional-dependencies) and the [syntax guide](https://jg-rp.github.io/python-jsonpath/syntax/) for details.

- **Bracket notation** - unquoted property names are no longer treated as quoted names.
- Before: `$[foo]`, `$['foo']`, and `$["foo"]` were equivalent.
- Now: `$[foo]` is a _singular query selector_. With an implicit root identifier, `$.a[b]` is equivalent to `$.a[$.b]`. See [Singular query selector](https://jg-rp.github.io/python-jsonpath/syntax/#singular-query-selector).
- **Filter expressions** - float literals must follow the RFC.
- `.1` is now invalid (use `0.1`)
- `1.` is now invalid (use `1.0`)
- **Slice selectors** - indexes and steps must follow the RFC.
- Leading zeros and negative zero are no longer valid and raise `JSONPathSyntaxError`.
- **Dot notation** - no whitespace is allowed between `.` or `..` and the following name. Whitespace before the dot is still permitted.

**JSONPath function extension changes**

- Added the `startswith(value, prefix)` function extension. `startswith` returns `True` if both arguments are strings and the second argument is a prefix of the first argument. See the [filter functions](https://jg-rp.github.io/python-jsonpath/functions/#startswith) documentation.
- The non-standard `keys()` function extension has been reimplemented. It used to be a simple Python function, `jsonpath.function_extensions.keys`. Now it is a "well-typed" class, `jsonpath.function_extensions.Keys`. See the [filter functions](https://jg-rp.github.io/python-jsonpath/functions/#keys) documentation.
- Added the `startswith(value, prefix)` function extension. Returns `True` if both arguments are strings and `prefix` is a prefix of `value`. See the [filter functions](https://jg-rp.github.io/python-jsonpath/functions/#startswith) documentation.
- Reimplemented the non-standard `keys()` function extension. It used to be a simple Python function, `jsonpath.function_extensions.keys`. Now it is a "well-typed" class, `jsonpath.function_extensions.Keys`. See the [filter functions](https://jg-rp.github.io/python-jsonpath/functions/#keys) documentation.
- Added `cache_capacity`, `debug` and `thread_safe` arguments to `jsonpath.function_extensions.Match` and `jsonpath.function_extensions.Search` constructors.

**JSONPath features**

- Added the [Keys filter selector](https://jg-rp.github.io/python-jsonpath/syntax/#keys-filter-selector).
- Added the [Singular query selector](https://jg-rp.github.io/python-jsonpath/syntax/#singular-query-selector).
- We now use the [regex] package, if available, instead of `re` for match and search function extensions. See [optional dependencies](https://jg-rp.github.io/python-jsonpath/#optional-dependencies).
- Added the `strict` argument to all [convenience functions](https://jg-rp.github.io/python-jsonpath/convenience/), the CLI and the `JSONPathEnvironment` constructor. When `strict=True`, all extensions to RFC 9535, any non-standard function extensions and any lax parsing rules will be disabled.
- Match and search function extensions now use the [`regex`](https://pypi.org/project/regex/) package (if installed) instead of `re`. See [optional dependencies](https://jg-rp.github.io/python-jsonpath/#optional-dependencies).
- Added the `strict` argument to all [convenience functions](https://jg-rp.github.io/python-jsonpath/convenience/), the CLI and the `JSONPathEnvironment` constructor. When `strict=True`, all non-standard extensions and relaxed parsing rules are disabled.
- Added class variable `JSONPathEnvironment.max_recursion_depth` to control the maximum recursion depth of descendant segments.
- Added pretty exception messages.
- Improved exception messages (prettier, more informative).

**Python API changes**

- Renamed class variable `JSONPathEnvironment.fake_root_token` to `JSONPathEnvironment.pseudo_root_token`.

**Low level API changes**

These breaking changes will only affect you if you're customizing the JSONPath lexer or parser.
These only affect projects customizing the JSONPath lexer or parser.

- The tokens produced by the JSONPath lexer have changed. Previously we broadly skipped some punctuation and whitespace. Now the parser can make better choices about when to accept whitespace and do a better job of enforcing dots.
- We've change the internal representation of compiled JSONPath queries. We now model segments and selectors explicitly and use terminology that matches RFC 9535.
Expand Down
15 changes: 12 additions & 3 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,20 @@ conda install -c conda-forge python-jsonpath

### Optional dependencies

By default, and without any additional dependencies, the syntax supported by Python JSONPath is **very close** to RFC 9535. For strict compatibility with the specification, install [regex](https://pypi.org/project/regex/) and [iregexp-check](https://pypi.org/project/iregexp-check/) packages too.
Python JSONPath works out of the box with **no extra dependencies**, and its syntax is already **very close** to [RFC 9535](https://www.rfc-editor.org/rfc/rfc9535).

With these two packages installed, the [`match()`](functions.md#match) and [`search()`](functions.md#search) filter functions will use [regex](https://pypi.org/project/regex/) instead of `re` from the standard library, and will validate regular expression patterns against [RFC 9485](https://datatracker.ietf.org/doc/html/rfc9485).
For strict compliance with the specification, [strict mode](syntax.md) and the `strict` extra were added in **version 2.0.0**.

See the [syntax guide](syntax.md) for more information about strict compatibility with RFC 9535 and extensions to the specification.
```console
pip install python-jsonpath[strict]
```

This installs [`regex`](https://pypi.org/project/regex/) and [`iregexp-check`](https://pypi.org/project/iregexp-check/), enabling:

- [`match()`](functions.md#match) and [`search()`](functions.md#search) to use `regex` instead of Python's built-in `re` module.
- Validation of regular expressions against [RFC 9485](https://datatracker.ietf.org/doc/html/rfc9485).

See the [syntax guide](syntax.md) for strict mode details and specification extensions.

## Example

Expand Down
31 changes: 29 additions & 2 deletions docs/syntax.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,35 @@
# JSONPath Syntax

Python JSONPath extends the [RFC 9535](https://datatracker.ietf.org/doc/html/rfc9535) specification with additional features and relaxed rules. If you need strict compliance with RFC 9535, set `strict=True` when calling [`findall()`](convenience.md#jsonpath.findall), [`finditer()`](convenience.md#jsonpath.finditer), etc., which enforces the standard without these extensions.
Python JSONPath extends the [RFC 9535](https://datatracker.ietf.org/doc/html/rfc9535) specification with extra selectors and relaxed rules for convenience. If you need strict compliance with RFC 9535, pass `strict=True` when calling [`findall()`](convenience.md#jsonpath.findall), [`finditer()`](convenience.md#jsonpath.finditer), and similar functions. In strict mode, the syntax and behavior conform to the specification, and no non-standard extensions are registered by default. You can still add them manually if needed.

In this guide, we first outline the standard syntax (see the specification for the formal definition), and then describe the non-standard extensions and their semantics in detail.
This guide first introduces the standard JSONPath syntax (see the RFC for the formal definition), then explains the non-standard extensions and their semantics.

??? info "Preconfigured JSONPath Environments"

Python JSONPath provides two ready-to-use environments:

- **Default environment** – includes relaxed syntax, non-standard selectors, and additional function extensions.
- **Strict environment** – starts with only the RFC 9535 selectors and functions registered. Non-standard extensions can still be enabled explicitly.

For custom setups, subclass [`JSONPathEnvironment`](./api.md#jsonpath.JSONPathEnvironment) and override `setup_function_extensions()`:

```python
from jsonpath import JSONPathEnvironment
from jsonpath.function_extensions import StartsWith


class MyJSONPathEnvironment(JSONPathEnvironment):
def __init__(self) -> None:
super().__init__(strict=True)

def setup_function_extensions(self) -> None:
super().setup_function_extensions()
self.function_extensions["startswith"] = StartsWith()


jsonpath = MyJSONPathEnvironment()
query = jsonpath.compile("...")
```

## JSONPath Terminology

Expand Down
16 changes: 8 additions & 8 deletions jsonpath/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@

# For convenience and to delegate to strict or non-strict environments.
DEFAULT_ENV = JSONPathEnvironment()
STRICT_ENV = JSONPathEnvironment(strict=True)
_STRICT_ENV = JSONPathEnvironment(strict=True)


def compile(path: str, *, strict: bool = False) -> Union[JSONPath, CompoundJSONPath]: # noqa: A001
Expand All @@ -112,7 +112,7 @@ def compile(path: str, *, strict: bool = False) -> Union[JSONPath, CompoundJSONP
JSONPathTypeError: If filter functions are given arguments of an
unacceptable type.
"""
return STRICT_ENV.compile(path) if strict else DEFAULT_ENV.compile(path)
return _STRICT_ENV.compile(path) if strict else DEFAULT_ENV.compile(path)


def findall(
Expand Down Expand Up @@ -146,7 +146,7 @@ def findall(
an incompatible way.
"""
return (
STRICT_ENV.findall(path, data, filter_context=filter_context)
_STRICT_ENV.findall(path, data, filter_context=filter_context)
if strict
else DEFAULT_ENV.findall(path, data, filter_context=filter_context)
)
Expand Down Expand Up @@ -183,7 +183,7 @@ async def findall_async(
an incompatible way.
"""
return (
await STRICT_ENV.findall_async(path, data, filter_context=filter_context)
await _STRICT_ENV.findall_async(path, data, filter_context=filter_context)
if strict
else await DEFAULT_ENV.findall_async(path, data, filter_context=filter_context)
)
Expand Down Expand Up @@ -219,7 +219,7 @@ def finditer(
an incompatible way.
"""
return (
STRICT_ENV.finditer(path, data, filter_context=filter_context)
_STRICT_ENV.finditer(path, data, filter_context=filter_context)
if strict
else DEFAULT_ENV.finditer(path, data, filter_context=filter_context)
)
Expand Down Expand Up @@ -256,7 +256,7 @@ async def finditer_async(
an incompatible way.
"""
return (
await STRICT_ENV.finditer_async(path, data, filter_context=filter_context)
await _STRICT_ENV.finditer_async(path, data, filter_context=filter_context)
if strict
else await DEFAULT_ENV.finditer_async(path, data, filter_context=filter_context)
)
Expand Down Expand Up @@ -292,7 +292,7 @@ def match(
an incompatible way.
"""
return (
STRICT_ENV.match(path, data, filter_context=filter_context)
_STRICT_ENV.match(path, data, filter_context=filter_context)
if strict
else DEFAULT_ENV.match(path, data, filter_context=filter_context)
)
Expand Down Expand Up @@ -359,7 +359,7 @@ def query(
an incompatible way.
"""
return (
STRICT_ENV.query(path, data, filter_context=filter_context)
_STRICT_ENV.query(path, data, filter_context=filter_context)
if strict
else DEFAULT_ENV.query(path, data, filter_context=filter_context)
)
Loading