Reserved URL characters (#, ?) in decoded path params corrupt request.url and url_path_for()

[Koded0214h]

Description

ASGI delivers scope["path"] already percent-decoded. Starlette uses that decoded string directly in two places without re-encoding reserved URL characters (#, ?), which means a literal # or ? in a decoded path segment gets reinterpreted as URL syntax (fragment/query separator) instead of staying part of the path.

1. Inbound: request.url / request.url.path

starlette/datastructures.py, URL.__init__:

path = scope["path"]
url = SplitResult(scheme=scheme, netloc=netloc, path=path, query=query, fragment="").geturl()

SplitResult.geturl() does no escaping — it just joins the components with their normal separators. If path contains a literal #, it becomes the start of the fragment when the resulting string is later parsed (e.g. by .path/.fragment property access via urlsplit), even though the router matched against the correct, un-truncated value.

2. Outbound: url_path_for() / Router.url_path_for()

starlette/routing.py:92-103 (replace_params) splices the converted parameter string directly into the path template with no escaping:

  value = convertor.to_string(value)
  path = path.replace("{" + key + "}", value)

StringConvertor.to_string / PathConvertor.to_string (starlette/convertors.py:26-30, 39-40) do no encoding either — they just return str(value). So a path param value containing # or ? produces a URL string that, if actually dereferenced, resolves to a different value than what was passed to url_for.

Example

  from starlette.applications import Starlette
  from starlette.responses import PlainTextResponse
  from starlette.routing import Route
  from starlette.testclient import TestClient

  async def homepage(request):
      return PlainTextResponse(
          f"path={request.url.path!r} full={str(request.url)!r} fragment={request.url.fragment!r}"
      )

  app = Starlette(routes=[Route("/foo/{name}", homepage, name="homepage")])
  client = TestClient(app)

  r = client.get("/foo/a%23b%3Fc=1")
  print(r.status_code, r.text)
  # routing itself resolves `name` to the correct 'a#b?c=1' (handler works fine
  # if it reads request.path_params directly), but:
  # request.url.path     == '/foo/a'        (should be '/foo/a#b?c=1')
  # request.url.fragment == 'b?c=1'         (should be '')
  # str(request.url)     == 'http://testserver/foo/a#b?c=1'

  up = app.url_path_for("homepage", name="a#b?c=1")
  print(up)
  # -> '/foo/a#b?c=1' -- if actually requested, resolves to name='a', not
  #    the value passed to url_for

Why this matters

• request.url.path is commonly used for logging, CSRF/origin checks, and
  building canonical links/redirects — a silently truncated value there is a
  correctness issue, not just cosmetic.
• The same un-escaped substitution feeds Router's redirect_slashes
  handling (URL(scope=redirect_scope)), so a Location header built from
  it can point at the wrong resource.
• url_for() producing a URL that doesn't round-trip to the value you asked
  for is a fairly sharp footgun for anyone building links from user-supplied
  path segments.

Environment

• OS: macOS (Darwin 25.4.0)
• Python: 3.12.2
• starlette: 1.3.1

I think the fix is to percent-encode reserved characters (at minimum #, ?, and probably % itself for round-tripping) when: (a) building the URL string from scope["path"] in datastructures.py, and (b) substituting converted values into the path template in replace_params. Wanted to raise this as a discussion first since it touches core URL construction and I'd like input on the right place/approach before attempting a PR, e.g. whether encoding belongs in Convertor.to_string() per-convertor, or centrally in replace_params/URL.

[VectorPeak]

This looks like a real bug, and the distinction you’re drawing is the important one: scope["path"] is a decoded routing path, not a string that can safely be interpolated back into a URL. A decoded # or ? is still valid path data at the ASGI layer, but once Starlette uses that decoded value to build a URL string, those characters regain their URL syntax meaning. That explains the split you’re seeing: routing and request.path_params can still be correct, while request.url, request.url.path, and generated URLs become semantically different from the original path. For the inbound side, the safest fix is probably not to use scope["path"] directly when constructing the URL text. Starlette should prefer scope["raw_path"] when available, since that is the path as bytes before decoding and before the query string. If raw_path is missing, the fallback should quote the decoded path before joining it into the URL, while preserving / as a path separator. Roughly, the behavior should be along these lines:

from urllib.parse import quote

raw_path = scope.get("raw_path")

if raw_path is not None:
 path = raw_path.decode("ascii")
else:
 path = quote(scope["path"], safe="/%")

The exact safe set needs a little care. In particular, the regression test should assert that:

client.get("/foo/a%23b%3Fc=1")

continues to route with:

request.path_params["name"] == "a#b?c=1"

while also producing:

request.url.path == "/foo/a#b?c=1"
request.url.fragment == ""
str(request.url) == "http://testserver/foo/a%23b%3Fc=1"

The outbound side is the same category of issue, but it should be fixed in route generation rather than in URLPath parsing. url_path_for() should treat converted path parameters as path-segment data and percent-encode them before substituting into the route template. For {name} this should encode /, #, and ?; for {path:path} it should preserve / but still encode # and ?. Something like this is the shape I’d expect, with the safe characters depending on the convertor:

from urllib.parse import quote

value = convertor.to_string(value)

if convertor is path_convertor:
 value = quote(value, safe="/")
else:
 value = quote(value, safe="")

path = path.replace("{" + key + "}", value)

That is intentionally illustrative rather than the final patch, because Starlette’s convertor API may need a cleaner way to express “characters safe for URL generation” per convertor. The important behavior is that url_path_for("homepage", name="a#b?c=1") should generate a path that can be dereferenced and route back to the same parameter value, rather than producing a string where #b?c=1 becomes a fragment. A good PR here would include both sets of tests: one for Request.url construction from an ASGI scope/TestClient request, and one for url_path_for() with at least str and path convertors. It would also be worth including %2F / / cases so we don’t accidentally change the semantics of {name} versus {path:path} while fixing # and ?. One caveat worth checking in the implementation: raw_path is optional in ASGI, so the fallback path still matters. Also, query_string must remain handled separately; the path escaping fix should not quote or otherwise alter the query string bytes.

---

If my answer solved your problem, you can click answered the question. I'm really here to help, and along the way I'm also collecting Galaxy Brain badges haha 😆

[Koded0214h]

@Kludex can i work on this?

[VectorPeak]

Sorry, I'm just passing by here, haha, but I wanted to say: this is a really thoughtful and useful report. Thanks for taking the time to dig into the details and explain it clearly. Keep going — cheering you on!

[Koded0214h]

Thank youu, just starting out on open source 🥲