Malformed URL Normalization in Standard Introduces SSRF Risks

[HackingRepo]

What is the issue with the URL Standard?

Problem

The current URL Standard mandates that malformed inputs such as https:foo or http:127.0.0.1/ be normalized into valid URLs. This behavior is correct per spec and ensures interoperability across browsers. However, in non‑browser contexts (e.g. server‑side libraries, WAFs, SSRF filters), this normalization introduces hidden security bypasses on swisskyrepo/PayloadsAllTheThings#809.

For example:

• A filter may reject http:127.0.0.1/ as malformed.
• The parser then normalizes it into http://127.0.0.1/.
• The request succeeds, bypassing the filter.

This mismatch between “invalid” at the filter level and “valid” at the parser level creates exploitable gaps.

---

Security Impact

• SSRF filters: Attackers can reach internal services by submitting malformed schemes that are later corrected.
• WAF rules: Defenders lose visibility because logs show the corrected input, not the original.
• Cross‑ecosystem risk: Node.js (new URL), Rust (url crate), Python (urllib`), and other libraries all follow the spec, so the bypass is portable across languages.

---

Proposal

Introduce a strict parsing mode in the URL Standard:

• In strict mode, malformed inputs must be rejected outright.
• No implicit corrections should occur.
• This mode would be opt‑in for non‑browser contexts (servers, libraries, security tools).

Additionally:

• Add a security note to the spec warning developers that spec‑compliant normalization is unsafe for validation.
• Encourage libraries to expose both “spec mode” (for browser compatibility) and “strict mode” (for security‑sensitive contexts).

---

Motivation

• Balance interoperability and security: Browsers need consistent parsing, but servers and filters need strict validation.
• Developer clarity: Many developers assume new URL() or equivalent is safe for validation. Explicit strict mode avoids this pitfall.
• Precedent: Other standards (HTML parsing) already distinguish between “quirks mode” and “standards mode.” A strict URL mode would follow this model.

---

Discussion Points

• Should strict mode be opt‑in or opt‑out?
• Which malformed cases should be rejected (schemes, hosts, paths)?
• How can libraries expose strict mode without breaking existing code?

---

This proposal aims to make the URL Standard safer for security‑sensitive contexts while preserving interoperability for browsers.

[jasnell]

-1 on treating this as a security issue. The behavior here is well-defined and has been long established within the whatwg url spec. Further input validation should be applied at the application level rather than the parsing level.

[HackingRepo]

Thanks for the feedback. I agree that the current behavior is well‑defined and correct per the URL Standard, and that application‑level validation is necessary today. My concern is that this creates a recurring security pitfall: developers often assume that spec‑compliant parsers can be safely used for validation, when in fact normalization introduces exploitable bypasses (e.g. http:127.0.0.1/ → http://127.0.0.1/).

I’m not suggesting changing browser behavior, but rather adding an opt‑in strict mode or a security note in the spec. That way:

• Browsers can continue to normalize for interoperability.
• Non‑browser contexts (servers, WAFs, SSRF filters) can reject malformed input explicitly.
• Developers are warned that relying solely on spec‑compliant normalization is unsafe in security‑sensitive contexts.

This proposal is about balancing interoperability with security guidance. Even if the default remains unchanged, a strict mode or explicit warning in the spec would help prevent hidden bypasses in ecosystems that depend on this parser.

[nektro]

maybe only special schemes should be called out?

[HackingRepo]

Thanks for clarifying. I agree that the current behavior is well‑defined for generic schemes. My concern is specifically with special schemes (http, https, ftp, file, etc.), because these are the ones most often used in SSRF/WAF contexts where normalization of malformed input creates exploitable bypasses.

For example:

• https:/127.0.0.1/ normalizes into https://127.0.0.1/, allowing an attacker to reach internal services.
• http:localhost normalizes into http://localhost/, bypassing filters that reject the malformed form.

If the spec were to call out special schemes explicitly:

• Browsers could continue to normalize for interoperability.
• Non‑browser contexts could reject malformed inputs for special schemes, preventing SSRF bypasses.
• This would align with the idea that “generic schemes” remain permissive, but “special schemes” get stricter validation.

So I’d support resolving this by explicitly documenting stricter expectations for special schemes. That way, implementers have clear guidance when security is a concern, without changing the general parsing rules for all schemes.

[annevk]

If the filter rejects it, why does it get to go to the parser? That doesn't seem like a sound security setup. Also, the premise of this entire issue seems to be https://blackhat.com/docs/us-17/thursday/us-17-Tsai-A-New-Era-Of-SSRF-Exploiting-URL-Parser-In-Trending-Programming-Languages.pdf from 2017 which is essentially re-documenting an issue known well before that. Which is that you want to use the same URL parser everywhere, which is the goal of this standard.

[the-moisrex]

Utilities like URL, are designed to convert strings into parsed valid URLs.

Validation of the original string is not the goal of the specs, conversion is AFIK.

You can validate by simply doing something like this: new URL(string).href == string.

Though I accept that we should have a validation-only solution that wouldn't require to actually do the conversion so the performance would be better.

[swhiteman]

You can validate by simply doing something like this: new URL(string).href == string.

Hmm, not really. For one, the scheme and hostname are canonicalized to lowercase.

new URL("HTTPS://EXAMPLE.COM?a=b").href == "HTTPS://EXAMPLE.COM?a=b" // false

For another, dot segments are resolved:

new URL("https://example.com/a/../b/c").href == "https://example.com/a/../b/c" // false

[the-moisrex]

Yes, I'm aware of that. And that is the point. Any modification to the href should make the string Not Normalized.

And using the term invalid for http:127.1 is not exactly correct either. We could consider them as non-failure errors, but not errors.

Ada-url for example simply ignores these non-failure errors AFIK. In my implementation which is a work in progress, I have them as warning flags.

special-scheme-missing-following-solidus non-failure validation error is only about file: scheme not having slashes, not other special schemes; maybe there's something there!

[swhiteman]

Any modification to the href should make the string Not Normalized.

OK, but failing to see the security case for flagging an all-caps hostname as “dirty“ at the same level as injecting //.

At some point we have to distinguish between:

• normalized but not dirty — e.g. empty path/trailing slash, explicit port that matches default/no port, host capitalization, dot segments, plus fixing up unnecessary percent encoding and x-www-form-urlencoded spaces (these last two wrt URLSearchParams, not URL)

and

• normalized via at least one modification that might be considered dirty, security-wise

[HackingRepo]

I understand your point, but this behavior isn’t just theoretical — it affects widely used libraries and runtimes across multiple ecosystems (Axios, Node’s url, Rust’s reqwest, Got, Ky, etc.). That means millions of developers are exposed to the same normalization pitfall.

Because of that, I think it’s important to treat this as more than “just conversion.” In practice, filters and parsers are often chained together, and normalization of malformed inputs creates exploitable bypasses. If you’d like, I can provide concrete test cases or screenshots showing the issue in these libraries.

Could you please test against those implementations and confirm? Even if the spec itself doesn’t change, having explicit guidance or a validation‑only mode would help prevent these recurring security gaps.

[annevk]

I think you still have to explain the attack vector a bit better. If I were to write a URL filter, I would never attempt to filter unparsed inputs. And I would ensure the entire system uses the same URL parser. If that cannot be guaranteed you'll have to perform some kind of normalization. But what normalization you have to perform is probably unique to the system if the URL parsers in it are non-conforming as they can be susceptible to different kinds of attacks.

For instance, the presentation I linked above mentions curl. curl's author doesn't want to implement the URL standard, nor does he want to implement the RFC specifications. Instead curl has some kind of garbage-in-gargabe-out policy, which seems like a huge security issue to me, but thus far it hasn't been exploited much(?). Perhaps because people have learned how to normalize URLs before passing them to curl.

[HackingRepo]

Thanks for the feedback. Let me clarify the attack vector more concretely:

The issue arises when a filter rejects a malformed input string, but the parser later normalizes it into a valid URL. That creates a mismatch between “invalid at filter stage” and “valid at parser stage,” which attackers can exploit.

For example:

• Input: http:127.0.0.1/
• Filter: rejects as malformed (no // after scheme)
• Parser: normalizes into http://127.0.0.1/
• Result: request succeeds, bypassing the filter

This matters because in practice:

• Filters often operate on raw input (e.g. WAF rules, SSRF guards).
• Parsers normalize aggressively for interoperability.
• If the filter and parser disagree, the attacker wins.

I agree that ideally the entire system should use the same parser. But in reality:

• Ecosystems mix parsers (Node’s url, Rust’s url, Python’s urllib, curl, etc.).
• Some parsers are spec‑compliant, others are “garbage‑in‑garbage‑out.”
• Normalization rules differ, so a filter written against one parser may be bypassed when another parser is used downstream.

That’s why I think the spec should explicitly call out special schemes (http, https, ftp, file) and warn that normalization of malformed inputs in these schemes can introduce SSRF/WAF bypasses. Even if the spec doesn’t change parsing rules, a security note would help implementers understand that conversion ≠ validation, and that filters must reject malformed inputs before parsing.

So the proposal isn’t to make the parser itself a validator, but to document the risk and encourage libraries to expose a strict mode or validation API for security‑sensitive contexts.

[the-moisrex]

I do agree with the idea of strict mode. Some parts of the specs have this mode too, but it's not restrictive enough.

And double solidus problem in my opinion is not that big of deal as other problems. These things come to my mind, but I'm sure there are more places where we can assert a more restrictive option to the specs:

• New lines and tabs (what a dumb thing to remove them, seriously)
• Octal and HEX IPv4
• Leading zeros and what not in IPs (We seriously can have infinite ways of writing any IP address, this is not okay, we can smuggle large streams of strings that are still valid IP addresses)
• Invalid Unicode code points
• Not Verifying DNS Length by default
• Trailing empty IPv4 Octets
• Empty label in domains
• Percent Encoded dots in paths
• Long punycode encoded domains can pretty much be a DDoS attack or resource draining attacks since punycode is very slow

I think we need a restrictive strict mode.

There are just so many random things in URL Specs.

For example:

Valid URLs:

file:
fILe://C|\\//\////\\\\/\/\/\\\/\
any-random-scheme:

Invalid URLs:

http:
https:
ws:
wss:
ftp:

---

This is a valid WHATWG URL with a hidden "Hello World" binary message encoded within:

• / is 1
• \ is 0

https:/\//\////\\//\/\/\\/\\///\\/\\///\\/\\\\//\//////\/\/\\\/\\/\\\\/\\\//\//\\/\\///\\//\//example.com

---

• file::::::: totally valid URL
• http:: hold on, that's not valid at all!

---

These two URLs result in different strings generated!

file://localhost/path/to/somethere
file://127.0.0.1/path/to/somewhere

https://url.spec.whatwg.org/#file-host-state

---

Parsing IPv4 in a host of a URL requires different algorithm than the good old inet_pton4. This means, all of these are invalid in inet_pton4, but valid in the URLs:

"0000000.0.0.0"
"0x000000.0.0.0"
"0X0000.0x.0.0"
"0"
"0x"
"0X
"00000"
"0x0000"
""
"."
"127.0.0.1"
"0x7f.1"
"0x7f000001"
"0x0000000007f.0x1"
"127.0.0x0.1"
"127.0x0.0x0.1"
"127.0x0.0x0.0x1"
"127.0.0x0.0x0001"
"0x7f.0x0000001"
"0x0007f.0x0001"
"0x007f.0.0x00001"
"0x7f.0.0.0x1"
"0x7f.0.0x000.0x1"
"2130706433"
"127.1"
"127.0x00.1"
"127.0x000000000000000.0.1"
"000123"
"0xff"
"1.256"

---

All of these URLs refer to the same place:

http://127.0.0.1/
http://0x7F.1/
http://0x7F000001
http://0x0000000007F.0X1
http://127.0.0x0.1
http://127.0X0.0x0.1
http://127.0X0.0x0.0x1
http://127.0.0x0.0x0000000000000000000000000000000000000000000000000000000000000001
http://0x7F.0x00000000000000000000000001
http://0x000000000000000007F.0x00000000000000000000000001
http://0x000000000000000007F.0.0x00000000000000000000000001
http://0x7F.0.0.0x1
http://0x7F.0.0x000.0x1
http://2130706433
http://127.1
http://127.0x00.1
http://127.0x000000000000000.0.1

😐😐😐😐😐

---

I mean, WTF?

file://C|\windows
file:///C:/windows

---

Browsers make mistakes too

There are a few bugs with the current implementation of URL Parser for firefox. (Chromium has them too, but that's another bug).

This URL:

http://127.0.0.1/..//./one/%2E./%2e/two/././././%2e/%2e/.././three/four/%2e%2e/five/.%2E/%2e

Should be equal to this URL (from ada-url (https://www.ada-url.com/playground?url=http%3A%2F%2F127.0.0.1%2F..%2F%2F.%2Fone%2F%252E.%2F%252e%2Ftwo%2F.%2F.%2F.%2F.%2F%252e%2F%252e%2F..%2F.%2Fthree%2Ffour%2F%252e%252e%2Ffive%2F.%252E%2F%252e)):

http://127.0.0.1//three/
     | |        |       
     | |        `------- pathname_start 16
     | |        `------- host_end 16
     | `---------------- host_start 7
     | `---------------- username_end 7
     `------------------ protocol_end 5

It seems like URL parser of Firefox has a bug which doesn't decode the last segment of the path for some reason.

For example, http://localhost/page/%2e%2e is just http://localhost/

---

These are some of the things I've noticed before and have written about them on my project's telegram channels and copied them here.

[the-moisrex]

*Firefox bug has been fixed but still, I'm sure there are other implementation that don't even know about %2e.

We should really introduce a strict mode of some sort or at least introduce some more validation errors.

[HackingRepo]

Glad to hear the Firefox bug was fixed but that highlights the deeper problem: implementations shouldn’t need to stumble across %2e quirks in the first place. The WHATWG spec currently normalizes too many malformed inputs, which makes it hard for developers to rely on URL parsing for security‑sensitive contexts.

A strict mode (or at least more validation errors) would give us a way to reject inputs like encoded dots, octal/hex IPv4, or empty labels outright instead of silently accepting them. That way, browsers can keep their permissive behavior for compatibility, but servers, libraries, and security tools can opt into a safer mode.

Fixing individual bugs is good, but without a strict mode we’ll keep rediscovering these edge cases across different implementations.