fix: http path and headers larger than available stream size - #1292
Merged
Conversation
Contributor
Dependency ReviewThe following issues were found:
Snapshot WarningsConsider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. License Issuesosal/threadx/CMakeLists.txt
external/crypto/micro-ecc/CMakeLists.txt
external/crypto/tiny-aes128/CMakeLists.txt
osal/freertos/CMakeLists.txt
external/crypto/mbedtls/CMakeLists.txt
external/protobuf/CMakeLists.txt
external/segger_rtt/CMakeLists.txt
lwip/lwip/CMakeLists.txt
infra/syntax/CMakeLists.txt
cmake/emil_test_helpers.cmake
external/args/CMakeLists.txt
OpenSSF ScorecardScorecard details
Scanned Files
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes HTTP request transmission when the formatted request line + headers exceed the maximum send stream size provided by the connection, ensuring headers (and long request targets) are emitted correctly across multiple send-stream allocations.
Changes:
- Cap initial
RequestSendStream()sizes toMaxSendStreamSize()inHttpClientImpl, avoiding oversize send-stream requests. - Make
HttpRequestFormatterable to write/consume headers incrementally via a runningheaderPosition, enabling header splitting across multiple send streams. - Add tests that cover headers exceeding the send stream size (including custom headers).
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| services/network/test/TestHttpClient.cpp | Adds tests for requests where the header block exceeds maxSendStreamSize, validating correct full request output. |
| services/network/HttpClientImpl.cpp | Requests send streams using min(request->Size(), MaxSendStreamSize()) to respect connection limits. |
| services/network/Http.hpp | Updates HttpRequestFormatter interface to support incremental header writing/consumption state. |
| services/network/Http.cpp | Implements incremental header writing (WriteHeader) and tracks header progress (headerPosition). |
Contributor
✅
|
| Descriptor | Linter | Files | Fixed | Errors | Warnings | Elapsed time |
|---|---|---|---|---|---|---|
| ✅ ACTION | actionlint | 11 | 0 | 0 | 0.32s | |
| ✅ ACTION | zizmor | 11 | 0 | 0 | 0 | 2.71s |
| ✅ CPP | clang-format | 1098 | 8 | 0 | 0 | 9.81s |
| ✅ DOCKERFILE | hadolint | 2 | 0 | 0 | 0.05s | |
| ✅ JSON | jsonlint | 7 | 0 | 0 | 0.37s | |
| ✅ JSON | prettier | 7 | 0 | 0 | 0 | 0.54s |
| markdownlint | 8 | 0 | 5 | 0 | 1.43s | |
| ✅ MARKDOWN | markdown-table-formatter | 8 | 0 | 0 | 0 | 0.27s |
| betterleaks | yes | 1 | 5 | 1.3s | ||
| ✅ REPOSITORY | checkov | yes | no | no | 34.27s | |
| ✅ REPOSITORY | git_diff | yes | no | no | 0.09s | |
| ✅ REPOSITORY | grype | yes | no | no | 67.64s | |
| ✅ REPOSITORY | ls-lint | yes | no | no | 0.01s | |
| ✅ REPOSITORY | osv-scanner | yes | no | no | 0.61s | |
| ✅ REPOSITORY | secretlint | yes | no | no | 16.88s | |
| ✅ REPOSITORY | syft | yes | no | no | 1.98s | |
| ✅ REPOSITORY | trivy | yes | no | no | 19.27s | |
| ✅ REPOSITORY | trivy-sbom | yes | no | no | 0.29s | |
| ✅ REPOSITORY | trufflehog | yes | no | no | 14.25s | |
| lychee | 140 | 1 | 0 | 115.99s | ||
| prettier | 21 | 1 | 1 | 0 | 0.66s | |
| ✅ YAML | v8r | 21 | 0 | 0 | 11.04s | |
| ✅ YAML | yamllint | 21 | 0 | 0 | 0.82s |
Detailed Issues
⚠️ REPOSITORY / betterleaks - 1 error
warning: private-key has detected secret for file services/network/CertificatesMbedTls.cpp.
┌─ services/network/CertificatesMbedTls.cpp:125:21
│
125 │ stream << "HIDDEN_BY_MEGALINTER\r\n";
│ ╰────────────────────────────────────────────────^
warning: generic-api-key has detected secret for file external/crypto/tiny-aes128/TinyAes.c.
┌─ external/crypto/tiny-aes128/TinyAes.c:17:4
│
17 │ key:
│ ╭────^
18 │ │ 2b7e151628aed2a6abf7158809cf4f3c
│ ╰────────────────────────────────────^
warning: generic-api-key has detected secret for file services/network/WebSocket.cpp.
┌─ services/network/WebSocket.cpp:81:50
│
81 │ headers.push_back(services::HttpHeader("Sec-Websocket-Key", "AQIDBAUGBbgJCgsMDQ4PEC=="));
│ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
warning: private-key has detected secret for file services/network/test_doubles/Certificates.cpp.
┌─ services/network/test_doubles/Certificates.cpp:56:15
│
56 │ "HIDDEN_BY_MEGALINTER\r\n";
│ ╰──────────────────────────────────────────^
warning: private-key has detected secret for file services/network/test_doubles/Certificates.cpp.
┌─ services/network/test_doubles/Certificates.cpp:108:15
│
108 │ "HIDDEN_BY_MEGALINTER\r\n";
│ ╰──────────────────────────────────────────^
warning: 5 warnings emitted
⚠️ SPELL / lychee - 1 error
📝 Summary
---------------------
🔍 Total..........697
🔗 Unique.........661
✅ Successful.....691
⏳ Timeouts.........0
🔀 Redirected.....310
👻 Excluded.........5
❓ Unknown..........0
🚫 Errors...........1
⛔ Unsupported......1
Errors in external/protoc/CMakeLists.txt
[404] https://github.com/protocolbuffers/protobuf/releases/download/v$%7Bprotobuf_tag%7D/protoc-$%7Bprotobuf_version%7D-$%7Bos_postfix%7D.zip (at 18:13) | Rejected status code: 404 Not Found
Hint: Followed 310 redirects. You might want to consider replacing redirecting URLs with the resolved URLs. Use verbose mode (`-v`/`-vv`) to see redirection details.
Hint: You can configure accepted/rejected response codes with `-a` or `--accept`
⚠️ MARKDOWN / markdownlint - 5 errors
.github/instructions/microtest.instructions.md:7 error MD041/first-line-heading/first-line-h1 First line in a file should be a top-level heading [Context: "## Google Test Suite Coding Ru..."]
external/crypto/tiny-aes128/README.md:1 error MD041/first-line-heading/first-line-h1 First line in a file should be a top-level heading [Context: "### Tiny AES128 in C"]
external/crypto/tiny-aes128/README.md:29 error MD046/code-block-style Code block style [Expected: fenced; Actual: indented]
external/crypto/tiny-aes128/README.md:39 error MD046/code-block-style Code block style [Expected: fenced; Actual: indented]
external/crypto/tiny-aes128/README.md:49 error MD046/code-block-style Code block style [Expected: fenced; Actual: indented]
⚠️ YAML / prettier - 1 error
[error] Explicitly specified pattern "documents/modules/ROOT/examples/clangformat.yaml" is a symbolic link.
.clusterfuzzlite/project.yaml 46ms (unchanged)
.github/dependabot.yml 13ms (unchanged)
.github/workflows/ci.yml 67ms (unchanged)
.github/workflows/dependency-scanner.yml 8ms (unchanged)
.github/workflows/documentation.yml 19ms (unchanged)
.github/workflows/fuzzing-batch.yml 6ms (unchanged)
.github/workflows/fuzzing-cron.yml 4ms (unchanged)
.github/workflows/fuzzing-pr.yml 8ms (unchanged)
.github/workflows/linting-formatting.yml 13ms (unchanged)
.github/workflows/release-please.yml 7ms (unchanged)
.github/workflows/security.yml 8ms (unchanged)
.github/workflows/static-analysis.yml 21ms (unchanged)
.github/workflows/validate-pr.yml 17ms (unchanged)
.ls-lint.yml 2ms
.mega-linter.yml 4ms (unchanged)
antora-playbook-branch.yml 5ms (unchanged)
antora-playbook-site.yml 3ms (unchanged)
documents/antora.yml 2ms (unchanged)
documents/supplemental-ui/ui.yml 1ms (unchanged)
mull.yml 2ms (unchanged)
Notices
📣 MegaLinter 9.5.0 is out! Discover the new features and security recommendations in the release announcement. (Skip this info by defining SECURITY_SUGGESTIONS: false)
See detailed reports in MegaLinter artifacts
Your project could benefit from a custom flavor, which would allow you to run only the linters you need, and thus improve runtime performances. (Skip this info by defining FLAVOR_SUGGESTIONS: false)
- Documentation: Custom Flavors
- Command:
npx mega-linter-runner@9.6.0 --custom-flavor-setup --custom-flavor-linters ACTION_ACTIONLINT,ACTION_ZIZMOR,CPP_CLANG_FORMAT,DOCKERFILE_HADOLINT,JSON_JSONLINT,JSON_PRETTIER,MARKDOWN_MARKDOWNLINT,MARKDOWN_MARKDOWN_TABLE_FORMATTER,REPOSITORY_CHECKOV,REPOSITORY_GIT_DIFF,REPOSITORY_BETTERLEAKS,REPOSITORY_GRYPE,REPOSITORY_LS_LINT,REPOSITORY_OSV_SCANNER,REPOSITORY_SECRETLINT,REPOSITORY_SYFT,REPOSITORY_TRIVY,REPOSITORY_TRIVY_SBOM,REPOSITORY_TRUFFLEHOG,SPELL_LYCHEE,YAML_PRETTIER,YAML_YAMLLINT,YAML_V8R
Show us your support by starring ⭐ the repository
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (3)
services/network/test/TestHttpClient.cpp:422
- Prefer EXPECT_THAT over EXPECT_EQ for clearer failure output and consistency with matcher-based assertions.
ExecuteAllActions();
EXPECT_EQ("GET /api/thing HTTP/1.1\r\nAuthorization:Basic Y29ubmVjdGl2aXR5OmlzY29vbA==\r\nConnection:close\r\nHost:localhost\r\n\r\n", connection.SentDataAsString());
services/network/test/TestHttpClient.cpp:401
- The test name refers to a “header” being larger than the send stream, but this scenario also depends on splitting the request line (which is not a header). Renaming to reflect that this is the full request header block (request line + headers) would make the intent clearer and easier to maintain.
TEST_F(HttpClientTest, header_larger_than_send_stream_is_split_across_multiple_send_streams)
services/network/test/TestHttpClient.cpp:409
- Prefer EXPECT_THAT over EXPECT_EQ for clearer failure output and consistency with the Google Test matcher style used elsewhere in this file.
This issue also appears on line 421 of the same file.
ExecuteAllActions();
EXPECT_EQ("GET /api/thing HTTP/1.1\r\nHost:localhost\r\n\r\n", connection.SentDataAsString());
Baris Tanyeri (BarisTanyeri)
approved these changes
Aug 5, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (1)
services/network/Http.cpp:128
- HttpRequestFormatter::Size() now always recomputes HeaderBlockSize() (which calls HeadersSize() and iterates over all headers). For large bodies sent over many small send streams, Size()/Consume() are called per fragment by HttpClientImpl, so this becomes repeated O(#headers) work even after headers are fully sent (previously Size() became O(1) once sentHeader=true). Consider caching the computed header block size in the formatter at construction (or lazily on first use) and using that cached value in Size()/Consume() to avoid repeated header-size scans.
std::size_t HttpRequestFormatter::Size() const
{
return (HeaderBlockSize() - headerPosition) + content.size();
}
André Jordan (andjordan)
disabled auto-merge
August 5, 2026 11:15
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (1)
services/network/Http.cpp:148
- HttpRequestFormatter::Write() always calls WriteHeader(), even after all header bytes have been sent (headerPosition == headerBlockSize). This re-walks and re-slices the entire header block for every subsequent content send-stream chunk, which is a performance regression compared to the previous sentHeader logic. Consider skipping header formatting once headers are fully consumed, and returning early when the stream has no space left after writing headers.
std::size_t HttpRequestFormatter::Write(infra::TextOutputStream stream) const
{
auto written = WriteHeader(stream, headerPosition);
auto available = std::min(stream.Available(), content.size());
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes HTTP request transmission when the formatted request line + headers exceed the maximum send stream size provided by the connection, ensuring headers (and long request targets) are emitted correctly across multiple send-stream allocations.
Changes:
RequestSendStream()sizes toMaxSendStreamSize()inHttpClientImpl, avoiding oversize send-stream requests.HttpRequestFormatterable to write/consume headers incrementally via a runningheaderPosition, enabling header splitting across multiple send streams.