[Doc] Add technical article series (Part 1 & Part 2) (#42)

* docs: drafts of part1 & part2

Signed-off-by: Zerohertz <ohg3417@gmail.com>

* docs: technical article accuracy

Signed-off-by: Zerohertz <ohg3417@gmail.com>

* [Doc] Update Chunked Prefill Section with Actual Implementation (#46)

Signed-off-by: Hyoseop Song <crad_on25@naver.com>

* Why continous batching and chunked prefill are slower than baseline in cpu (#48)

* [Feature] Implement continuous batching scheduler  (#37)

* fix: resolve incomplete type in JsonValue variant

Use unique_ptr<JsonObject> instead of JsonObject directly in variant
to avoid incomplete type errors with recursive type definitions.

Signed-off-by: teddygood <ibear6954@gmail.com>

* refactor: extend request progress tracking for chunked prefill

Signed-off-by: teddygood <ibear6954@gmail.com>

* refactor: switch scheduler to single-type batches with decode-first policy

Signed-off-by: teddygood <ibear6954@gmail.com>

* add: continuous batching loop with interleaved execution

Signed-off-by: teddygood <ibear6954@gmail.com>

* add: paged attention batch slot mapping

Signed-off-by: teddygood <ibear6954@gmail.com>

* add: integrate chunked prefill utilities

Signed-off-by: teddygood <ibear6954@gmail.com>

* [Feature] Async request submission with `arrival_delay_ms` support (#39)

* feat: async request submission with arrival_delay_ms support

Signed-off-by: Zerohertz <ohg3417@gmail.com>

* fix: add `async_queue.has_pending()` to both exit conditions

Signed-off-by: Zerohertz <ohg3417@gmail.com>

---------

Signed-off-by: Zerohertz <ohg3417@gmail.com>

Signed-off-by: Hyogeun Oh (오효근) <ohg3417@gmail.com>

* fix: address Copilot review for scheduler prefill and non-paged fallback

Signed-off-by: teddygood <ibear6954@gmail.com>

---------

Signed-off-by: teddygood <ibear6954@gmail.com>
Signed-off-by: Hyogeun Oh (오효근) <ohg3417@gmail.com>
Co-authored-by: Zerohertz <ohg3417@gmail.com>
Signed-off-by: roundtable <ryu071511@gmail.com>
Signed-off-by: RRoundTable <ryu071511@gmail.com>

* [Feat] Show Effect of Continuous Batching & Chunked Prefill on CPU (#43)

* feat: add BenchmarkResult struct with JSON serialization and deserialization

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: roundtable <ryu071511@gmail.com>

* feat: add Comparison class for side-by-side result display

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: roundtable <ryu071511@gmail.com>

* feat: add --save-results flag to persist benchmark metrics as JSON

Runner functions now return BenchmarkResult instead of int.
Single prompt mode tracks prefill/decode timing separately.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: roundtable <ryu071511@gmail.com>

* feat: add --compare-a/--compare-b mode for side-by-side result comparison

Comparison mode pre-scans argv before argparser, so it works without
the positional model path argument. Supports --output-format table|json.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: roundtable <ryu071511@gmail.com>

* feat: add --max-tokens-per-batch flag to control chunked prefill size

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: roundtable <ryu071511@gmail.com>

* fix: estimate paged attention memory from sequence length when blocks are freed

In sequential mode, blocks are freed after each request completes,
leaving block_tables empty. Now falls back to estimating from
(prompt_tokens + generated_tokens) / block_size.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: roundtable <ryu071511@gmail.com>

* feat: add comparison workload example for benchmarking

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: roundtable <ryu071511@gmail.com>

* fix: link against Homebrew LLVM libc++ to resolve __hash_memory symbol error on macOS

Homebrew Clang compiles against its newer libc++ headers but Xcode's
linker was linking against the older macOS SDK libc++, causing undefined
symbol errors for std::__hash_memory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: RRoundTable <ryu071511@gmail.com>
Signed-off-by: roundtable <ryu071511@gmail.com>

* docs: update README with build instructions and benchmark/comparison usage

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: RRoundTable <ryu071511@gmail.com>
Signed-off-by: roundtable <ryu071511@gmail.com>

* docs: add CPU benchmark experiments for paged attention, batching, and chunked prefill

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: RRoundTable <ryu071511@gmail.com>
Signed-off-by: roundtable <ryu071511@gmail.com>

* fix pre commit

Signed-off-by: RRoundTable <ryu071511@gmail.com>
Signed-off-by: roundtable <ryu071511@gmail.com>

* [Feat] Add `--max-tokens-per-batch` CLI option (#45)

* feat: add --max-tokens-per-batch CLI option

Signed-off-by: Hyoseop Song <crad_on25@naver.com>

* feat: add chunked prefill test example

Signed-off-by: Hyoseop Song <crad_on25@naver.com>

---------

Signed-off-by: Hyoseop Song <crad_on25@naver.com>
Signed-off-by: roundtable <ryu071511@gmail.com>

* docs: update benchmark experiments with mixed-length workload and chunked prefill trace

- Redesign comparison_workload.json with 6 requests mixing long (70-79 tok)
  and short (6-8 tok) prompts to trigger chunked prefill with -bt 64
- Add per-request chunk logging in batched_runner for prefill visibility
- Re-run all 4 experiments and update results tables
- Add scheduling trace section showing chunked prefill behavior step-by-step

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: roundtable <ryu071511@gmail.com>

* fix markdown lint

Signed-off-by: roundtable <ryu071511@gmail.com>

---------

Signed-off-by: roundtable <ryu071511@gmail.com>
Signed-off-by: RRoundTable <ryu071511@gmail.com>
Signed-off-by: Hyoseop Song <crad_on25@naver.com>
Co-authored-by: Hyoseop Song <crad_on25@naver.com>
Signed-off-by: roundtable <ryu071511@gmail.com>
Signed-off-by: RRoundTable <ryu071511@gmail.com>

* add continous batch and chunked prefill: why continous batch and chunked prefill are slower

Signed-off-by: roundtable <ryu071511@gmail.com>
Signed-off-by: RRoundTable <ryu071511@gmail.com>

* Re-edit

Signed-off-by: roundtable <ryu071511@gmail.com>
Signed-off-by: RRoundTable <ryu071511@gmail.com>

* [Doc] Add section 9.6 GPU benchmark after CPU analysis (#50)

* docs: add section 9.6 GPU benchmark after CPU analysis

Signed-off-by: teddygood <ibear6954@gmail.com>

* docs: refresh 0.6B GPU benchmark results and add raw JSON artifacts

Signed-off-by: teddygood <ibear6954@gmail.com>

* docs: visualization of benchmark data

Signed-off-by: Zerohertz <ohg3417@gmail.com>

* docs: clarify 9.6 scope and add HF request-level batching control

Signed-off-by: teddygood <ibear6954@gmail.com>

---------

Signed-off-by: teddygood <ibear6954@gmail.com>
Signed-off-by: Zerohertz <ohg3417@gmail.com>
Co-authored-by: Zerohertz <ohg3417@gmail.com>
Signed-off-by: roundtable <ryu071511@gmail.com>
Signed-off-by: RRoundTable <ryu071511@gmail.com>

* fix: pre-commit formatting

Signed-off-by: RRoundTable <ryu071511@gmail.com>

* docs: convert ASCII diagrams to Mermaid charts

Signed-off-by: loveysuby <loveysuby@users.noreply.github.com>

---------

Signed-off-by: teddygood <ibear6954@gmail.com>
Signed-off-by: Hyogeun Oh (오효근) <ohg3417@gmail.com>
Signed-off-by: roundtable <ryu071511@gmail.com>
Signed-off-by: RRoundTable <ryu071511@gmail.com>
Signed-off-by: Hyoseop Song <crad_on25@naver.com>
Signed-off-by: Zerohertz <ohg3417@gmail.com>
Signed-off-by: loveysuby <loveysuby@users.noreply.github.com>
Co-authored-by: Chanho Lee <ibear6954@gmail.com>
Co-authored-by: Zerohertz <ohg3417@gmail.com>
Co-authored-by: Hyoseop Song <crad_on25@naver.com>
Co-authored-by: loveysuby <loveysuby@users.noreply.github.com>

---------

Signed-off-by: Zerohertz <ohg3417@gmail.com>
Signed-off-by: Hyoseop Song <crad_on25@naver.com>
Signed-off-by: teddygood <ibear6954@gmail.com>
Signed-off-by: Hyogeun Oh (오효근) <ohg3417@gmail.com>
Signed-off-by: roundtable <ryu071511@gmail.com>
Signed-off-by: RRoundTable <ryu071511@gmail.com>
Signed-off-by: loveysuby <loveysuby@users.noreply.github.com>
Co-authored-by: Hyoseop Song <crad_on25@naver.com>
Co-authored-by: Wontak Ryu <ryu071511@gmail.com>
Co-authored-by: Chanho Lee <ibear6954@gmail.com>
Co-authored-by: loveysuby <loveysuby@users.noreply.github.com>

Author: 4 people

.markdownlint.json

@@ -1,9 +1,8 @@
 {
   "MD013": false,
-  "MD033": {
-    "allowed_elements": ["h1", "h3"]
-  },
+  "MD033": false,
   "MD040": false,
+  "MD045": false,
   "MD049": { "style": "asterisk" },
   "MD051": false
 }