Skip to content

Commit 48d1601

Browse files
committed
Document vertical slice methodology and add test-serdes just recipe
SERDES_PLAN.md updates: - Added 'Strategic Approach: Vertical Slice Methodology' section - Documents philosophy: prove entire pipeline with ONE example first - Explains rationale: early validation, fast feedback, template creation - Detailed Phase 2 completion status (PUBLISH vertical slice) - Documents key technical discoveries during implementation: * txaio initialization requirement * Autobahn serializer API details * Publish message structure (no options attribute) * Non-bijective serialization handling * At least one matching semantics - Phase 3 outlines horizontal scaling strategy - Prioritized list of all WAMP message types - Updated phase structure with clear tactical roadmap justfile updates: - Added test-serdes recipe for running WAMP message serdes tests - Usage: just test-serdes [venv] - Runs examples/serdes/tests/test_publish.py with pytest - Auto-detects system Python venv if not specified - Follows existing justfile patterns and conventions This provides clear documentation of implementation approach and easy test execution for developers. Related: wamp-proto#556
1 parent b72f8f7 commit 48d1601

File tree

2 files changed

+144
-20
lines changed

2 files changed

+144
-20
lines changed

examples/serdes/SERDES_PLAN.md

Lines changed: 128 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -376,30 +376,138 @@ for serializer in [JsonSerializer(), MsgPackSerializer(), FlatBuffersSerializer(
376376

377377
## Implementation Plan
378378

379-
### Phase 1: Foundation (Current) ✅
380-
- [x] Performance benchmarking infrastructure
379+
### Strategic Approach: Vertical Slice Methodology
380+
381+
**Philosophy**: Prove the entire pipeline end-to-end with ONE example before scaling horizontally.
382+
383+
**Why Vertical Slice?**
384+
1. **Early Validation**: Discover design issues with real implementation
385+
2. **Fast Feedback**: Working tests provide immediate value
386+
3. **Template Creation**: First example becomes the pattern for all others
387+
4. **Risk Reduction**: Find problems before investing in full coverage
388+
5. **Iterative Refinement**: Adjust based on lessons learned
389+
390+
**Vertical Slice Definition**:
391+
- Pick ONE WAMP message type (e.g., PUBLISH)
392+
- Implement ENTIRE pipeline:
393+
- Test vector in wamp-proto repository ✅
394+
- Test framework in autobahn-python ✅
395+
- All three test dimensions ✅
396+
- Local pytest execution ✅
397+
- GitHub Actions CI/CD integration (next)
398+
- Get it working, polished, and reviewed
399+
- THEN scale to all other message types
400+
401+
### Phase 1: Foundation ✅ COMPLETED
402+
- [x] Performance benchmarking infrastructure (`examples/benchmarks/serialization/`)
381403
- [x] Flamegraph visualization
382404
- [x] Multi-serializer testing
383405
- [x] Normal vs Transparent mode benchmarks
384406

385-
### Phase 2: Single-Serializer Correctness ⚠️ **TODO**
386-
- [ ] Design test framework for roundtrip testing
387-
- [ ] Implement test generator for all WAMP message types
388-
- [ ] Create reference test data (manual + generated)
389-
- [ ] Test all serializers against reference data
390-
- [ ] Document failures and edge cases
391-
392-
### Phase 3: Cross-Serializer Correctness ⚠️ **TODO**
393-
- [ ] Cross-serializer comparison framework
394-
- [ ] Systematic testing across all serializer pairs
395-
- [ ] Identify and fix any attribute loss issues
396-
397-
### Phase 4: Protocol Test Vectors 🔮 **Future**
398-
- [ ] Design machine-readable test vector format
399-
- [ ] Extract examples from WAMP spec
400-
- [ ] Generate comprehensive test vector suite
401-
- [ ] Integrate into Autobahn|Python test suite
402-
- [ ] Contribute to wamp-proto repository
407+
### Phase 2: Vertical Slice - PUBLISH Message ✅ COMPLETED
408+
409+
**Rationale**: PUBLISH is a good starting point because:
410+
- Representative complexity (Options dict, args, kwargs)
411+
- Used in common workflows
412+
- Tests both basic and advanced features (publisher exclusion, etc.)
413+
414+
**Implementation**:
415+
- [x] Create test vector schema in wamp-proto (Issue #556, PR #557)
416+
- [x] Implement testsuite/singlemessage/basic/publish.json ✅
417+
- [x] Design test framework structure in autobahn-python ✅
418+
- [x] Implement examples/serdes/tests/ directory ✅
419+
- [x] conftest.py - pytest fixtures and configuration
420+
- [x] utils.py - helper functions (load_test_vector, validate_message, etc.)
421+
- [x] test_publish.py - complete test suite
422+
- [x] Implement all three test dimensions:
423+
- [x] Dimension 2: Single-serializer roundtrip correctness
424+
- [x] test_publish_deserialize_from_bytes
425+
- [x] test_publish_serialize_to_bytes
426+
- [x] test_publish_roundtrip
427+
- [x] Dimension 3: Cross-serializer preservation
428+
- [x] test_publish_cross_serializer_preservation
429+
- [x] Expected attributes validation
430+
- [x] test_publish_expected_attributes
431+
- [x] Validate locally with pytest (20/20 tests passing)
432+
- [x] Document design decisions and lessons learned
433+
434+
**Key Technical Discoveries**:
435+
1. **txaio initialization required**: Must call `txaio.use_asyncio()` before importing autobahn serializers
436+
2. **Autobahn serializer API**: `serialize()` returns `(bytes, is_binary)`, `unserialize()` returns `List[IMessage]`
437+
3. **Publish message structure**: No `options` attribute - individual option fields instead
438+
4. **Non-bijective serialization**: JSON has multiple valid representations (whitespace variations)
439+
5. **"At least one" semantics**: Critical for handling serialization variants
440+
441+
**Test Results**:
442+
- ✅ 20 tests passing (JSON, msgpack serializers)
443+
- ✅ All three dimensions validated
444+
- ✅ Template established for other message types
445+
446+
### Phase 3: Horizontal Scaling - All Basic Message Types ⚠️ **IN PROGRESS**
447+
448+
**Approach**: Now that the vertical slice is proven, scale horizontally using the established template.
449+
450+
**Priority Order** (by usage frequency and complexity):
451+
1. [x] PUBLISH (16) - ✅ Complete (vertical slice)
452+
2. [ ] EVENT (36) - Subscriber receives
453+
3. [ ] SUBSCRIBE (32) - Simple subscription
454+
4. [ ] SUBSCRIBED (33) - Subscription confirmed
455+
5. [ ] CALL (48) - RPC invocation
456+
6. [ ] RESULT (50) - RPC result
457+
7. [ ] REGISTER (64) - Procedure registration
458+
8. [ ] REGISTERED (65) - Registration confirmed
459+
9. [ ] INVOCATION (68) - Callee receives
460+
10. [ ] YIELD (70) - Callee responds
461+
11. [ ] HELLO (1) - Session establishment
462+
12. [ ] WELCOME (2) - Session accepted
463+
13. [ ] UNSUBSCRIBE (34), UNSUBSCRIBED (35)
464+
14. [ ] UNREGISTER (66), UNREGISTERED (67)
465+
15. [ ] GOODBYE (6) - Session close
466+
16. [ ] ABORT (3) - Session rejected
467+
17. [ ] ERROR (8) - Generic error (all contexts)
468+
18. [ ] PUBLISHED (17) - Acknowledgment (if requested)
469+
470+
**For Each Message Type**:
471+
1. Create test vector in wamp-proto: `testsuite/singlemessage/basic/<message>.json`
472+
2. Add tests in autobahn-python: `examples/serdes/tests/test_<message>.py`
473+
3. Run pytest locally
474+
4. Commit and push
475+
476+
### Phase 4: Advanced Features & Edge Cases ⚠️ **TODO**
477+
- [ ] Multiple samples per message type (args only, kwargs only, both, payload)
478+
- [ ] Advanced profile options:
479+
- [ ] Publisher exclusion/inclusion (PUBLISH options)
480+
- [ ] Pattern-based subscriptions (SUBSCRIBE match policies)
481+
- [ ] Shared registration (REGISTER invocation policies)
482+
- [ ] Progressive call results (CALL/RESULT)
483+
- [ ] Caller/publisher identification (Details)
484+
- [ ] Edge cases:
485+
- [ ] Empty args/kwargs
486+
- [ ] Null values
487+
- [ ] Binary data in payload
488+
- [ ] Deeply nested structures
489+
- [ ] Unicode handling
490+
- [ ] Large payloads
491+
492+
### Phase 5: Additional Serializers ⚠️ **TODO**
493+
- [ ] Add CBOR byte representations to test vectors
494+
- [ ] Add UBJSON byte representations to test vectors
495+
- [ ] Test flatbuffers (if applicable to test vector format)
496+
- [ ] Ensure all serializers pass all tests
497+
498+
### Phase 6: CI/CD Integration ⚠️ **TODO**
499+
- [ ] Add GitHub Actions workflow for serdes tests
500+
- [ ] Test on CPython 3.9, 3.10, 3.11, 3.12, 3.13
501+
- [ ] Test on PyPy 3.9, 3.10, 3.11
502+
- [ ] Test on multiple platforms (Linux, macOS, Windows)
503+
- [ ] Add coverage reporting
504+
- [ ] Badge in README
505+
506+
### Phase 7: Protocol Test Vectors Expansion 🔮 **Future**
507+
- [ ] Multi-message sequences (singlesession/)
508+
- [ ] Multi-session interactions (multisession/)
509+
- [ ] Router behavior validation (requires Crossbar.io)
510+
- [ ] Advanced feature semantics (publisher exclusion, pattern matching, etc.)
403511

404512
### Phase 5: Advanced Features ⚠️ **TODO**
405513
- [ ] Message batching tests

justfile

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -930,6 +930,22 @@ test-asyncio venv="" use_nvx="": (install-tools venv) (install venv)
930930
USE_ASYNCIO=1 ${VENV_PYTHON} -m pytest -s -v -rfP \
931931
--ignore=./autobahn/twisted ./autobahn
932932

933+
# Run WAMP message serdes conformance tests (usage: `just test-serdes cpy311`)
934+
test-serdes venv="": (install-tools venv) (install venv)
935+
#!/usr/bin/env bash
936+
set -e
937+
VENV_NAME="{{ venv }}"
938+
if [ -z "${VENV_NAME}" ]; then
939+
echo "==> No venv name specified. Auto-detecting from system Python..."
940+
VENV_NAME=$(just --quiet _get-system-venv-name)
941+
echo "==> Defaulting to venv: '${VENV_NAME}'"
942+
fi
943+
VENV_PYTHON=$(just --quiet _get-venv-python "${VENV_NAME}")
944+
945+
echo "==> Running WAMP message serdes conformance tests in ${VENV_NAME}..."
946+
echo "==> Test vectors loaded from: wamp-proto/testsuite/"
947+
${VENV_PYTHON} -m pytest -v examples/serdes/tests/test_publish.py
948+
933949
# -----------------------------------------------------------------------------
934950
# -- Documentation
935951
# -----------------------------------------------------------------------------

0 commit comments

Comments
 (0)