Skip to content

perf(rpc): cut output formatting time on large list responses#68

Open
mike1o1 wants to merge 2 commits intoash-project:mainfrom
mike1o1:improve-formatting-hotpath
Open

perf(rpc): cut output formatting time on large list responses#68
mike1o1 wants to merge 2 commits intoash-project:mainfrom
mike1o1:improve-formatting-hotpath

Conversation

@mike1o1
Copy link
Copy Markdown
Contributor

@mike1o1 mike1o1 commented Apr 21, 2026

I started using Ash Typescript on an internal project which returns a fairly large number of resources (around 1500) and noticed that there is a decent amount of overhead when it comes to serializing the responses. After some investigating, there were two areas that can be improved where we try and figure out how to format the same field multiple times in a single response. This PR attempts to improve that hotspot by memoizing the format of the field name (per process) so that multiple records only try and figure out how to format a field once, and also adds a quick exit for known Ash types, instead of having to perform some costly introspection on each field for each resource.

Summary

Two independent, measured perf improvements to the RPC output formatting hot path. Combined they drop OutputFormatter.format/4 on a 1500-record list from ~70 ms to ~4.6 ms in the included benchmark (~15× faster, ~10× less allocated memory). No behavior change; no public API change.

  • Added a guard clause to ValueFormatter.format/5 for Ash.Type.{String, UUID, UUIDv7, Boolean, Integer, Float, Decimal, Date, Time, DateTime, UtcDatetime, UtcDatetimeUsec, NaiveDatetime, Atom, Binary, CiString, Duration, Term}. These already land on the true -> value passthrough branch today, but only after running Introspection.unwrap_new_type/2 (which calls Spark.implements_behaviour?/2) plus eight cond branches. Values for these types are already JSON-normalized upstream in ResultProcessor.normalize_primitive, so the dispatch work is pure overhead.

  • Memoizes format_field_name/2 and format_field_for_client/3 on the process dictionary when inputs are (atom, module | nil, :camel_case | :snake_case | :pascal_case). A single list response hits the same (field, resource, formatter) triple ~once per record, so the cache resolves after the first call and stays warm for the duration of the request. Strings and custom {Mod, fun} formatters fall through unchanged - no unbounded cache growth and no assumptions about custom-formatter purity.

Why the process dictionary

  • No global state. Cache dies with the request process, so dev code reloads, tests, and module recompilation need no invalidation logic.
  • No memory growth across requests. Key cardinality is bounded by (fields × resources × formatters), typically a few hundred entries per process, regardless of record count.

Numbers

Micro (benchmarks/value_formatter.exs, per-call):

Case Before After
scalar/string 720 ns 8 ns
scalar/utc_datetime 4,479 ns 37 ns
scalar/uuid 727 ns 36 ns
composite/embedded 11,494 ns 7,607 ns (noise — unguarded)
composite/typed_struct 341 µs 364 µs (noise — unguarded)

Micro (benchmarks/field_formatter.exs, per-call):

Case Before After
format_field_for_client/resource_unmapped 2,000 ns 40 ns
format_field_name/atom 1,120 ns 67 ns
per_record_loop/todo_all_fields (21 fields) 34,230 ns 933 ns
format_field_name/string 1,050 ns 1,038 ns (unchanged — strings not cached)

End-to-end (benchmarks/output_formatter.exs, per call on 1500 records):

Step Time Allocated
main 70.08 ms 61.6 MB
+ guard clause 39.36 ms 18.5 MB
+ field name cache 4.60 ms 5.72 MB

Testing

  • mix test — 2219 tests, 0 failures, 1 skipped (unchanged from main).
  • Composite types (resources, embedded resources, typed structs, unions, tuples, keywords, maps with field constraints) are explicitly exercised by regression rows in benchmarks/value_formatter.exs and by the existing test suite.
  • Custom {Mod, fun} formatters tested - they fall through to the uncached path as designed.

Benchmarks

New benchmarks/ directory with three standalone scripts:

mix run benchmarks/value_formatter.exs
mix run benchmarks/field_formatter.exs
mix run benchmarks/output_formatter.exs

Adds :benchee ~> 1.3 as a dev/test-only dependency.

Contributor checklist

Leave anything that you believe does not apply unchecked.

  • I accept the AI Policy, or AI was not used in the creation of this PR.
  • Bug fixes include regression tests
  • Chores
  • Documentation changes
  • Features include unit/acceptance tests
  • Refactoring
  • Update dependencies

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant