Replace upickle with circe

[korlowski]

#474

Some caveats:

• Configured derivation in circe needs to be implemented separately for scala 2 (shapeless-backed circe-generic-extras) and scala 3 (sits in circe-core). That's also why codecs are not stored in companion objects but implemented in dedicated files for scala 2 and scala 3
• I've renamed some models so that configured derivation can pick the right type, reducing manual boilerplate but breaking backwards compatibility

[adamw]

Review: Replace upickle with circe

Overall: This is an unusually careful migration. The two biggest risks of such a swap are handled well: the None→null regression is avoided by deepDropNullValues on every request body, and the nested/flat/string discriminator shapes are faithfully reproduced with custom codecs (deriveAdtEncoder, tagged(...), manual string codecs) rather than blindly trusting circe's flat discriminator. Snake_case conversion was checked field-by-field with no edge-case mismatches. No high-severity correctness bug found. The items below are behavioral edge cases and cleanup.

Findings (most-severe first)

1. openai/.../json/OpenAIJson.scala (asJson) + SchemaSupport.scala — deepDropNullValues recurses into user-supplied JSON, stripping legitimate nulls. Request bodies are serialized as b.asJson.deepDropNullValues.noSpaces, which recurses into embedded free-form JSON (a tool's parameters, a json_schema's inner schema). A caller who hand-builds a schema containing a meaningful null — {"default": null}, {"const": null}, {"enum": [null]} — has it silently removed before the request is sent, changing the schema the API sees. uPickle preserved null inside Map[String, Value]. Low likelihood with tapir-generated schemas, real with hand-built ones — worth a conscious sign-off.

2. claude/.../json/ClaudeDerivedCodecs.scala (scala-2 ~L309, scala-3 ~L368) — emptyIterableAsNone/emptyMapAsNone imported at top level apply globally on decode. Every Option[List]/Option[Map] Claude response field now decodes an explicit empty []/{} to None instead of Some(Nil). Downstream code that distinguishes "present but empty" from "absent" (e.g. on citations, required, enum) sees None for an explicitly-empty array. Behavioral, and broader than per-field would be.

3. project/Dependencies.scala (L33–34) — two dead JSON-integration artifacts in sttpClient. The "com.softwaremill.sttp.client4" %%% "upickle" artifact is kept although its only consumer (SttpUpickleApiExtension.scala, now removed) is gone, and a new "circe" artifact is added even though OpenAIJson.asJson hand-rolls the body via StringBody + io.circe.syntax (no sttp.client4.circe import exists). Both can be removed.

4. Public-API breaking renames under-documented for release notes. The openai/.../requests/assistants/Tool.scala renames (FunctionTool→Function, CodeInterpreterTool→CodeInterpreter, FileSearchTool→FileSearch) are source-breaking but, unlike the ContentBlock.* renames, are not in the README diff. Also worth flagging the swap in claude/.../models/ContentBlock.scala where WebSearchToolResultContent→WebSearchToolResult while the inner trait WebSearchToolResult→WebSearchToolResultBlock. Wire formats are correct; this is a changelog/migration-note gap.

5. claude/.../ClaudeClient.scala (L423) — unused imports Json, JsonObject. import io.circe.{Decoder, Json, JsonObject} — only Decoder is used.

6. openai/src/main/scala-3/.../OpenAIDerivedCodecs.scala (L4042 & L4050) — import io.circe.syntax.* duplicated at both file and object scope (the Scala-2 sibling imports it once, targeted). Remove one.

7. README.md (L42 & L193) — duplicated word "a a circe" introduced in both the Claude createMessageAs[T] and OpenAI createChatCompletionAs[T] sections.

8. OpenAIDerivedCodecs.scala — omitFalse comment is factually wrong + asymmetric. The comment claims it "matches the previous uPickle default-omission," but uPickle's OptionWriter mapped Some(false)→false (it did not omit it). No API break (OpenAI treats absent strict as false), but the rationale will mislead, and omitFalse is wired only for the chat MsgTool, not the structurally-identical Tool.Function used elsewhere — worth confirming the asymmetry is intentional.

Findings 1 and 2 are the two behavioral changes worth consciously accepting; 3–8 are cleanup and documentation.

_{🤖 Review generated with Claude Code}

[korlowski]

@adamw I think the stripping of nulls in the json schemas / parameters is okay and won't affect anyone unless they write an atypical schema by hand

[adamw]

@korlowski yeah that's what you'd think, but sometimes servers are sensitive to certain fields being present (even if they have a null value). So in some cases it makes a difference - but we can see if it's a real problem on specific usages

[adamw]

Merging & releasing, thanks! :)