feat: add Schema.NullableMode for explicit per-property nullable override (#5160)

[thejeff77]

Summary

Introduces Schema.NullableMode { AUTO, NULLABLE, NOT_NULLABLE } mirroring the established RequiredMode (#4221) and AccessMode (#2675) pattern, providing the missing explicit per-property override for nullable.

Closes #5160.

Motivation

The existing nullable() boolean defaults to false, which is indistinguishable from "not specified" via reflection. Users cannot opt a property out of nullable auto-detection (added in #5018 for @Nullable annotations, and used downstream by springdoc-openapi for Kotlin T? reflection in springdoc/springdoc-openapi#3256, since reverted in springdoc/springdoc-openapi#3276 pending this proposal).

This is the same problem class that prompted RequiredMode in 2022. From commit b1729fc: "so we can have a property annotated @NotNull but still have required = false in the openapi spec" — identical situation, just nullable instead of required.

Precedence

Mirrors PR #4533 ("give precedence to requiredMode annotation"):

1. nullableMode = NULLABLE or NOT_NULLABLE → use that
2. Legacy nullable = true (deprecated path) → treat as NULLABLE
3. nullableMode = AUTO (default) → run heuristics (@Nullable annotations from 5001: Add support for @Nullable annotations in OpenAPI 3.1 schemas #5018, downstream Kotlin reflection, etc.)

Changes

modules/swagger-annotations/src/main/java/io/swagger/v3/oas/annotations/media/Schema.java

• Add NullableMode enum (AUTO, NULLABLE, NOT_NULLABLE).
• Add nullableMode() annotation field defaulting to AUTO.
• Deprecate nullable() boolean pointing to nullableMode().

modules/swagger-core/src/main/java/io/swagger/v3/core/jackson/ModelResolver.java

• resolveNullable honors NullableMode precedence. When the @Schema annotation is not passed directly (typical for property-level resolution), it is extracted from the annotations array.
• Adds isSchemaAnnotationNullable helper for downstream "is this nullable" checks (used at the defaultValue="null" and example="null" literal handling sites, where the existing schema.nullable() checks now route through the helper to also honor NullableMode).

modules/swagger-core/src/main/java/io/swagger/v3/core/util/AnnotationsUtils.java

• Parity with RequiredMode handling in hasSchemaAnnotation (NullableMode != AUTO counts as "has annotation"), equals (compares nullableMode), and mergeSchemaAnnotations (master-wins precedence on nullableMode).
• Property-level @Schema processing (line ~842) honors NullableMode. When NOT_NULLABLE is set, explicitly clears any prior nullable indication on the schema object (e.g. null already added to the types array by upstream @Nullable auto-detection from 5001: Add support for @Nullable annotations in OpenAPI 3.1 schemas #5018).

modules/swagger-core/src/test/java/io/swagger/v3/core/util/AnnotationsUtilsTest.java

• One existing anonymous Schema implementation needed the new nullableMode() method override (delegating to the wrapped schema, mirroring the existing requiredMode() delegation).

Tests

modules/swagger-core/src/test/java/io/swagger/v3/core/issues/Issue5160Test.java — 9 new tests covering all combinations across OAS 3.0 and OAS 3.1:

• testModeNullableForcesNullable{OAS30,OAS31} — nullableMode=NULLABLE produces nullable output.
• testModeNotNullableOverridesNullableAnnotation{OAS30,OAS31} — nullableMode=NOT_NULLABLE overrides upstream @Nullable auto-detection.
• testModeAutoWithAnnotationStillDetectsNullable{OAS30,OAS31} — AUTO (default) lets @Nullable detection apply unchanged.
• testModeAutoWithoutAnyNullableSignalIsNotNullableOAS30 — AUTO with no signals is not nullable (no regression).
• testLegacyNullableTrueStillWorks{OAS30,OAS31} — backward compatibility for nullable = true.

Full existing suite passes (660 tests, 0 failures, 0 errors).

OAS 3.0 vs 3.1 mapping

Same as the existing handling for nullable: true:

• OAS 3.0: sets nullable: true on the schema object.
• OAS 3.1: adds "null" to the type array.
• NOT_NULLABLE suppresses any of the above and clears any prior nullable indication.

Impact on downstream

• @Nullable auto-detection added in 5001: Add support for @Nullable annotations in OpenAPI 3.1 schemas #5018 continues to work unchanged when nullableMode = AUTO.
• Provides the override mechanism springdoc-openapi needs to safely re-introduce its Kotlin T? auto-detection (revert: auto-set nullable for Kotlin nullable types (#3256) springdoc/springdoc-openapi#3276 reverted that work pending this proposal).
• nullable() boolean is now @Deprecated but still honored, mirroring the deprecation pattern used for required/readOnly/writeOnly.

Test plan

• CI passes
• Local: ./mvnw -pl modules/swagger-core test -Dmaven.javadoc.skip=true → 660/660 tests pass including 9 new
• Manual verification of OAS 3.0 / 3.1 output for all five test cases via the snapshot dump in Issue5160Test

[Mattias-Sehlstedt]

As far as I am aware the general idea is that an Annotation[] should never have any @Schema annotation considered, that is explicitly what the separate schema field is for.

This is especially of importance since sibling handling is managed with the annotation passed in schema.