fix(bulk-cdk): handle null generationId/syncId in DestinationStreamFactory (AI-Triage PR)#75211
fix(bulk-cdk): handle null generationId/syncId in DestinationStreamFactory (AI-Triage PR)#75211devin-ai-integration[bot] wants to merge 3 commits intomasterfrom
Conversation
…ctory to prevent NPE When ConfiguredAirbyteStream has null generationId, minimumGenerationId, or syncId (which occurs on certain OSS platform versions or when using custom connector definitions), the DestinationStreamFactory crashes with a NullPointerException during Kotlin auto-unboxing from Long? to Long. This adds null-safe defaults (0L) in both the modern and legacy-task-loader versions of DestinationStreamFactory, matching the default behavior in CatalogGenerationSetter which also defaults to 0L. Resolves airbytehq/oncall#11702 Related: #75210 Related: #69351 Related: #70197 Related: #69219 Co-Authored-By: bot_apk <apk@cognition.ai>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. 💡 Show Tips and TricksPR Slash CommandsAirbyte Maintainers (that's you!) can execute the following slash commands on your PR:
📚 Show Repo GuidanceHelpful Resources
|
Co-Authored-By: bot_apk <apk@cognition.ai>
Co-Authored-By: bot_apk <apk@cognition.ai>
|
Deploy preview for airbyte-kotlin-cdk ready! ✅ Preview Built with commit afcba6a. |
|
↪️ Triggering Reason: Draft PR with Green 4/5 triage score and CI mostly passing (47 passed, 2 failed). Fixes NPE in DestinationStreamFactory for null generationId/syncId affecting all bulk-CDK destinations. |
|
Fix Validation EvidenceOutcome: Could not Run Tests Evidence SummaryPre-release publish of Additionally, this bug only manifests on self-managed/OSS deployments where the platform does not populate Code analysis confirms the fix is correct and minimal: adding Next Steps
Connector & PR DetailsConnector: Evidence PlanProving CriteriaA sync that previously failed with NPE in Disproving CriteriaThe same NPE still occurs after applying the fix, or new errors appear that were not present before. Cases Attempted
LimitationThis NPE only occurs on OSS/self-managed deployments where the platform does not populate Pre-flight Checks
Detailed Evidence Log
Note: Connection IDs and detailed logs are recorded in the linked private issue. Validation by Clumsy Tucker Devin · Triggered by |
|
What
Fixes a
NullPointerExceptioninDestinationStreamFactory.make()that crashes all bulk-CDK destinations during initialization whengenerationId,minimumGenerationId, orsyncIdare null on the incomingConfiguredAirbyteStream.This affects multiple destinations (S3, ClickHouse, HubSpot, and others) on certain OSS platform versions or when using custom connector definitions where the platform does not populate these fields.
Resolves https://github.com/airbytehq/oncall/issues/11702:
Related issues:
How
The
ConfiguredAirbyteStream(fromio.airbyte.protocol.models.v0) is a Java class wheregenerationId,minimumGenerationId, andsyncIdare boxedLong— nullable at runtime. TheDestinationStreamdata class expects non-nullableLong. When the platform doesn't populate these fields, Kotlin's auto-unboxing throws an NPE.The fix adds
?: 0Lnull-coalescing in both the modern and legacy-task-loaderDestinationStreamFactory.make()methods. The default of0Lis consistent withCatalogGenerationSetterin the platform, which also defaults generation IDs to0.Note: This is a defensive CDK-side fix. The underlying platform issue (not populating these fields in certain OSS/custom-connector scenarios) may warrant a separate investigation.
Review guide
airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/command/DestinationStreamFactory.kt— modern path (lines 56-58)airbyte-cdk/bulk/toolkits/legacy-task-loader/src/main/kotlin/io/airbyte/cdk/load/command/DestinationStream.kt— legacy path (lines 204-206)airbyte-cdk/bulk/core/load/version.properties— version bump 1.0.6 → 1.0.7airbyte-cdk/bulk/core/load/changelog.md— changelog entry for 1.0.7Key question for reviewers: Is
0Lthe correct default? WithgenerationId=0andminimumGenerationId=0,shouldBeTruncatedAtEndOfSync()returnsfalse(safe — no accidental data deletion). Should a warning be logged when falling back to defaults?Human review checklist
0Lis safe across all code paths that readgenerationId/syncId/minimumGenerationId— not justshouldBeTruncatedAtEndOfSync(). Grep for usages in the bulk CDK and destination connectors.Updates since last revision
bulk-cdk-core-loadversion from 1.0.6 → 1.0.7 (required by CIcheckLoadVersiongate).User Impact
Destinations using the bulk CDK will no longer crash with "Failed to initialize connector operation" / NPE when the platform does not provide
generationId/syncId. Syncs will proceed normally with default values.Can this PR be safely reverted and rolled back?
Link to Devin session: https://app.devin.ai/sessions/7268e4116a494cd3a87597f78b4d8b70