v4.88.0

For installation instructions check out the getting started guide.

Added

• mysql_cdc: Added integration tests validating MySQL CDC connection drop and reconnection scenarios with zero data loss. (@mmatczuk, #4241)
• oracledb_cdc: Added multi-tenant support for Oracle CDC input to handle both container and non-container databases. (@josephwoodward, #4237)
• oracledb_cdc: Added source timestamp metadata to Oracle CDC messages to propagate the redo log timestamp to consumers. (@josephwoodward, #4250)
• otlp: Added OpenTelemetry metrics exporter supporting both gRPC and HTTP protocols. (@mmatczuk, #4230)
• arc: add output plugin for Arc columnar database (@xe-nvdk, #4265)

Fixed

• amqp1: Fixed data race in randomString by replacing non-goroutine-safe package-level rand with thread-safe top-level rand.Intn. (@mmatczuk, #4260)
• general: Fixed Docker latest tag for connect image from incorrect latest-cloud to latest. (@mmatczuk, #4267)
• mysql_cdc: Fixed snapshot column order consistency by using deterministic primary key slice iteration instead of non-deterministic map iteration. (@josephwoodward, #4262)
• postgresql: Fixed handling of tsvector type in PostgreSQL replication decoder to return raw PostgreSQL text representation. (@mmatczuk, #4261)
• protobuf: Fixed memory leak in hyperpb parser caused by pool holding unused messages and added profile-guided optimization support. (@mmatczuk, #4240)

Changed

• iceberg: Updated Iceberg fallback behavior to infer column type when schema doesn't contain the field, enabling support for transforms. (@rockwotj, #4263)
• oracledb_cdc: Switched from buffering all redo events to streaming them through a callback to reduce memory allocation. (@josephwoodward, #4243)
• mysql_cdc: write checkpoint after snapshot (@josephwoodward, #4269)

The full change log can be found here.

v4.88.0-rc3

For installation instructions check out the getting started guide.

Added

oracledb_cdc: add source_ts_ms metadata - (@josephwoodward, #4250)

The full change log can be found here.

v4.88.0-rc2

For installation instructions check out the getting started guide.

Added

• oracledb_cdc: Add multi-tenancy support (@josephwoodward, #4237)

v4.88.0-rc1

For installation instructions check out the getting started guide.

Added

• oracledb_cdc: Add multi-tenancy support (@josephwoodward, #4237)

v4.87.0

For installation instructions check out the getting started guide.

Added

• amqp_1: AMQP 1.0 input now extracts all standard message properties as metadata, and output supports setting them via interpolated string config fields with UUID/uint64/string auto-detection. (@mmatczuk, #4225)
• general: Added cmd/tools/integration, a CLI tool for running integration tests package-by-package with caching, resume capability, and detailed failure output. (@mmatczuk, #4198)
• mssqlserver_cdc: Added support for remote checkpoint caching in MSSQL Server CDC input via new checkpoint_cache_connection_string configuration field. (@josephwoodward, #4214)
• oracledb_cdc: Added Oracle Wallet authentication support to oracledb_cdc input via wallet_path and wallet_password configuration fields. (@josephwoodward, #4219)
• sql: Added batching support to sql_raw output with per-message conditional query execution via new when field. (@squiidz, #4200)

Fixed

• oracledb_cdc: Fixed Oracle CDC LogMiner handling of zero-padded HEXTORAW values and corrected DELETE/UPDATE operations to include proper message content and primary keys. (@josephwoodward, #4217)

Changed

• avro: Replaced linkedin/goavro and hamba/avro libraries with twmb/avro across confluent, avro, and salesforce packages, simplifying type handling and enabling support for RFC 3339 timestamp strings in JSON input. (@twmb, #4195)
• iceberg: Apache Iceberg table output now automatically upgrades v1 tables to v2 format during commit transactions. (@rockwotj, #4212)

The full change log can be found here.

v4.86.0

For installation instructions check out the getting started guide.

Added

• oracledb_cdc: Added support for parsing complex Oracle connection configurations with override options and early connection validation. (@josephwoodward, #4179)
• text: Added a new string_split processor that can be used instead of bloblang's split method in certain high-performance scenarios. (@rockwotj, #4187)
• sql: Add ClickHouse Map type support for sql_insert (@rickysaltzer, #4178)

Fixed

• aws: Fixed DynamoDB CDC shard discovery race condition that could miss writes between Connect() and async discovery, and added handling for TrimmedDataAccessException errors. (@squiidz, #4175)
• migrator: Fixed concurrent DFS race condition that could cause import mode to be restored while other walkers were still syncing by deferring all mode restoration to Close() and removing the refcount mechanism. (@mmatczuk, #4174)
• s3: Fixed AWS S3 client to properly respect CA certificates specified in the AWS_CA_BUNDLE environment variable. (@ttreptow, #4093)

Changed

• aws_sqs: Reverted client-side 256 KB message and batch size limit enforcement for SQS output to restore prior behavior. (@twmb, #4193)
• general: Promoted 50 Beta and Experimental connectors to Stable status across inputs, outputs, processors, caches, metrics, and Bloblang functions. (@prakhargarg105, #4205)

The full change log can be found here.

v4.85.0

For installation instructions check out the getting started guide.

Added

• iceberg: Added three-stage type resolution pipeline for Iceberg schema evolution with optional schema metadata override and Bloblang type mapping. (@rockwotj, #4167)
• redpanda: Added tracing span mapping support to redpanda input and output for trace context propagation across Kafka-connected pipelines. (@mmatczuk, #4146)
• salesforce: Added Salesforce processor implementation with REST client, GraphQL support, real-time streaming, and bulk operations for reading and writing data. (@ness-david-dedu, #4151)

Fixed

• aws: Fixed send-on-closed-channel panic in dynamodb_cdc by ensuring shard reader goroutines complete before closing message channel. (@squiidz, #4168)
• redpanda: Fixed error handling in redpanda output when unable to create franz writer to prevent nil pointer dereference. (@josephwoodward, #4149)

The full change log can be found here.

v4.85.0-rc1

For installation instructions check out the getting started guide.

Added

• oracledb_cdc: Adds support for streaming LOB columns (@josephwoodward)

Changed

• schema_registry_encode: Avro encoding now handles timestamps from CDC sources (RFC3339 strings and time.Time values) automatically, nullable union fields are auto-wrapped regardless of avro.raw_json, and extra fields not in the schema are silently dropped rather than producing an error. (@Jeffail)

Fix

• dynamodb_cdc: Fix shard readers polling too slowly. (@squiidz)

The full change log can be found here.

v4.84.1

For installation instructions check out the getting started guide.

Added

• oracledb_cdc: Adds support for streaming LOB columns (@josephwoodward)

Changed

• schema_registry_encode: Avro encoding now handles timestamps from CDC sources (RFC3339 strings and time.Time values) automatically, nullable union fields are auto-wrapped regardless of avro.raw_json, and extra fields not in the schema are silently dropped rather than producing an error. (@Jeffail)

Fix

• dynamodb_cdc: Fix shard readers polling too slowly. (@squiidz)

The full change log can be found here.

v4.84.0

For installation instructions check out the getting started guide.

Added

• oracledb_cdc: Input now adds schema metadata to consumed messages. Schema is fetched from Oracle's ALL_TAB_COLUMNS catalog with precision-aware NUMBER mapping. Column additions are detected automatically via addition-only drift detection; dropped columns are reflected after a connector restart. This can be used for automatic schema registration in processors such as schema_registry_encode. (@Jeffail)
• iceberg: Allow specifying aws credentials explicitly for sigv4 auth with glue. (@rockwotj)
• redis_streams: Add interpolation support for entry ID. (@twmb)
• nats: Add user/password and token authentication. (@ghstahl)

Fixed

• oracledb_cdc: Fixed snapshot/streaming value type inconsistency where NUMBER columns produced json.Number during snapshot but plain strings during streaming. Bare numeric literals in SQL_REDO are now converted to int64 (for integers that fit) or json.Number (for decimals), matching the snapshot path. Quoted string values from VARCHAR columns are no longer incorrectly converted. (@Jeffail)
• oracledb_cdc: Reduce the number of log files loaded into LogMiner to those only containing SCN range. (@josephwoodward)
• iceberg: Fix credential renewal for vendored credentials as well as oauth2 authentication with the catalog. (@rockwotj)
• iceberg: Remove usage of a disallowed table property for Databricks Unity Catalog. (@rockwotj)

Changed

• aws_sqs: Enforce 256 KB message and batch size limits. (@twmb)
• nats: Use JetStream package. (@nickchomey)

The full change log can be found here.