Skip to content

Aren't writes acknowledged too early due to the batching mechanism? #24

@arianvp

Description

@arianvp

We seem to call send_feedback imemdiately after self.writer.put_message

https://github.com/surbas/pg2kinesis/blob/master/pg2kinesis/__main__.py#L103

However put_message aggregates records into batches. This means that we might lose access to LSNs that have not been written to Kinesis yet if we crash before writing or during writing the batch.

I think we should only send feedback after the entire batch has been written?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions