We seem to call send_feedback imemdiately after self.writer.put_message
https://github.com/surbas/pg2kinesis/blob/master/pg2kinesis/__main__.py#L103
However put_message aggregates records into batches. This means that we might lose access to LSNs that have not been written to Kinesis yet if we crash before writing or during writing the batch.
I think we should only send feedback after the entire batch has been written?
We seem to call
send_feedbackimemdiately afterself.writer.put_messagehttps://github.com/surbas/pg2kinesis/blob/master/pg2kinesis/__main__.py#L103
However
put_messageaggregates records into batches. This means that we might lose access to LSNs that have not been written to Kinesis yet if we crash before writing or during writing the batch.I think we should only send feedback after the entire batch has been written?