timestamp_format is ignored

[davidjmemmett]

Describe the bug
Given standard high-precision syslog format and config: "timestamp_format": "%Y-%m-%dT%H:%M:%S.%f+00:00" and "trim_timestamp": true, the timestamp in CloudWatch Logs differs from the timestamp in the log message, and the timestamp isn't trimmed.

Steps to reproduce

                    {
                        "file_path": "/var/log/syslog",
                        "log_group_class": "STANDARD",
                        "log_group_name": "syslog",
                        "log_stream_name": "{instance_id}",
                        "retention_in_days": 30,
                        "timestamp_format": "%Y-%m-%dT%H:%M:%S.%f+00:00",
                        "trim_timestamp": true
                    },

What did you expect to see?
No timestamp in the Message field and the correct timestamp in the Timestamp field

What did you see instead?
The correct timestamp in the Message field, and an incorrect timestamp in the Timestamp field

What version did you use?
CWAgent/1.300064.0b1337 (go1.25.5; linux; arm64)

What config did you use?
As above

Environment
Ubuntu 24.04.2 LTS

Additional context

[davidjmemmett]

To confirm, I ensured that the time format is correct, as per the docs and the standard strptime/strftime functions:

>>> datetime.strptime("2026-02-04T11:27:44.945314+00:00", "%Y-%m-%dT%H:%M:%S.%f+00:00")
datetime.datetime(2026, 2, 4, 11, 27, 44, 945314)

[davidjmemmett]

The generated TOML config file contains the following lines, which appears to miss the dot before the high-precision timestamp in the timestamp_regex:

      timestamp_layout = ["2006-01-_2T15:04:05.000+00:00", "2006-1-_2T15:04:05.000+00:00"]
      timestamp_regex = "(\\d{4}-\\s{0,1}\\d{1,2}-\\s{0,1}\\d{1,2}T\\d{2}:\\d{2}:\\d{2}(\\d{1,9})+00:00)"

[davidjmemmett]

After some messing about, I've tweaked the expression in RegEx101 in order to make it match: https://regex101.com/r/QI1Xe8/1

The new expression: (\d{4}-\s{0,1}\d{1,2}-\s{0,1}\d{1,2}T\d{2}:\d{2}:\d{2}\.(\d{1,9})\+00:00)

So I believe the original expression is missing the \. between the seconds and the microseconds, as well as escaping on the \+ before the hard-coded timezone.

[varunch77]

Hi David, thanks for deep diving on this issue and bringing this to our attention. I will bring this up to my team for further discussion.

[jefchien]

Tested the configuration and it gets generated with the escaped .. The issue is that the agent doesn't escape the +.

      timestamp_layout = ["2006-01-_2T15:04:05..000+00:00", "2006-1-_2T15:04:05..000+00:00"]
      timestamp_regex = "(\\d{4}-\\s{0,1}\\d{1,2}-\\s{0,1}\\d{1,2}T\\d{2}:\\d{2}:\\d{2}\\.(\\d{1,9})+00:00)"

[davidjmemmett]

Hi @jefchien @varunch77 ,
I've got AWS CloudWatch Agent installed via a AWS EC2 ImageBuilder recipe - is there an easy way for me to update it to an alpha build containing this fix to verify?
Cheers,
David

[davidjmemmett]

Any progress @jefchien @varunch77 ?