Skip to content

logging: Avoid high-cardinality keys for structured log messages #8571

@mcpherrinm

Description

@mcpherrinm

While high cardinality values are fine (and expected, even), we don't want the JSON keys to be high-cardinality.

The OpenTelemetry Collector stores keys in a LogAttributesKeys Array(LowCardinality(String)) column. Clickhouse's performance guides suggests that no more than 10,000 unique values should go in a LowCardinality(...).

There's only one case I've seen so far, which is

type certificateRequestEvent struct {
        ...

	// Authorizations is a map of identifier names to certificateRequestAuthz
	// objects. It can be used to understand how the names in a certificate
	// request were authorized.
	Authorizations map[string]certificateRequestAuthz
}

This map ends up with high-cardinality keys like Authorizations.thisdomainnameis.morethan64characterslong.forthesakeoftesting.c84281.com.ChallengeType

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions