Skip to content

fix: handle bare container IDs from CRI client in matchContainerID#3157

Merged
brancz merged 2 commits intoparca-dev:mainfrom
michaelfindlater:fix/cri-container-id-prefix
Mar 16, 2026
Merged

fix: handle bare container IDs from CRI client in matchContainerID#3157
brancz merged 2 commits intoparca-dev:mainfrom
michaelfindlater:fix/cri-container-id-prefix

Conversation

@michaelfindlater
Copy link
Copy Markdown
Contributor

Summary

  • Make containerIDPattern regex accept both prefixed (containerd://hex) and bare (hex) 64-char container IDs
  • Downgrade log.Error to log.Debugf in getKubernetesPodMetadata to match the level used in addPodContainerLabels for the same error
  • Add unit tests for matchContainerID covering both formats and edge cases

Problem

The CRI client fast path (cri_client.go:convertSandboxToPod) sets ContainerStatus.ContainerID to the bare runtime ID from the CRI API (container.Id), which is a 64-char hex string without the containerd:// prefix. The kubelet normally adds this prefix when constructing ContainerStatus for the Kubernetes API, but the CRI client bypasses the kubelet.

The containerIDPattern regex (.+://([0-9a-f]{64})) requires the :// prefix, so every CRI-sourced container lookup fails. Combined with log.Error level logging, this generates N_processes * M_containers error logs per profiling cycle - over 10,000 logs/hour across our fleet of 134 agents.

Fix

Commit 1 - Regex: .+://([0-9a-f]{64}) to ^(?:.+://)?([0-9a-f]{64})$

  • ^...$ anchors prevent substring matching (stricter than original)
  • (?:.+://)? makes the prefix optional
  • Handles both Kubernetes API format and CRI API format

Commit 2 - Log level: log.Error to log.Debugf in getKubernetesPodMetadata

  • The function does a linear scan over all containers to find the one matching a PID
  • Non-matching containers are expected, not errors
  • Matches the level already used in addPodContainerLabels for the same error

Test plan

  • Unit tests for matchContainerID covering prefixed IDs, bare IDs, and invalid inputs (empty, short, non-hex, 65 chars, cgroup paths, trailing chars)
  • Regex verified against live GKE 1.32 cluster ContainerID values

Fixes #3156

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 16, 2026

CLA assistant check
All committers have signed the CLA.

michaelfindlater and others added 2 commits March 16, 2026 14:21
The CRI client fast path (cri_client.go:convertSandboxToPod) sets
ContainerStatus.ContainerID to the bare runtime ID from the CRI API,
which is a 64-char hex string without the "containerd://" prefix that
the kubelet adds for the Kubernetes API.

The containerIDPattern regex required this prefix, causing every
CRI-sourced container lookup to fail.

Change regex from `.+://([0-9a-f]{64})` to `^(?:.+://)?([0-9a-f]{64})$`:
- Make the runtime:// prefix optional
- Add ^ and $ anchors (stricter than original - no substring matching)
- Handles both Kubernetes API format and CRI API format

Fixes parca-dev#3156

Co-Authored-By: Claude Opus 4.6 <[email protected]>
The getKubernetesPodMetadata function iterates all containers on a
node to find the one matching a profiled PID. Non-matching containers
are expected - logging each at Error level produces N_processes *
M_containers error logs per profiling cycle.

Downgrade to Debugf to match the level already used in
addPodContainerLabels for the same matchContainerID error.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@michaelfindlater michaelfindlater force-pushed the fix/cri-container-id-prefix branch from 2e576a1 to d8f6e6b Compare March 16, 2026 05:22
Copy link
Copy Markdown
Member

@brancz brancz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great find! Thank you! lgtm

@brancz brancz merged commit 515e856 into parca-dev:main Mar 16, 2026
7 checks passed
@michaelfindlater michaelfindlater deleted the fix/cri-container-id-prefix branch March 16, 2026 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CRI client sets ContainerID without runtime prefix, breaking matchContainerID regex

3 participants