Retention job fails with "can not get valid member entity" on proxy cache project (Docker Hub)

[zhangxiaopeng-zgci]

Update: Corrected root cause (2026-07-24)

After deeper investigation, the retention failure and the can not get valid member entity errors are two separate issues.

---

Issue 1: Retention failure — missing required parameter nDaysSinceLastPull

Actual error (from jobservice logs)

2026-07-24T05:50:19Z [ERROR] [/pkg/retention/job.go:246]: retention job: missing required parameter nDaysSinceLastPull for rule nDaysSinceLastPull
2026-07-24T05:50:19Z [ERROR] [/jobservice/runner/redis.go:123]: Job RETENTION exit with error: run error: retention job: missing required parameter nDaysSinceLastPull for rule nDaysSinceLastPull

Steps to reproduce

1. Create a proxy cache project targeting Docker Hub
2. Create a retention policy with template nDaysSinceLastPull, n_days_since_last_pull: 30
3. Delete the retention policy and recreate it with identical configuration via API
4. Trigger retention execution (even dry-run)

Key observations

• The retention_policy.data JSON in the database is identical to working policies on other projects
• Only affects the recreated policy — policies that were never deleted work fine
• The error is missing required parameter nDaysSinceLastPull even though the parameter exists in the DB JSON
• This suggests a deserialization bug in the jobservice when loading a recreated retention policy

Environment

• Harbor version: v2.15.2
• Deployment: HA (2 nodes), external PostgreSQL 15, external Redis 7
• Project type: Proxy Cache (Docker Hub)

---

Issue 2: can not get valid member entity errors — external script, NOT a Harbor bug

Root cause

An external script (User-Agent: curl/8.5.0, source IP: HAProxy LB) is sending POST /api/v2.0/projects/{id}/members requests with an empty body to Harbor at 03:00 UTC daily. This causes the can not get valid member entity error with ProjectID:0, UserID:0.

Evidence from Harbor proxy (nginx) access log

10.1.132.41 - POST /api/v2.0/projects/16/members HTTP/1.1 500 66 curl/8.5.0
10.1.132.41 - POST /api/v2.0/projects/17/members HTTP/1.1 500 66 curl/8.5.0
10.1.132.41 - GET /api/v2.0/usergroups HTTP/1.1 200 379 curl/8.5.0

These requests occur at 03:00 UTC — a different time than the retention job (00:00 UTC). The two issues are unrelated.

Reporter: zhangxiaopeng@zgci.ac.cn

[zhangxiaopeng-zgci]

Root Cause Update (2026-07-24)

After deeper investigation, the original root cause description was incorrect. Here's the corrected analysis:

1. Retention Failure — Real Root Cause: Parameter Parsing Bug

The retention execution failure is NOT caused by a permission/member entity issue. The actual error from jobservice is:

missing required parameter nDaysSinceLastPull

Key findings:

• The retention_policy.data JSON in the database is identical to policies that work correctly (e.g., policy ID 1 for other proxy cache projects like quay/ghcr/k8s)
• Despite identical DB data, jobservice cannot extract the nDaysSinceLastPull parameter during execution
• This only affects the dockerhub-proxy project (project ID 13); other proxy cache projects with the same policy structure work fine
• The policy was recreated, and while the DB data matches, the jobservice still fails to parse the parameter

This appears to be a parameter parsing/serialization bug in Harbor v2.15.2's jobservice — the retention job cannot correctly deserialize the nDaysSinceLastPull parameter from the policy data during execution.

2. "can not get valid member entity" — Unrelated

The can not get valid member entity error (ProjectID:0, UserID:0) is a separate, unrelated issue. It is caused by an external script sending HTTP POST requests with an empty body to the Harbor member API (POST /api/v2.0/projects/{project}/members). The timing (03:00 UTC) does not coincide with retention execution (00:00 UTC), confirming no causal relationship.

Reproduction

1. Create a proxy cache project (e.g., dockerhub-proxy)
2. Harbor auto-creates a 7-day retention policy with nDaysSinceLastPull rule
3. Wait for scheduled retention execution or trigger manually
4. All tasks fail with "missing required parameter nDaysSinceLastPull"

Environment

• Harbor v2.15.2
• Docker Compose deployment
• PostgreSQL database

[zhangxiaopeng-zgci]

Root cause found and fixed ✅

Root cause: Parameter name casing mismatch between API and jobservice.

When a retention policy is created via the REST API (POST /api/v2.0/retentions), the params field uses snake_case keys (e.g., n_days_since_last_pull), which is the format stored in the database retention_policy.data JSON.

However, when jobservice executes the retention job, it checks for parameters using camelCase keys defined in the rule's Go source code:

// src/pkg/retention/policy/rule/dayspl/evaluator.go
const (
    TemplateID = "nDaysSinceLastPull"
    ParameterN = TemplateID  // "nDaysSinceLastPull" (camelCase)
)

// src/pkg/retention/policy/rule/index/index.go - Get()
if p.Required {
    _, exists = parameters[p.Name]  // p.Name = "nDaysSinceLastPull"
    if !exists {
        return nil, errors.Errorf("missing required parameter %s for rule %s", p.Name, templateID)
    }
}

The internal WithNDaysSinceLastPull() function creates policies with camelCase keys ("nDaysSinceLastPull": n), so policies created programmatically work fine. But policies created via the REST API store snake_case keys ("n_days_since_last_pull": 30), causing the mismatch.

Workaround: Directly update the retention_policy.data column in the database to use camelCase keys:

UPDATE retention_policy 
SET data = replace(data, '\"n_days_since_last_pull\"', '\"nDaysSinceLastPull\"') 
WHERE id = <policy_id>;

Proper fix needed: The API layer should convert snake_case parameter names to camelCase before storing in the database, or the rule evaluator should accept both casing variants.

This issue is likely reproducible with other rule templates that use camelCase parameter names (e.g., latestPushedK, latestPulledN, etc.).

[jUDASmILE]

@zhangxiaopeng-zgci please file a separate issue for issue #2. we use this issue to track the first issue. thanks.

[stonezdj]

The can not get valid member entity error happen in member.go it is used to maintain the project member info.
Retention job failed bcause of missing required parameter nDaysSinceLastPull, they have no relationship.

[zhangxiaopeng-zgci]

@jUDASmILE Thanks for the suggestion. After deeper investigation, the can not get valid member entity error was caused by an external script sending empty-body POST requests to the Harbor member API (POST /api/v2.0/projects/{project}/members) — not a Harbor bug. The timing (03:00 UTC) did not coincide with retention execution (00:00 UTC), confirming no causal relationship.

So there is no need for a separate issue. #23608 will track only the missing required parameter nDaysSinceLastPull problem.

@stonezdj Thanks for confirming! Your analysis matches exactly what we found — the two errors are indeed unrelated. This issue focuses on the retention job failing with missing required parameter nDaysSinceLastPull on proxy cache projects (specifically dockerhub-proxy, while other proxy projects with identical policy structure work fine).