REMOVE action stops working after upgrade from .12 to .13 on Crossplane v2

[tanderson]

What happened?

We have a clustered Request object that would reliably delete the external HTTP resource on REMOVE under provider-http 1.0.12, which stopped working after upgrading to .13 (to resolve the DisposableRequest shouldLoopInfinitely issue)

How can we reproduce it?

• Set up a mock REST API
• Set up a Request object that implements CREATE, OBSERVE, REMOVE actions
• Note that when the Request object is deleted, the DELETE method is never sent (and doesn't perform GET either)

Logs for provider-http 1.0.12:

2026-01-20T19:42:34Z	INFO	provider-http	http request sent: {"method":"GET","url":"http://rest-test.rest-test.svc.cluster.local","headers":{"Content-Type":["application/json"]}}	{"request": "rest-test"}
2026-01-20T19:42:34Z	INFO	provider-http	http request sent: {"method":"DELETE","url":"http://rest-test.rest-test.svc.cluster.local","headers":{"Content-Type":["application/json"]}}	{"request": "rest-test"}
2026-01-20T19:42:34Z	DEBUG	provider-http	Successfully requested deletion of external resource	{"controller": "managed/request.http.crossplane.io", "request": {"name":"rest-test"}, "uid": "df0511f4-b98a-4d93-9cbc-1a465043fa44", "version": "522230", "external-name": "rest-test", "deletion-timestamp": "2026-01-20 19:42:34 +0000 UTC"}
2026-01-20T19:42:36Z	INFO	provider-http	http request sent: {"method":"GET","url":"http://rest-test.rest-test.svc.cluster.local","headers":{"Content-Type":["application/json"]}}	{"request": "rest-test"}
2026-01-20T19:42:36Z	DEBUG	provider-http	Successfully deleted managed resource	{"controller": "managed/request.http.crossplane.io", "request": {"name":"rest-test"}, "uid": "df0511f4-b98a-4d93-9cbc-1a465043fa44", "version": "522234", "external-name": "rest-test", "deletion-timestamp": "2026-01-20 19:42:34 +0000 UTC"}

In .12, GET, DELETE, GET http requests are issued by the controller, and everything works as expected.

Logs for provider-http 1.0.13:

2026-01-20T19:47:50Z	DEBUG	provider-http	Request is being deleted, skipping observation	{"request": "rest-test"}
2026-01-20T19:47:50Z	DEBUG	provider-http	Waiting for external resource existence to be confirmed	{"controller": "managed/request.http.crossplane.io", "request": {"name":"rest-test"}, "uid": "e8b537c2-fcbc-4b58-8c75-58bc7c74475b", "version": "522917", "external-name": "rest-test"}
2026-01-20T19:47:51Z	DEBUG	provider-http	Request is being deleted, skipping observation	{"request": "rest-test"}
2026-01-20T19:47:51Z	DEBUG	provider-http	Waiting for external resource existence to be confirmed	{"controller": "managed/request.http.crossplane.io", "request": {"name":"rest-test"}, "uid": "e8b537c2-fcbc-4b58-8c75-58bc7c74475b", "version": "522917", "external-name": "rest-test"}
2026-01-20T19:47:53Z	DEBUG	provider-http	Request is being deleted, skipping observation	{"request": "rest-test"}
2026-01-20T19:47:53Z	DEBUG	provider-http	Waiting for external resource existence to be confirmed	{"controller": "managed/request.http.crossplane.io", "request": {"name":"rest-test"}, "uid": "e8b537c2-fcbc-4b58-8c75-58bc7c74475b", "version": "522917", "external-name": "rest-test"}
2026-01-20T19:47:57Z	DEBUG	provider-http	Request is being deleted, skipping observation	{"request": "rest-test"}
2026-01-20T19:47:57Z	DEBUG	provider-http	Waiting for external resource existence to be confirmed	{"controller": "managed/request.http.crossplane.io", "request": {"name":"rest-test"}, "uid": "e8b537c2-fcbc-4b58-8c75-58bc7c74475b", "version": "522917", "external-name": "rest-test"}
2026-01-20T19:48:05Z	DEBUG	provider-http	Request is being deleted, skipping observation	{"request": "rest-test"}
2026-01-20T19:48:05Z	DEBUG	provider-http	Waiting for external resource existence to be confirmed	{"controller": "managed/request.http.crossplane.io", "request": {"name":"rest-test"}, "uid": "e8b537c2-fcbc-4b58-8c75-58bc7c74475b", "version": "522917", "external-name": "rest-test"}
2026-01-20T19:48:21Z	DEBUG	provider-http	Request is being deleted, skipping observation	{"request": "rest-test"}
2026-01-20T19:48:21Z	DEBUG	provider-http	Successfully deleted managed resource	{"controller": "managed/request.http.crossplane.io", "request": {"name":"rest-test"}, "uid": "e8b537c2-fcbc-4b58-8c75-58bc7c74475b", "version": "522917", "external-name": "rest-test", "deletion-timestamp": "2026-01-20 19:47:50 +0000 UTC"}

Note that neither GET nor DELETE methods are ever sent, instead we only get this message:

Request is being deleted, skipping observation

Looking at the source code, that message is coming from this check during external.Observe:

provider-http/internal/controller/cluster/request/request.go

Lines 187 to 192 in e63374d

	if meta.WasDeleted(mg) {
	c.logger.Debug("Request is being deleted, skipping observation")
	return managed.ExternalObservation{
	ResourceExists: false,
	}, nil
	}

Commenting out this block results in the REMOVE action starting to work again. Note the linked code is for the clustered Request object; the namespaced Request object has it as well, suggesting it might also suffer from the same issue.

Logs for provider-http 1.0.13-patched:

2026-01-20T20:10:12Z	INFO	provider-http	http request sent: {"method":"GET","url":"http://rest-test.rest-test.svc.cluster.local","headers":{"Content-Type":["application/json"]}}	{"request": "rest-test"}
2026-01-20T20:10:12Z	INFO	provider-http	http request sent: {"method":"DELETE","url":"http://rest-test.rest-test.svc.cluster.local","headers":{"Content-Type":["application/json"]}}	{"request": "rest-test"}
2026-01-20T20:10:12Z	DEBUG	provider-http	Request is being deleted, skipping secret injection	{"request": "rest-test"}
2026-01-20T20:10:12Z	DEBUG	provider-http	Successfully requested deletion of external resource	{"controller": "managed/request.http.crossplane.io", "request": {"name":"rest-test"}, "uid": "b3f768b2-ae4e-4be6-b2c9-2f3c65ac0f2b", "version": "524798", "external-name": "rest-test", "deletion-timestamp": "2026-01-20 20:10:12 +0000 UTC"}
2026-01-20T20:10:13Z	INFO	provider-http	http request sent: {"method":"GET","url":"http://rest-test.rest-test.svc.cluster.local","headers":{"Content-Type":["application/json"]}}	{"request": "rest-test"}
2026-01-20T20:10:13Z	DEBUG	provider-http	Successfully deleted managed resource	{"controller": "managed/request.http.crossplane.io", "request": {"name":"rest-test"}, "uid": "b3f768b2-ae4e-4be6-b2c9-2f3c65ac0f2b", "version": "524802", "external-name": "rest-test", "deletion-timestamp": "2026-01-20 20:10:12 +0000 UTC"}

However, this "fix" also causes the test to break:

--- FAIL: TestObserve_DeletionMonitoring (0.00s)
    --- FAIL: TestObserve_DeletionMonitoring/ResourceBeingDeleted (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered, repanicked]
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x174bad5]

goroutine 59 [running]:
testing.tRunner.func1.2({0x19205c0, 0x2beef60})
	/usr/local/go/src/testing/testing.go:1872 +0x237
testing.tRunner.func1()
	/usr/local/go/src/testing/testing.go:1875 +0x35b
panic({0x19205c0?, 0x2beef60?})
	/usr/local/go/src/runtime/panic.go:783 +0x132
github.com/crossplane-contrib/provider-http/internal/service/request.IsUpToDate(0xc00031d0e0, 0xc00057c680)
	/home/tanderson/teck/da-platform/http-xrd/provider-http/internal/service/request/observe.go:70 +0x275
github.com/crossplane-contrib/provider-http/internal/controller/cluster/request.(*external).Observe(0x26efc38?, {0x1e30758, 0x2c49de0}, {0x1e46938?, 0xc000454308})
	/home/tanderson/teck/da-platform/http-xrd/provider-http/internal/controller/cluster/request/request.go:197 +0x20d
github.com/crossplane-contrib/provider-http/internal/controller/cluster/request.TestObserve_DeletionMonitoring.func1(0xc000103340)
	/home/tanderson/teck/da-platform/http-xrd/provider-http/internal/controller/cluster/request/request_test.go:1023 +0xbb
testing.tRunner(0xc000103340, 0xc000115b80)
	/usr/local/go/src/testing/testing.go:1934 +0xea
created by testing.(*T).Run in goroutine 58
	/usr/local/go/src/testing/testing.go:1997 +0x465
FAIL	github.com/crossplane-contrib/provider-http/internal/controller/cluster/request	0.029s

What environment did it happen in?

• Crossplane version: v2.1.3
• kubectl version: v1.34.0
• Kubernetes version: v1.35.0
• Kubernetes distribution: k3s
• OS: Ubuntu 24.04.3 LTS
• Kernel: Linux 6.14.0-37-generic

[samuelbaena]

Any updates?

[freakapoorv]

Hi guys, I got the fix.
so the fix is change ResourceExists: false, to ResourceExists: true, and it works. It does not break any logic.

The reason why we are getting this issue is :-
When we delete the claim, it refers to the OBSERVE section and checks if the resource exists, check L190.
Here it says FALSE so no DELETE is being triggered. For the fix if you change this to TRUE, meaning the resource and claim exists and hence it will trigger a DELETE. which will then make a DELETE call and finally DELETE the underlying resource.

You can get the fix here
main...freakapoorv:provider-http-delete-issue:main

i tried to fix on a fork and this works for me.