Skip to content

Issue with kube-prometheus-stack addons #154

@contact2rames

Description

@contact2rames

Hello,
We have used eks-blueprints-add-ons for argo and kube_prometheus_stack and installed in 3 environments. We are using EKS 1.23.

  1. In one environment, we see only namespace is created and not any resources/pods under it.

[root@ip-10-71-0-1 ~]# kubectl get ns
NAME STATUS AGE
argocd Active 26d
cert-manager Active 26d
karpenter Active 26d
kube-node-lease Active 26d
kube-prometheus-stack Active 26d

[root@ip-10-71-0-1 ~]# kubectl get pods -n kube-prometheus-stack
No resources found in kube-prometheus-stack namespace.

  1. In other environment, kube-prometheus-stack-grafana,kube-state-metrics, prometheus-node-exporter pods got created and running, but not stack-operator and below is the error:

[root@ip-10-73-0-13 ~]# kubectl events kube-prometheus-stack-operator-6f4f8975fb-qwcfk -n kube-prometheus-stack
LAST SEEN TYPE REASON OBJECT MESSAGE
33m (x267 over 44h) Warning FailedMount Pod/kube-prometheus-stack-operator-6f4f8975fb-vvxtg Unable to attach or mount volumes: unmounted volumes=[tls-secret], unattached volumes=[kube-api-access-tck4q tls-secret]: timed out waiting for the condition
8m39s (x1328 over 44h) Warning FailedMount Pod/kube-prometheus-stack-operator-6f4f8975fb-vvxtg MountVolume.SetUp failed for volume "tls-secret" : secret "kube-prometheus-stack-admission" not found
3m40s (x915 over 44h) Warning FailedMount Pod/kube-prometheus-stack-operator-6f4f8975fb-vvxtg Unable to attach or mount volumes: unmounted volumes=[tls-secret], unattached volumes=[tls-secret kube-api-access-tck4q]: timed out waiting for the condition
3m8s Normal Scheduled Pod/kube-prometheus-stack-operator-6f4f8975fb-qwcfk Successfully assigned kube-prometheus-stack/kube-prometheus-stack-operator-6f4f8975fb-qwcfk to ip-10-73-2-246.ap-southeast-2.compute.internal
3m8s Normal SuccessfulCreate ReplicaSet/kube-prometheus-stack-operator-6f4f8975fb Created pod: kube-prometheus-stack-operator-6f4f8975fb-qwcfk
65s Warning FailedMount Pod/kube-prometheus-stack-operator-6f4f8975fb-qwcfk Unable to attach or mount volumes: unmounted volumes=[tls-secret], unattached volumes=[tls-secret kube-api-access-xh9wx]: timed out waiting for the condition
60s (x9 over 3m8s) Warning FailedMount Pod/kube-prometheus-stack-operator-6f4f8975fb-qwcfk MountVolume.SetUp failed for volume "tls-secret" : secret "kube-prometheus-stack-admission" not found

60s (x9 over 3m8s) Warning FailedMount Pod/kube-prometheus-stack-operator-6f4f8975fb-qwcfk MountVolume.SetUp failed for volume "tls-secret" : secret "kube-prometheus-stack-admission" not found

Could you please let me know what i am missing to resolve this issues.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions