[alibaba/higress]helm部署的higress 1.4.2版本升级到v2.0.0-rc.1时higress-controller一直报错Not ready to switch validation to fail-closed

2025-10-27 870 views
5
Ⅰ. Issue Description

Helm部署的higress 1.4.2版本,为了解决长链场景下批量添加/修改ingress引起的内存不释放问题,希望升级到higress 2.0版本,按照步骤升级到v2.0.0-rc.1后higress controller疯狂报错。

  • 操作步骤
  • 1、更新CRD资源
    kubectl apply -f https://raw.githubusercontent.com/alibaba/higress/v2.0.0-rc.1/helm/core/crds/customresourcedefinitions.gen.yaml
  • 2、因为看到helm repo没有更新版本,就手动更改了higress controller 、 higress gateway的image tag为v2.0.0-rc.1,higress controller启动成功后便一直报错
    2024-09-11T08:27:59.009727Z info    validationController    Not ready to switch validation to fail-closed: dummy invalid rejected for the wrong reason: gateways.networking.istio.io is forbidden: User "system:serviceaccount:higress-test:higress-controller" cannot create resource "gateways" in API group "networking.istio.io" in the namespace "higress-test"
    2024-09-11T08:27:59.022513Z info    validationController    Successfully updated validatingwebhookconfiguration istio-validator-istio-system (failurePolicy=Ignore,resourceVersion=1016868101)
    2024-09-11T08:27:59.022544Z error   controllers error handling istio-validator-istio-system, retrying (retry count: 1): webhook is not ready, retry controller=validation
    2024-09-11T08:27:59.024060Z info    validationController    Not ready to switch validation to fail-closed: dummy invalid rejected for the wrong reason: gateways.networking.istio.io is forbidden: User "system:serviceaccount:higress-test:higress-controller" cannot create resource "gateways" in API group "networking.istio.io" in the namespace "higress-test"
    2024-09-11T08:27:59.034137Z info    validationController    Successfully updated validatingwebhookconfiguration istiod-default-validator (failurePolicy=Ignore,resourceVersion=1016868102)
    2024-09-11T08:27:59.034183Z error   controllers error handling istiod-default-validator, retrying (retry count: 1): webhook is not ready, retry controller=validation
    2024-09-11T08:27:59.035834Z info    validationController    Not ready to switch validation to fail-closed: dummy invalid rejected for the wrong reason: gateways.networking.istio.io is forbidden: User "system:serviceaccount:higress-test:higress-controller" cannot create resource "gateways" in API group "networking.istio.io" in the namespace "higress-test"
    2024-09-11T08:27:59.035857Z info    validationController    validatingwebhookconfiguration istio-validator-istio-system (failurePolicy=Ignore, resourceVersion=1016868101) is up-to-date. No change required.
    2024-09-11T08:27:59.035877Z error   controllers error handling istio-validator-istio-system, retrying (retry count: 2): webhook is not ready, retry controller=validation
    2024-09-11T08:27:59.037174Z info    validationController    Not ready to switch validation to fail-closed: dummy invalid rejected for the wrong reason: gateways.networking.istio.io is forbidden: User "system:serviceaccount:higress-test:higress-controller" cannot create resource "gateways" in API group "networking.istio.io" in the namespace "higress-test"
    2024-09-11T08:27:59.037194Z info    validationController    validatingwebhookconfiguration istiod-default-validator (failurePolicy=Ignore, resourceVersion=1016868102) is up-to-date. No change required.
    2024-09-11T08:27:59.037208Z error   controllers error handling istiod-default-validator, retrying (retry count: 2): webhook is not ready, retry controller=validation
    2024-09-11T08:27:59.044542Z info    validationController    Not ready to switch validation to fail-closed: dummy invalid rejected for the wrong reason: gateways.networking.istio.io is forbidden: User "system:serviceaccount:higress-test:higress-controller" cannot create resource "gateways" in API group "networking.istio.io" in the namespace "higress-test"
    2024-09-11T08:27:59.051794Z info    validationController    Successfully updated validatingwebhookconfiguration istio-validator-istio-system (failurePolicy=Ignore,resourceVersion=1016868104)
    2024-09-11T08:27:59.051820Z error   controllers error handling istio-validator-istio-system, retrying (retry count: 3): webhook is not ready, retry controller=validation
    2024-09-11T08:27:59.052880Z info    validationController    Not ready to switch validation to fail-closed: dummy invalid rejected for the wrong reason: gateways.networking.istio.io is forbidden: User "system:serviceaccount:higress-test:higress-controller" cannot create resource "gateways" in API group "networking.istio.io" in the namespace "higress-test"
    2024-09-11T08:27:59.052895Z info    validationController    validatingwebhookconfiguration istio-validator-istio-system (failurePolicy=Ignore, resourceVersion=1016868104) is up-to-date. No change required.
    2024-09-11T08:27:59.052903Z error   controllers error handling istio-validator-istio-system, retrying (retry count: 4): webhook is not ready, retry controller=validation
Ⅵ. Environment:
  • Higress version: 1.4.2
  • 集群内安装过istio,存在istio-validator
    $ kubectl get ValidatingWebhookConfiguration | grep istio
    istio-validator-istio-system                 1          519d
    istiod-default-validator                     1          614d

回答

4

k edit deploy higress-controller -n higres-test

找到 pilot-discovery 这个容器,加上下面这个环境变量:

          - name: VALIDATION_WEBHOOK_CONFIG_NAME
            value: ""
8

好的我试了,添加该env之后该报错消除了,谢谢大佬