[alibaba/higress]关于higress部署的一点疑问

2025-11-04 196 views
1

社区提供了helm部署的方式,我本地使用helm部署也是正常的,但是我们的生产环境没法使用helm进行部署,只能使用kubectl apply进行部署,如下是我的部署步骤 1.使用helm template 命令获取到发布资源的yaml文件 2.使用kubectl apply -f 加载这个文件

结果是higress-gateway 这个pod的状态一直有问题,如下是日志: `

    Events:
  Type     Reason     Age                     From               Message
  ----     ------     ----                    ----               -------
  Normal   Scheduled  5m58s                   default-scheduler  Successfully assigned higress-system/higress-gateway-6558d5965b-fb5wq to node2
  Normal   Created    5m38s (x2 over 5m58s)   kubelet            Created container higress-gateway
  Normal   Started    5m38s (x2 over 5m58s)   kubelet            Started container higress-gateway
  Warning  Unhealthy  5m30s (x19 over 5m57s)  kubelet            Readiness probe failed: Get "http://10.244.104.45:15021/healthz/ready": dial tcp 10.244.104.45:15021: connect: connection refused

`

想问下如何解决这个问题

环境: `

Linux master 3.10.0-1160.71.1.el7.x86_64 #1 SMP Tue Jun 28 15:37:28 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.0", GitCommit:"ab69524f795c42094a6630298ff53f3c3ebab7f4", GitTreeState:"clean", BuildDate:"2021-12-07T18:16:20Z", GoVersion:"go1.17.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.0", GitCommit:"ab69524f795c42094a6630298ff53f3c3ebab7f4", GitTreeState:"clean", BuildDate:"2021-12-07T18:09:57Z", GoVersion:"go1.17.3", Compiler:"gc", Platform:"linux/amd64"}

`

回答

3

补充信息 直接使用helm template生成的yaml进行部署,会有如下的问题

`

Events:
Type     Reason                  Age                 From               Message
----     ------                  ----                ----               -------
Normal   Scheduled               49s                 default-scheduler  Successfully assigned higress-system/higress-gateway-795454b45f-bp6bv to node1
Warning  FailedCreatePodSandBox  37s (x13 over 49s)  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to start sandbox container for pod "higress-gateway-795454b45f-bp6bv": Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: open /proc/sys/net/ipv4/ip_unprivileged_port_start: no such file or directory: unknown

`

我将上图中的配置调整为(通过本地helm部署后查看higress-gateway的deployment文件,发现和下述配置一致): securityContext: {}

就会出现健康检测通不过的问题

4

根据issue #226 进行了修改,问题还是健康检测通不过

`

 Events:
Type     Reason          Age                    From               Message
----     ------          ----                   ----               -------
Normal   Scheduled       2m24s                  default-scheduler  Successfully assigned higress-system/higress-gateway-96897b77c-cvz9b to node2
Normal   Pulled          2m24s                  kubelet            Container image "higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/gateway:2.0.7" already present on machine
Normal   Created         2m24s                  kubelet            Created container higress-gateway
Normal   Started         2m23s                  kubelet            Started container higress-gateway
Warning  Unhealthy       104s (x21 over 2m22s)  kubelet            Readiness probe failed: Get "http://10.244.104.58:15021/healthz/ready": dial tcp 10.244.104.58:15021: connect: connection refused
Normal   SandboxChanged  104s                   kubelet            Pod sandbox changed, it will be killed and re-created.

`

9

发现如果使用#issue226 的配置,15021端口不存在监听: `

 [root@master test]# kubectl exec -it higress-gateway-96897b77c-cvz9b -n higress-system -- netstat -tuln
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 127.0.0.1:15000         0.0.0.0:*               LISTEN     
tcp        0      0 127.0.0.1:15004         0.0.0.0:*               LISTEN     
tcp6       0      0 :::15020                :::*                    LISTEN 

`

如果使用原始helm create生成的配置,则如下:

`

  [root@master test]# kubectl exec -it higress-gateway-64cd888cb4-p9c6m -n higress-system -- netstat -tuln
  Active Internet connections (only servers)
  Proto Recv-Q Send-Q Local Address           Foreign Address         State      
  tcp        0      0 0.0.0.0:15021           0.0.0.0:*               LISTEN     
  tcp        0      0 0.0.0.0:15090           0.0.0.0:*               LISTEN     
  tcp        0      0 127.0.0.1:15000         0.0.0.0:*               LISTEN     
  tcp        0      0 127.0.0.1:15004         0.0.0.0:*               LISTEN     
  tcp6       0      0 :::15020                :::*                    LISTEN

[root@master test]# kubectl exec -it higress-gateway-64cd888cb4-p9c6m -n higress-system -- curl -v http://localhost:15021/healthz/ready
*   Trying 127.0.0.1:15021...
* Connected to localhost (127.0.0.1) port 15021 (#0)
> GET /healthz/ready HTTP/1.1
> Host: localhost:15021
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 503 Service Unavailable
< date: Mon, 14 Apr 2025 06:04:45 GMT
< content-length: 0
< req-cost-time: 1301
< req-arrive-time: 1744610684641
< resp-start-time: 1744610685943
< x-envoy-upstream-service-time: 1300
< server: envoy
< 
* Connection #0 to host localhost left intact

`

2

看一下 gateway 容器的日志

3

看一下 gateway 容器的日志

`

  [root@master test]# kubectl describe pod higress-gateway-5789bd988d-9qpnn -n higress-system
Name:         higress-gateway-5789bd988d-9qpnn
Namespace:    higress-system
Priority:     0
Node:         node2/172.20.48.147
Start Time:   Mon, 14 Apr 2025 15:14:02 +0800
Labels:       app=higress-gateway
              higress=higress-system-higress-gateway
              pod-template-hash=5789bd988d
              sidecar.istio.io/inject=false
Annotations:  cni.projectcalico.org/podIP: 10.244.104.37/32
              cni.projectcalico.org/podIPs: 10.244.104.37/32
              prometheus.io/path: /stats/prometheus
              prometheus.io/port: 15020
              prometheus.io/scrape: true
              sidecar.istio.io/inject: false
Status:       Running
IP:           10.244.104.37
IPs:
  IP:           10.244.104.37
Controlled By:  ReplicaSet/higress-gateway-5789bd988d
Containers:
  higress-gateway:
    Container ID:  docker://484f66011f80828f207fdc858b7ee084c55dcb4aee5f14e5bbe77b8dbe531a7a
    Image:         higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/gateway:2.0.7
    Image ID:      docker-pullable://harbor.hwwt2.com/ai-engineering/higress/gateway@sha256:44678f489c27c7ea089253c5a95eb786c847bb17736e36358a7dd70078ec0d6d
    Ports:         15020/TCP, 15090/TCP
    Host Ports:    0/TCP, 0/TCP
    Args:
      proxy
      router
      --domain
      $(POD_NAMESPACE).svc.cluster.local
      --proxyLogLevel=warning
      --proxyComponentLogLevel=misc:error
      --log_output_level=all:info
      --serviceCluster=higress-gateway
    State:          Running
      Started:      Mon, 14 Apr 2025 15:14:02 +0800
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     2
      memory:  2Gi
    Requests:
      cpu:      2
      memory:   2Gi
    Readiness:  http-get http://:15021/healthz/ready delay=1s timeout=3s period=2s #success=1 #failure=30
    Environment:
      NODE_NAME:                    (v1:spec.nodeName)
      POD_NAME:                    higress-gateway-5789bd988d-9qpnn (v1:metadata.name)
      POD_NAMESPACE:               higress-system (v1:metadata.namespace)
      INSTANCE_IP:                  (v1:status.podIP)
      HOST_IP:                      (v1:status.hostIP)
      SERVICE_ACCOUNT:              (v1:spec.serviceAccountName)
      PROXY_XDS_VIA_AGENT:         true
      ENABLE_INGRESS_GATEWAY_SDS:  false
      JWT_POLICY:                  third-party-jwt
      ISTIO_META_HTTP10:           1
      ISTIO_META_CLUSTER_ID:       Kubernetes
      INSTANCE_NAME:               higress-gateway
    Mounts:
      /etc/istio/config from config (rw)
      /etc/istio/pod from podinfo (rw)
      /etc/istio/proxy from proxy-socket (rw)
      /var/lib/istio/data from istio-data (rw)
      /var/run/secrets/credential-uds from credential-socket (rw)
      /var/run/secrets/istio from higress-ca-root-cert (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-xm44p (ro)
      /var/run/secrets/tokens from istio-token (ro)
      /var/run/secrets/workload-spiffe-credentials from workload-certs (rw)
      /var/run/secrets/workload-spiffe-uds from workload-socket (rw)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  workload-socket:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  credential-socket:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  workload-certs:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  istio-token:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  43200
  higress-ca-root-cert:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      higress-ca-root-cert
    Optional:  false
  config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      higress-config
    Optional:  false
  istio-data:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  proxy-socket:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  podinfo:
    Type:  DownwardAPI (a volume populated by information about the pod)
    Items:
      metadata.labels -> labels
      metadata.annotations -> annotations
      requests.cpu -> cpu-request
      limits.cpu -> cpu-limit
  kube-api-access-xm44p:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Guaranteed
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age                 From               Message
  ----     ------     ----                ----               -------
  Normal   Scheduled  15m                 default-scheduler  Successfully assigned higress-system/higress-gateway-5789bd988d-9qpnn to node2
  Normal   Pulled     15m                 kubelet            Container image "higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/gateway:2.0.7" already present on machine
  Normal   Created    15m                 kubelet            Created container higress-gateway
  Normal   Started    15m                 kubelet            Started container higress-gateway
  Warning  Unhealthy  14m (x13 over 15m)  kubelet            Readiness probe failed: Get "http://10.244.104.37:15021/healthz/ready": dial tcp 10.244.104.37:15021: connect: connection refused
  Warning  Unhealthy  1s (x453 over 14m)  kubelet            Readiness probe failed: HTTP probe failed with statuscode: 503

`

5

不是describe,是log

5

不是describe,是log

`

    [root@master test]# kubectl logs higress-gateway-5789bd988d-kbfxf  -n higress-system
2025-04-14T07:34:02.224019Z     info    FLAG: --concurrency="0"
2025-04-14T07:34:02.224078Z     info    FLAG: --domain="higress-system.svc.cluster.local"
2025-04-14T07:34:02.224085Z     info    FLAG: --help="false"
2025-04-14T07:34:02.224088Z     info    FLAG: --log_as_json="false"
2025-04-14T07:34:02.224091Z     info    FLAG: --log_caller=""
2025-04-14T07:34:02.224094Z     info    FLAG: --log_output_level="all:info"
2025-04-14T07:34:02.224096Z     info    FLAG: --log_rotate=""
2025-04-14T07:34:02.224099Z     info    FLAG: --log_rotate_max_age="30"
2025-04-14T07:34:02.224102Z     info    FLAG: --log_rotate_max_backups="1000"
2025-04-14T07:34:02.224105Z     info    FLAG: --log_rotate_max_size="104857600"
2025-04-14T07:34:02.224108Z     info    FLAG: --log_stacktrace_level="default:none"
2025-04-14T07:34:02.224116Z     info    FLAG: --log_target="[stdout]"
2025-04-14T07:34:02.224119Z     info    FLAG: --meshConfig="./etc/istio/config/mesh"
2025-04-14T07:34:02.224122Z     info    FLAG: --outlierLogPath=""
2025-04-14T07:34:02.224125Z     info    FLAG: --profiling="true"
2025-04-14T07:34:02.224136Z     info    FLAG: --proxyComponentLogLevel="misc:error"
2025-04-14T07:34:02.224138Z     info    FLAG: --proxyLogLevel="warning"
2025-04-14T07:34:02.224141Z     info    FLAG: --serviceCluster="higress-gateway"
2025-04-14T07:34:02.224144Z     info    FLAG: --stsPort="0"
2025-04-14T07:34:02.224147Z     info    FLAG: --templateFile=""
2025-04-14T07:34:02.224149Z     info    FLAG: --tokenManagerPlugin="GoogleTokenExchange"
2025-04-14T07:34:02.224153Z     info    FLAG: --vklog="0"
2025-04-14T07:34:02.224157Z     info    Version 1.19-dev-unknown-Modified
2025-04-14T07:34:02.226683Z     info    Maximum file descriptors (ulimit -n): 5242880
2025-04-14T07:34:02.226975Z     info    Proxy role      ips=[10.244.166.161] type=router id=higress-gateway-5789bd988d-kbfxf.higress-system domain=higress-system.svc.cluster.local
2025-04-14T07:34:02.227125Z     info    Apply mesh config from file accessLogEncoding: TEXT
accessLogFile: /var/log/higress/access/access.log
accessLogFormat: |
  {"ai_log":"%FILTER_STATE(wasm.ai_log:PLAIN)%","authority":"%REQ(X-ENVOY-ORIGINAL-HOST?:AUTHORITY)%","bytes_received":"%BYTES_RECEIVED%","bytes_sent":"%BYTES_SENT%","downstream_local_address":"%DOWNSTREAM_LOCAL_ADDRESS%","downstream_remote_address":"%DOWNSTREAM_REMOTE_ADDRESS%","duration":"%DURATION%","istio_policy_status":"%DYNAMIC_METADATA(istio.mixer:status)%","method":"%REQ(:METHOD)%","path":"%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%","protocol":"%PROTOCOL%","request_id":"%REQ(X-REQUEST-ID)%","requested_server_name":"%REQUESTED_SERVER_NAME%","response_code":"%RESPONSE_CODE%","response_flags":"%RESPONSE_FLAGS%","route_name":"%ROUTE_NAME%","start_time":"%START_TIME%","trace_id":"%REQ(X-B3-TRACEID)%","upstream_cluster":"%UPSTREAM_CLUSTER%","upstream_host":"%UPSTREAM_HOST%","upstream_local_address":"%UPSTREAM_LOCAL_ADDRESS%","upstream_service_time":"%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%","upstream_transport_failure_reason":"%UPSTREAM_TRANSPORT_FAILURE_REASON%","user_agent":"%REQ(USER-AGENT)%","x_forwarded_for":"%REQ(X-FORWARDED-FOR)%","response_code_details":"%RESPONSE_CODE_DETAILS%"}
configSources:
- address: xds://127.0.0.1:15051
- address: k8s://
defaultConfig:
  discoveryAddress: higress-controller.higress-system.svc:15012
  proxyStatsMatcher:
    inclusionRegexps:
    - .*
  tracing: {}
dnsRefreshRate: 200s
enableAutoMtls: false
enablePrometheusMerge: true
ingressControllerMode: "OFF"
mseIngressGlobalConfig:
  enableH3: false
  enableProxyProtocol: false
protocolDetectionTimeout: 100ms
rootNamespace: higress-system
trustDomain: cluster.local
2025-04-14T07:34:02.231416Z     info    Effective config: binaryPath: /usr/local/bin/envoy
concurrency: 2
configPath: ./etc/istio/proxy
controlPlaneAuthPolicy: MUTUAL_TLS
discoveryAddress: higress-controller.higress-system.svc:15012
drainDuration: 45s
proxyAdminPort: 15000
proxyStatsMatcher:
  inclusionRegexps:
  - .*
serviceCluster: istio-proxy
statNameLength: 189
statusPort: 15020
terminationDrainDuration: 5s
tracing: {}

2025-04-14T07:34:02.231459Z     info    JWT policy is third-party-jwt
2025-04-14T07:34:02.231465Z     info    using credential fetcher of JWT type in cluster.local trust domain
2025-04-14T07:34:02.435608Z     info    Workload SDS socket not found. Starting Istio SDS Server
2025-04-14T07:34:02.435642Z     info    CA Endpoint higress-controller.higress-system.svc:15012, provider Citadel
2025-04-14T07:34:02.435690Z     info    Using CA higress-controller.higress-system.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem
2025-04-14T07:34:02.436361Z     info    Opening status port 15020
2025-04-14T07:34:02.460305Z     info    ads     All caches have been synced up in 236.854716ms, marking server ready
2025-04-14T07:34:02.461240Z     info    xdsproxy        Initializing with upstream address "higress-controller.higress-system.svc:15012" and cluster "Kubernetes"
2025-04-14T07:34:02.461381Z     info    sds     Starting SDS grpc server
2025-04-14T07:34:02.461846Z     info    starting Http service at 127.0.0.1:15004
2025-04-14T07:34:02.463698Z     info    Pilot SAN: [higress-controller.higress-system.svc]
2025-04-14T07:34:02.465618Z     info    Starting proxy agent
2025-04-14T07:34:02.465659Z     info    starting
2025-04-14T07:34:02.465680Z     info    Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields --log-format %Y-%m-%dT%T.%fZ  %l      envoy %n %g:%#  %v      thread=%t -l warning --component-log-level misc:error --concurrency 2]
2025-04-14T07:34:22.438446Z     warn    ca      ca request failed, starting attempt 1 in 100.237403ms
2025-04-14T07:34:22.539800Z     warn    ca      ca request failed, starting attempt 2 in 189.485894ms
2025-04-14T07:34:22.729906Z     warn    ca      ca request failed, starting attempt 3 in 399.552017ms
2025-04-14T07:34:22.905008Z     info    xdsproxy        connected to upstream XDS server: higress-controller.higress-system.svc:15012
2025-04-14T07:34:22.991129Z     warning envoy config external/envoy/source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:128    gRPC config for type.googleapis.com/envoy.config.listener.v3.Listener rejected: Error adding/updating listener(s) 0.0.0.0_80: unable to open file '/var/log/higress/access/access.log': No such file or directory
0.0.0.0_443: unable to open file '/var/log/higress/access/access.log': No such file or directory
        thread=25
2025-04-14T07:34:23.130179Z     warn    ca      ca request failed, starting attempt 4 in 764.14061ms
2025-04-14T07:34:23.903987Z     info    cache   generated new workload certificate      latency=21.442808985s ttl=23h59m59.096020504s
2025-04-14T07:34:23.904039Z     info    cache   Root cert has changed, start rotating root cert
2025-04-14T07:34:23.904066Z     info    ads     XDS: Incremental Pushing ConnectedEndpoints:0 Version:
2025-04-14T07:34:23.904134Z     info    cache   returned workload trust anchor from cache       ttl=23h59m59.09586746s
2025-04-14T07:34:24.877313Z     warn    Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected
2025-04-14T07:34:26.875766Z     warn    Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected
2025-04-14T07:34:28.876274Z     warn    Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected
2025-04-14T07:34:30.876080Z     warn    Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected
2025-04-14T07:34:32.876020Z     warn    Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected
2025-04-14T07:34:34.876470Z     warn    Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected
2025-04-14T07:34:36.875824Z     warn    Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected
2025-04-14T07:34:38.875803Z     warn    Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected
2025-04-14T07:34:40.875699Z     warn    Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected
2025-04-14T07:34:42.875729Z     warn    Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected

`

4

感觉挺奇怪的,我把access_log,改成一个文件目录/var/log/higress/access/access.log,pod健康检查就不通过,如果是/dev/stdout就没问题

4

如下是我使用的yaml: `

       # Source: higress/charts/higress-core/templates/controller-serviceaccont.yaml
  apiVersion: v1
  kind: ServiceAccount
  metadata:
    name: higress-controller
    namespace: higress-system
    labels:
      helm.sh/chart: higress-core-2.0.7
      app: higress-controller
      higress: higress-controller
      app.kubernetes.io/version: "2.0.7"
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: higress-controller
  ---
  # Source: higress/charts/higress-core/templates/serviceaccount.yaml
  apiVersion: v1
  kind: ServiceAccount
  metadata:
    name: higress-gateway
    namespace: higress-system
    labels:
      helm.sh/chart: higress-core-2.0.7
      app: higress-gateway
      higress: higress-system-higress-gateway
      app.kubernetes.io/version: "2.0.7"
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: higress-gateway
  ---
  # Source: higress/charts/higress-core/templates/configmap.yaml
  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: higress-config
    namespace: higress-system
    labels:
      helm.sh/chart: higress-core-2.0.7
      app: higress-gateway
      higress: higress-system-higress-gateway
      app.kubernetes.io/version: "2.0.7"
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: higress-gateway    
  data:
    higress: |-
      downstream:
        connectionBufferLimits: 32768
        http2:
          initialConnectionWindowSize: 1048576
          initialStreamWindowSize: 65535
          maxConcurrentStreams: 100
        idleTimeout: 180
        maxRequestHeadersKb: 60
        routeTimeout: 0
      upstream:
        connectionBufferLimits: 10485760
        idleTimeout: 10
    # Configuration file for the mesh networks to be used by the Split Horizon EDS.
    meshNetworks: |-
      networks: {}

    mesh: |-
      accessLogEncoding: TEXT
      accessLogFile: /var/log/higress/access/access.log
      accessLogFormat: |
        {"ai_log":"%FILTER_STATE(wasm.ai_log:PLAIN)%","authority":"%REQ(X-ENVOY-ORIGINAL-HOST?:AUTHORITY)%","bytes_received":"%BYTES_RECEIVED%","bytes_sent":"%BYTES_SENT%","downstream_local_address":"%DOWNSTREAM_LOCAL_ADDRESS%","downstream_remote_address":"%DOWNSTREAM_REMOTE_ADDRESS%","duration":"%DURATION%","istio_policy_status":"%DYNAMIC_METADATA(istio.mixer:status)%","method":"%REQ(:METHOD)%","path":"%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%","protocol":"%PROTOCOL%","request_id":"%REQ(X-REQUEST-ID)%","requested_server_name":"%REQUESTED_SERVER_NAME%","response_code":"%RESPONSE_CODE%","response_flags":"%RESPONSE_FLAGS%","route_name":"%ROUTE_NAME%","start_time":"%START_TIME%","trace_id":"%REQ(X-B3-TRACEID)%","upstream_cluster":"%UPSTREAM_CLUSTER%","upstream_host":"%UPSTREAM_HOST%","upstream_local_address":"%UPSTREAM_LOCAL_ADDRESS%","upstream_service_time":"%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%","upstream_transport_failure_reason":"%UPSTREAM_TRANSPORT_FAILURE_REASON%","user_agent":"%REQ(USER-AGENT)%","x_forwarded_for":"%REQ(X-FORWARDED-FOR)%","response_code_details":"%RESPONSE_CODE_DETAILS%"}
      configSources:
      - address: xds://127.0.0.1:15051
      - address: k8s://
      defaultConfig:
        discoveryAddress: higress-controller.higress-system.svc:15012
        proxyStatsMatcher:
          inclusionRegexps:
          - .*
        tracing: {}
      dnsRefreshRate: 200s
      enableAutoMtls: false
      enablePrometheusMerge: true
      ingressControllerMode: "OFF"
      mseIngressGlobalConfig:
        enableH3: false
        enableProxyProtocol: false
      protocolDetectionTimeout: 100ms
      rootNamespace: higress-system
      trustDomain: cluster.local
  ---
  # Source: higress/charts/higress-core/templates/clusterrole.yaml
  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRole
  metadata:
    name: higress-gateway-higress-system
  rules:
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get", "watch", "list"]
  ---
  # Source: higress/charts/higress-core/templates/controller-clusterrole.yaml
  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRole
  metadata:
    name: higress-controller-higress-system
    labels:
      helm.sh/chart: higress-core-2.0.7
      app: higress-controller
      higress: higress-controller
      app.kubernetes.io/version: "2.0.7"
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: higress-controller
  rules:
    # ingress controller
    - apiGroups: ["extensions", "networking.k8s.io"]
      resources: ["ingresses"]
      verbs: ["create", "get", "list", "watch", "update", "delete", "patch"]
    - apiGroups: ["extensions", "networking.k8s.io"]
      resources: ["ingresses/status"]
      verbs: ["*"]
    - apiGroups: ["networking.k8s.io"]
      resources: ["ingresses", "ingressclasses"]
      verbs: ["get", "list", "watch"]
    - apiGroups: ["networking.k8s.io"]
      resources: ["ingresses/status"]
      verbs: ["*"]

    # required for CA's namespace controller
    - apiGroups: [""]
      resources: ["configmaps"]
      verbs: ["create", "get", "list", "watch", "update"]

    # Use for Kubernetes Service APIs
    - apiGroups: ["networking.x-k8s.io"]
      resources: ["*"]
      verbs: ["get", "watch", "list"]
    - apiGroups: ["networking.x-k8s.io"]
      resources: ["*"] # TODO: should be on just */status but wildcard is not supported
      verbs: ["update"]

    # Gateway api controller
    - apiGroups: ["gateway.networking.k8s.io"]
      resources: ["*"]
      verbs: ["get", "watch", "list", "create", "update", "delete", "patch"]

    # Needed for multicluster secret reading, possibly ingress certs in the future
    - apiGroups: [""]
      resources: ["secrets"]
      verbs: ["get", "watch", "list", "create", "update", "delete", "patch"]

    - apiGroups: ["networking.higress.io"]
      resources: ["mcpbridges"]
      verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

    - apiGroups: ["extensions.higress.io"]
      resources: ["wasmplugins"]
      verbs: ["get", "list", "watch"]

    - apiGroups: ["networking.higress.io"]
      resources: ["http2rpcs"]
      verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

    - apiGroups: [""]
      resources: ["services"]
      verbs: ["get", "watch", "list", "update", "patch", "create", "delete"]

    # auto-detect installed CRD definitions
    - apiGroups: ["apiextensions.k8s.io"]
      resources: ["customresourcedefinitions"]
      verbs: ["get", "list", "watch"]

    # discovery and routing
    - apiGroups: [""]
      resources: ["pods", "nodes", "services", "namespaces", "endpoints", "deployments"]
      verbs: ["get", "list", "watch"]
    - apiGroups: ["discovery.k8s.io"]
      resources: ["endpointslices"]
      verbs: ["get", "list", "watch"]

    # Istiod and bootstrap.
    - apiGroups: ["certificates.k8s.io"]
      resources:
        - "certificatesigningrequests"
        - "certificatesigningrequests/approval"
        - "certificatesigningrequests/status"
      verbs: ["update", "create", "get", "delete", "watch"]
    - apiGroups: ["certificates.k8s.io"]
      resources:
        - "signers"
      resourceNames:
      - "kubernetes.io/legacy-unknown"
      verbs: ["approve"]

    # Used by Istiod to verify the JWT tokens
    - apiGroups: ["authentication.k8s.io"]
      resources: ["tokenreviews"]
      verbs: ["create"]

    # Used by Istiod to verify gateway SDS
    - apiGroups: ["authorization.k8s.io"]
      resources: ["subjectaccessreviews"]
      verbs: ["create"]

    # Used for MCS serviceexport management
    - apiGroups: ["multicluster.x-k8s.io"]
      resources: ["serviceexports"]
      verbs: [ "get", "watch", "list", "create", "delete"]

    # Used for MCS serviceimport management
    - apiGroups: ["multicluster.x-k8s.io"]
      resources: ["serviceimports"]
      verbs: ["get", "watch", "list"]

    # sidecar injection controller
    - apiGroups: ["admissionregistration.k8s.io"]
      resources: ["mutatingwebhookconfigurations"]
      verbs: ["get", "list", "watch", "update", "patch"]

    # configuration validation webhook controller
    - apiGroups: ["admissionregistration.k8s.io"]
      resources: ["validatingwebhookconfigurations"]
      verbs: ["get", "list", "watch", "update"]

    # istio configuration
    # removing CRD permissions can break older versions of Istio running alongside this control plane (https://github.com/istio/istio/issues/29382)
    # please proceed with caution
    - apiGroups: ["config.istio.io", "security.istio.io", "networking.istio.io", "authentication.istio.io", "rbac.istio.io", "telemetry.istio.io", "extensions.istio.io"]
      verbs: ["get", "watch", "list"]
      resources: ["*"]
    # knative KIngress configuration
    - apiGroups: ["networking.internal.knative.dev"]
      verbs: ["get","list","watch"]
      resources: ["ingresses"]
    - apiGroups: ["networking.internal.knative.dev"]
      resources: ["ingresses/status"]
      verbs: ["get","patch","update"]
    # gateway api need
    - apiGroups: ["apps"]
      verbs: [ "get", "watch", "list", "update", "patch", "create", "delete" ]
      resources: [ "deployments" ]
    - apiGroups: [""]
      verbs: [ "get", "watch", "list", "update", "patch", "create", "delete" ]
      resources: [ "serviceaccounts"]
  ---
  # Source: higress/charts/higress-core/templates/clusterrole.yaml
  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRoleBinding
  metadata:
    name: higress-gateway-higress-system
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: higress-gateway-higress-system
  subjects:
  - kind: ServiceAccount
    name: higress-gateway
    namespace: higress-system
  ---
  # Source: higress/charts/higress-core/templates/controller-clusterrolebinding.yaml
  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRoleBinding
  metadata:
    name: higress-controller-higress-system
    labels:
      helm.sh/chart: higress-core-2.0.7
      app: higress-controller
      higress: higress-controller
      app.kubernetes.io/version: "2.0.7"
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: higress-controller    
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: higress-controller-higress-system
  subjects:
    - kind: ServiceAccount
      name: higress-controller
      namespace: higress-system
  ---
  # Source: higress/charts/higress-core/templates/controller-role.yaml
  apiVersion: rbac.authorization.k8s.io/v1
  kind: Role
  metadata:
    name: higress-controller
    namespace: higress-system
    labels:
      helm.sh/chart: higress-core-2.0.7
      app: higress-controller
      higress: higress-controller
      app.kubernetes.io/version: "2.0.7"
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: higress-controller
  rules:
  # For storing CA secret
  - apiGroups: [""]
    resources: ["secrets"]
    # TODO lock this down to istio-ca-cert if not using the DNS cert mesh config
    verbs: ["create", "get", "watch", "list", "update", "delete"]
  ---
  # Source: higress/charts/higress-core/templates/role.yaml
  apiVersion: rbac.authorization.k8s.io/v1
  kind: Role
  metadata:
    name: higress-gateway
    namespace: higress-system
  rules:
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get", "watch", "list"]
  ---
  # Source: higress/charts/higress-core/templates/controller-rolebinding.yaml
  apiVersion: rbac.authorization.k8s.io/v1
  kind: RoleBinding
  metadata:
    name: higress-controller
    namespace: higress-system
    labels:
      helm.sh/chart: higress-core-2.0.7
      app: higress-controller
      higress: higress-controller
      app.kubernetes.io/version: "2.0.7"
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: higress-controller
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: Role
    name: higress-controller
  subjects:
    - kind: ServiceAccount
      name: higress-controller
      namespace: higress-system
  ---
  # Source: higress/charts/higress-core/templates/role.yaml
  apiVersion: rbac.authorization.k8s.io/v1
  kind: RoleBinding
  metadata:
    name: higress-gateway
    namespace: higress-system
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: Role
    name: higress-gateway
  subjects:
  - kind: ServiceAccount
    name: higress-gateway
  ---
  # Source: higress/charts/higress-core/templates/controller-service.yaml
  apiVersion: v1
  kind: Service
  metadata:
    name: higress-controller
    namespace: higress-system
    labels:
      helm.sh/chart: higress-core-2.0.7
      app: higress-controller
      higress: higress-controller
      app.kubernetes.io/version: "2.0.7"
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: higress-controller
  spec:
    type: ClusterIP
    ports:
      - name: http
        port: 8888
        protocol: TCP
        targetPort: 8888
      - name: http-solver
        port: 8889
        protocol: TCP
        targetPort: 8889
      - name: grpc
        port: 15051
        protocol: TCP
        targetPort: 15051
      - port: 15010
        name: grpc-xds # plaintext
        protocol: TCP
      - port: 15012
        name: https-dns # mTLS with k8s-signed cert
        protocol: TCP
      - port: 443
        name: https-webhook # validation and injection
        targetPort: 15017
        protocol: TCP
      - port: 15014
        name: http-monitoring # prometheus stats
        protocol: TCP
    selector:
      app: higress-controller
      higress: higress-controller
  ---
  # Source: higress/charts/higress-core/templates/service.yaml
  apiVersion: v1
  kind: Service
  metadata:
    name: higress-gateway
    namespace: higress-system
    labels:
      helm.sh/chart: higress-core-2.0.7
      app: higress-gateway
      higress: higress-system-higress-gateway
      app.kubernetes.io/version: "2.0.7"
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: higress-gateway
    annotations:
      {}
  spec:
    type: LoadBalancer
    ports:
      - name: http2
        port: 80
        protocol: TCP
        targetPort: 80
      - name: https
        port: 443
        protocol: TCP
        targetPort: 443
    selector:
      app: higress-gateway
      higress: higress-system-higress-gateway
  ---
  # Source: higress/charts/higress-core/templates/controller-deployment.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: higress-controller
    namespace: higress-system
    labels:
      helm.sh/chart: higress-core-2.0.7
      app: higress-controller
      higress: higress-controller
      app.kubernetes.io/version: "2.0.7"
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: higress-controller
  spec:
    replicas: 1
    selector:
      matchLabels:
        app: higress-controller
        higress: higress-controller
    template:
      metadata:
        labels:
          app: higress-controller
          higress: higress-controller
      spec:
        serviceAccountName: higress-controller
        securityContext:
          {}
        containers:
          - name: higress-core
            securityContext:
              {}
            image: "higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/higress:2.0.7"
            args:
            - "serve"
            - --gatewaySelectorKey=higress
            - --gatewaySelectorValue=higress-system-higress-gateway
            - --gatewayHttpPort=80
            - --gatewayHttpsPort=443
            - --ingressClass=higress
            - --enableAutomaticHttps=true
            - --automaticHttpsEmail=
            env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
            - name: SERVICE_ACCOUNT
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: spec.serviceAccountName
            - name: DOMAIN_SUFFIX
              value: cluster.local
            - name: GATEWAY_NAME
              value: higress-gateway
            - name: PILOT_ENABLE_GATEWAY_API
              value: "false"
            - name: PILOT_ENABLE_ALPHA_GATEWAY_API
              value: "false"
            ports:
              - name: http
                containerPort: 8888
                protocol: TCP
              - name: http-solver
                containerPort: 8889
                protocol: TCP
              - name: grpc
                containerPort: 15051
                protocol: TCP
            readinessProbe:
              httpGet:
                path: /ready
                port: 8888
              initialDelaySeconds: 1
              periodSeconds: 3
              timeoutSeconds: 5
            resources:
              limits:
                cpu: 500m
                memory: 512Mi
              requests:
                cpu: 500m
                memory: 512Mi
            volumeMounts:
            - name: log
              mountPath: /var/log
          - name: discovery
            image: "higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/pilot:2.0.7"
            args:
            - "discovery"
            - --monitoringAddr=:15014
            - --log_output_level=default:info
            - --domain
            - cluster.local
            - --keepaliveMaxServerConnectionAge
            - "30m"
            ports:
            - containerPort: 8080
              protocol: TCP
            - containerPort: 15010
              protocol: TCP
            - containerPort: 15017
              protocol: TCP
            readinessProbe:
              httpGet:
                path: /ready
                port: 8080
              initialDelaySeconds: 1
              periodSeconds: 3
              timeoutSeconds: 5
            env:
            - name: ENABLE_PUSH_ALL_MCP_CLUSTERS
              value: "true"
            - name: PILOT_ENABLE_LDS_CACHE
              value: "false"
            - name: PILOT_ENABLE_QUIC_LISTENERS
              value: "true"
            - name: VALIDATION_WEBHOOK_CONFIG_NAME
              value: ""
            - name: ISTIO_DUAL_STACK
              value: "false"
            - name: PILOT_ENABLE_HEADLESS_SERVICE_POD_LISTENERS
              value: "false"
            - name: PILOT_ENABLE_ALPN_FILTER
              value: "false"
            - name: ENABLE_OPTIMIZED_CONFIG_REBUILD
              value: "false"
            - name: PILOT_ENABLE_K8S_SELECT_WORKLOAD_ENTRIES
              value: "false"
            - name: HIGRESS_SYSTEM_NS
              value: "higress-system"
            - name: DEFAULT_UPSTREAM_CONCURRENCY_THRESHOLD
              value: "10000"
            - name: ISTIO_GPRC_MAXRECVMSGSIZE
              value: "104857600"
            - name: ENBALE_SCOPED_RDS
              value: "true"
            - name: ON_DEMAND_RDS
              value: "false"
            - name: HOST_RDS_MERGE_SUBSET
              value: "false"
            - name: PILOT_FILTER_GATEWAY_CLUSTER_CONFIG
              value: "true"
            - name: HIGRESS_CONTROLLER_SVC
              value: "127.0.0.1"
            - name: HIGRESS_CONTROLLER_PORT
              value: "15051"
            - name: REVISION
              value: "default"
            - name: JWT_POLICY
              value: third-party-jwt
            - name: PILOT_CERT_PROVIDER
              value: "istiod"
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
            - name: SERVICE_ACCOUNT
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: spec.serviceAccountName
            - name: KUBECONFIG
              value: /var/run/secrets/remote/config
            - name: PRIORITIZED_LEADER_ELECTION
              value: "false"
            - name: INJECT_ENABLED
              value: "false"
            - name: PILOT_ENABLE_CROSS_CLUSTER_WORKLOAD_ENTRY
              value: "false"
            - name: PILOT_ENABLE_METADATA_EXCHANGE
              value: "false"
            - name: PILOT_SCOPE_GATEWAY_TO_NAMESPACE
              value: "false"
            - name: VALIDATION_ENABLED
              value: "false"
            - name: PILOT_TRACE_SAMPLING
              value: "1"
            - name: PILOT_ENABLE_PROTOCOL_SNIFFING_FOR_OUTBOUND
              value: "true"
            - name: PILOT_ENABLE_PROTOCOL_SNIFFING_FOR_INBOUND
              value: "true"
            - name: ISTIOD_ADDR
              value: istiod.higress-system.svc:15012
            - name: PILOT_ENABLE_ANALYSIS
              value: "false"
            - name: CLUSTER_ID
              value: "Kubernetes"
            # HIGRESS_ENABLE_ISTIO_API is only used to restart the controller pod after the config change
            - name: HIGRESS_ENABLE_ISTIO_API
              value: "true"
            - name: PILOT_ENABLE_GATEWAY_API
              value: "false"
            - name: PILOT_ENABLE_ALPHA_GATEWAY_API
              value: "false"
            - name: PILOT_ENABLE_GATEWAY_API_STATUS
              value: "false"
            - name: PILOT_ENABLE_GATEWAY_API_DEPLOYMENT_CONTROLLER
              value: "false"
            - name: CUSTOM_CA_CERT_NAME
              value: "higress-ca-root-cert"
            resources:
              requests:
                cpu: 500m
                memory: 512Mi
            securityContext:
              readOnlyRootFilesystem: true
              runAsUser: 1337
              runAsGroup: 1337
              runAsNonRoot: true
              capabilities:
                drop:
                - ALL
            volumeMounts:
            - name: config
              mountPath: /etc/istio/config
            - name: istio-token
              mountPath: /var/run/secrets/tokens
              readOnly: true
            - name: local-certs
              mountPath: /var/run/secrets/istio-dns
            - name: cacerts
              mountPath: /etc/cacerts
              readOnly: true
            - name: istio-kubeconfig
              mountPath: /var/run/secrets/remote
              readOnly: true
        volumes:
        - name: log
          emptyDir: {}
        - name: config
          configMap:
            name: higress-config
        # Technically not needed on this pod - but it helps debugging/testing SDS
        # Should be removed after everything works.
        - emptyDir:
            medium: Memory
          name: local-certs
        - name: istio-token
          projected:
            sources:
              - serviceAccountToken:
                  audience: istio-ca
                  expirationSeconds: 43200
                  path: istio-token
        # Optional: user-generated root
        - name: cacerts
          secret:
            secretName: cacerts
            optional: true
        - name: istio-kubeconfig
          secret:
            secretName: istio-kubeconfig
            optional: true
  ---
  # Source: higress/charts/higress-core/templates/deployment.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: higress-gateway
    namespace: higress-system
    labels:
      helm.sh/chart: higress-core-2.0.7
      app: higress-gateway
      higress: higress-system-higress-gateway
      app.kubernetes.io/version: "2.0.7"
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: higress-gateway
    annotations:
      {}
  spec:
    replicas: 1
    selector:
      matchLabels:
        app: higress-gateway
        higress: higress-system-higress-gateway
    strategy:
      rollingUpdate:
        maxSurge: 100%
        maxUnavailable: 25%
    template:
      metadata:
        annotations:
          prometheus.io/path: /stats/prometheus
          prometheus.io/port: "15020"
          prometheus.io/scrape: "true"
          sidecar.istio.io/inject: "false"
        labels:
          sidecar.istio.io/inject: "false"
          app: higress-gateway
          higress: higress-system-higress-gateway
      spec:
        containers:
        - args:
          - proxy
          - router
          - --domain
          - $(POD_NAMESPACE).svc.cluster.local
          - --proxyLogLevel=warning
          - --proxyComponentLogLevel=misc:error
          - --log_output_level=all:info
          - --serviceCluster=higress-gateway
          env:
          - name: NODE_NAME
            valueFrom:
              fieldRef:
                apiVersion: v1
                fieldPath: spec.nodeName
          - name: POD_NAME
            valueFrom:
              fieldRef:
                apiVersion: v1
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                apiVersion: v1
                fieldPath: metadata.namespace
          - name: INSTANCE_IP
            valueFrom:
              fieldRef:
                apiVersion: v1
                fieldPath: status.podIP
          - name: HOST_IP
            valueFrom:
              fieldRef:
                apiVersion: v1
                fieldPath: status.hostIP
          - name: SERVICE_ACCOUNT
            valueFrom:
              fieldRef:
                apiVersion: v1
                fieldPath: spec.serviceAccountName
          - name: PROXY_XDS_VIA_AGENT
            value: "true"
          - name: ENABLE_INGRESS_GATEWAY_SDS
            value: "false"
          - name: JWT_POLICY
            value: third-party-jwt
          - name: ISTIO_META_HTTP10
            value: "1"
          - name: ISTIO_META_CLUSTER_ID
            value: Kubernetes
          - name: INSTANCE_NAME
            value: higress-gateway
          image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/gateway:2.0.7
          imagePullPolicy: IfNotPresent
          name: higress-gateway
          ports:
          - containerPort: 15020
            name: istio-prom
            protocol: TCP
          - containerPort: 15090
            name: http-envoy-prom
            protocol: TCP
          readinessProbe:
            failureThreshold: 30
            httpGet:
              path: /healthz/ready
              port: 15021
              scheme: HTTP
            initialDelaySeconds: 1
            periodSeconds: 2
            successThreshold: 1
            timeoutSeconds: 3
          resources:
            limits:
              cpu: "2"
              memory: 2Gi
            requests:
              cpu: "2"
              memory: 2Gi
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              add:
              - NET_BIND_SERVICE
              drop:
              - ALL
            runAsGroup: 1337
            runAsNonRoot: false
            runAsUser: 0
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          volumeMounts:
          - mountPath: /var/run/secrets/workload-spiffe-uds
            name: workload-socket
          - mountPath: /var/run/secrets/credential-uds
            name: credential-socket
          - mountPath: /var/run/secrets/workload-spiffe-credentials
            name: workload-certs
          - mountPath: /var/run/secrets/tokens
            name: istio-token
            readOnly: true
          - mountPath: /etc/istio/config
            name: config
          - mountPath: /var/run/secrets/istio
            name: higress-ca-root-cert
          - mountPath: /var/lib/istio/data
            name: istio-data
          - mountPath: /etc/istio/pod
            name: podinfo
          - mountPath: /etc/istio/proxy
            name: proxy-socket
        dnsPolicy: ClusterFirst
        restartPolicy: Always
        schedulerName: default-scheduler
        securityContext: {}
        serviceAccount: higress-gateway
        serviceAccountName: higress-gateway
        terminationGracePeriodSeconds: 30
        volumes:
        - emptyDir: {}
          name: workload-socket
        - emptyDir: {}
          name: credential-socket
        - emptyDir: {}
          name: workload-certs
        - name: istio-token
          projected:
            defaultMode: 420
            sources:
            - serviceAccountToken:
                audience: istio-ca
                expirationSeconds: 43200
                path: istio-token
        - configMap:
            defaultMode: 420
            name: higress-ca-root-cert
          name: higress-ca-root-cert
        - configMap:
            defaultMode: 420
            name: higress-config
          name: config
        - emptyDir: {}
          name: istio-data
        - emptyDir: {}
          name: proxy-socket
        - downwardAPI:
            defaultMode: 420
            items:
            - fieldRef:
                apiVersion: v1
                fieldPath: metadata.labels
              path: labels
            - fieldRef:
                apiVersion: v1
                fieldPath: metadata.annotations
              path: annotations
            - path: cpu-request
              resourceFieldRef:
                containerName: higress-gateway
                divisor: 1m
                resource: requests.cpu
            - path: cpu-limit
              resourceFieldRef:
                containerName: higress-gateway
                divisor: 1m
                resource: limits.cpu
          name: podinfo
  ---
  # Source: higress/charts/higress-core/templates/ingressclass.yaml
  apiVersion: networking.k8s.io/v1
  kind: IngressClass
  metadata:
    name: higress
  spec:
    controller: higress.io/higress-controller
  ---
  # Source: higress/charts/higress-core/templates/fallback-envoyfilter.yaml
  apiVersion: networking.istio.io/v1alpha3
  kind: EnvoyFilter
  metadata:
    name: higress-gateway-global-custom-response
    namespace: higress-system
    labels:
      helm.sh/chart: higress-core-2.0.7
      app: higress-gateway
      higress: higress-system-higress-gateway
      app.kubernetes.io/version: "2.0.7"
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: higress-gateway
  spec:
    configPatches:
      - applyTo: HTTP_FILTER
        match:
          context: GATEWAY
          listener:
            filterChain:
              filter:
                name: envoy.filters.network.http_connection_manager
        patch:
          operation: INSERT_FIRST
          value:
            name: envoy.filters.http.custom_response
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.custom_response.v3.CustomResponse

`