You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

User:JMeybohm/Kubernetes/Ingress

From Wikitech-static
< User:JMeybohm
Revision as of 13:54, 6 September 2021 by imported>JMeybohm (→‎TLS Stuff / cert-manager)
Jump to navigation Jump to search

SIG-Networks new standard for ingress etc. (still v1alpha unfortunately):

Ingress stuff from Istio (k8s only and backed by istio CRD's):


Istio 1.8 latest version supported with k8s 1.16

Istio 1.11 tested with k8s 1.16 but not supported

ml currently uses istio 1.9 which is tested but not supported on k8s 1.16

https://istio.io/latest/docs/releases/supported-releases/#support-status-of-istio-releases


IstioOperator Options: https://istio.io/v1.9/docs/reference/config/istio.operator.v1alpha1/

istioctl builtin helm charts: https://github.com/istio/istio/tree/1.9.5/manifests/charts

https://phabricator.wikimedia.org/P17224

TLS Stuff / cert-manager

There is a cert-manager issuer for cfssl: https://github.com/OpenSource-THG/cfssl-issuer (does not look very maintained/active)

But it might be easier (and less dependencies) to generate a new intermediate and use that with the ca-issuer that's build in to cert-manager https://cert-manager.io/docs/configuration/ca/

Could we have namespaces create "kind: Certificate" objects, let cert-manager create the corresponding "kind: Secret" (in the same namespace) and sync (with something like https://github.com/kubeops/kubed)those secrets to the istio-system namespace (for ingressgateway to be able to pick them up)?


Figure out pilotCertProvider setting, https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/ and https://cert-manager.io/docs/usage/kube-csr/

Kubernetes Ingress

By using only the kubernetes ingress resource it is possible to create a fan out ingress and route "foo.example.com/a/*" to a service in namespace "a" while routing "foo.example.com/b/*" to a service in namespace "b". This is done by simply creating a Ingress for "foo.example.com" with a corresponding path matcher.

When the same host and path is specified in two ingress resources, the first one (created) wins.

This means it is not possible to break existing ingress objects while it is possible to "hijack" subpaths of a domain by creating another ingress (if the first one uses subpath matching rather than "/*" ofc.). If a subpath ingress is created first (e.g. "path: /one/*" ad afterwards created match all ingress ("path: /*") is not able to take over "/one/*". But this is pretty dangerous as de-deploying might lead to a different creation order and yield a different result

Ingress gateway as DaemonSet

May spare a network hop and increase resiliency as the gateway would run on every node then. Should be possible with overlays https://istio.io/v1.9/docs/reference/config/istio.operator.v1alpha1/#K8sObjectOverlay:?

components:
  ingressGateways:
    - name: foo
      overlays:
        - kind: Deployment
          name: foo
          patches:
            - path: kind
              value: Daemonset