You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

User:Accraze/MachineLearning/Local Kserve: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Accraze
(→‎Minikube: trying insecure-registry flag)
imported>Quiddity
m (fix dbl rdr)
 
(6 intermediate revisions by one other user not shown)
Line 1: Line 1:
== Summary ==
#REDIRECT [[Machine Learning/LiftWing/ML-Sandbox/Configuration]]
This page is a guide for installing the KServe stack locally using WMF tools and images. The install steps diverge from the official KServe quick_install script in order to run on WMF infrastructure. All upstream changes to YAML configs were first published in the KServe chart's README for the [[gerrit:admin/repos/operations/deployment-charts|deployment-charts]] repository. In deployment-charts/custom_deploy.d/istio/ml-serve there is the config.yaml that we apply in prod.
== Minikube ==
We are running a small cluster using Minikube, which can be installed with the following command: <syntaxhighlight lang="bash">
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube
</syntaxhighlight>
 
To match production, we want to make sure we set our k8s version to v.1.16.15: <syntaxhighlight lang="bash">
minikube start --kubernetes-version=v1.16.15 --insecure-registry="docker-registry.wikimedia.org:443"
</syntaxhighlight>
 
You will also need to install kubectl, or you can use the one provided by minikube with an alias: <syntaxhighlight lang="bash">
alias kubectl="minikube kubectl --"
</syntaxhighlight>
 
== Istio ==
Istio is installed using the istioctl package, which has been added to the WMF APT repository, you can use it (https://wikitech.wikimedia.org/wiki/APT_repository, debian buster).
See: https://apt-browser.toolforge.org/buster-wikimedia/main/ , we want to install Istio 1.9.5 (istioctl: 1.9.5-1)
 
For Wikimedia servers and Cloud VPS instances, the repositories are automatically configured via Puppet. You can install it as follows
<syntaxhighlight lang="bash">
sudo apt install istioctl
</syntaxhighlight>
 
Now we need to create the istio-system namespace:<syntaxhighlight lang="bash">
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Namespace
metadata:
  name: istio-system
  labels:
    istio-injection: disabled
EOF
</syntaxhighlight>
 
Next you will need to create a file called istio-minimal-operator.yaml: <syntaxhighlight lang="yaml">
apiVersion: install.istio.io/v1beta1
kind: IstioOperator
spec:
  values:
    global:
      proxy:
        autoInject: disabled
      useMCP: false
      # The third-party-jwt is not enabled on all k8s.
      # See: https://istio.io/docs/ops/best-practices/security/#configure-third-party-service-account-tokens
      jwtPolicy: first-party-jwt
 
  meshConfig:
    accessLogFile: /dev/stdout
 
  addonComponents:
    pilot:
      enabled: true
 
  components:
    ingressGateways:
      - name: istio-ingressgateway
        enabled: true
</syntaxhighlight>
 
Next, you can apply the manifest using istioctl: <syntaxhighlight lang="bash">
/usr/bin/istioctl-1.9.5 manifest apply -f istio-minimal-operator.yaml -y
</syntaxhighlight>
 
== Knative ==
We are currently running Knative Serving v0.18.1.
First let's install the Knative serving-crds.yaml. The CRDs are copied from upstream:
https://github.com/knative/serving/releases/download/v0.18.1/serving-crds.yaml
 
We have them included in our deployment-charts repo:
https://gerrit.wikimedia.org/r/plugins/gitiles/operations/deployment-charts/+/refs/heads/master/charts/knative-serving-crds/templates/crds.yaml
 
You can install using the following command: <syntaxhighlight lang="bash">
kubectl apply -f crds.yaml
</syntaxhighlight>
 
For knative gateway there is a specific setting in a config-map to change, otherwise it won't work.
 
=== Images ===
* Webhook: https://docker-registry.wikimedia.org/knative-serving-webhook/tags/
* Queue: https://docker-registry.wikimedia.org/knative-serving-queue/tags/
* Controller: https://docker-registry.wikimedia.org/knative-serving-controller/tags/
* Autoscaler: https://docker-registry.wikimedia.org/knative-serving-autoscaler/tags/
* Activator: https://docker-registry.wikimedia.org/knative-serving-activator/tags/
* Net-istio webhook: https://docker-registry.wikimedia.org/knative-net-istio-webhook/tags/
* Net-istio controller: https://docker-registry.wikimedia.org/knative-net-istio-controller/tags/
 
== KServe ==
=== Images ===
* KServe agent: https://docker-registry.wikimedia.org/kserve-agent/tags/
* Kserve controller: https://docker-registry.wikimedia.org/kserve-controller/tags/
* KServe storage-initializer: https://docker-registry.wikimedia.org/kserve-storage-initializer/tags/

Latest revision as of 21:52, 12 March 2022

Redirect to:

Retrieved from "https://wikitech-static.wikimedia.org/w/index.php?title=User:Accraze/MachineLearning/Local_Kserve&oldid=587307"