You are browsing a read-only backup copy of Wikitech. The live site can be found at

User:Accraze/MachineLearning/Local Kserve: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
(→‎Minikube: trying insecure-registry flag)
m (fix dbl rdr)
(6 intermediate revisions by one other user not shown)
Line 1: Line 1:
== Summary ==
#REDIRECT [[Machine Learning/LiftWing/ML-Sandbox/Configuration]]
This page is a guide for installing the KServe stack locally using WMF tools and images. The install steps diverge from the official KServe quick_install script in order to run on WMF infrastructure. All upstream changes to YAML configs were first published in the KServe chart's README for the [[gerrit:admin/repos/operations/deployment-charts|deployment-charts]] repository. In deployment-charts/custom_deploy.d/istio/ml-serve there is the config.yaml that we apply in prod.
== Minikube ==
We are running a small cluster using Minikube, which can be installed with the following command: <syntaxhighlight lang="bash">
curl -LO
sudo install minikube-linux-amd64 /usr/local/bin/minikube
To match production, we want to make sure we set our k8s version to v.1.16.15: <syntaxhighlight lang="bash">
minikube start --kubernetes-version=v1.16.15 --insecure-registry=""
You will also need to install kubectl, or you can use the one provided by minikube with an alias: <syntaxhighlight lang="bash">
alias kubectl="minikube kubectl --"
== Istio ==
Istio is installed using the istioctl package, which has been added to the WMF APT repository, you can use it (, debian buster).
See: , we want to install Istio 1.9.5 (istioctl: 1.9.5-1)
For Wikimedia servers and Cloud VPS instances, the repositories are automatically configured via Puppet. You can install it as follows
<syntaxhighlight lang="bash">
sudo apt install istioctl
Now we need to create the istio-system namespace:<syntaxhighlight lang="bash">
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Namespace
  name: istio-system
    istio-injection: disabled
Next you will need to create a file called istio-minimal-operator.yaml: <syntaxhighlight lang="yaml">
kind: IstioOperator
        autoInject: disabled
      useMCP: false
      # The third-party-jwt is not enabled on all k8s.
      # See:
      jwtPolicy: first-party-jwt
    accessLogFile: /dev/stdout
      enabled: true
      - name: istio-ingressgateway
        enabled: true
Next, you can apply the manifest using istioctl: <syntaxhighlight lang="bash">
/usr/bin/istioctl-1.9.5 manifest apply -f istio-minimal-operator.yaml -y
== Knative ==
We are currently running Knative Serving v0.18.1.
First let's install the Knative serving-crds.yaml. The CRDs are copied from upstream:
We have them included in our deployment-charts repo:
You can install using the following command: <syntaxhighlight lang="bash">
kubectl apply -f crds.yaml
For knative gateway there is a specific setting in a config-map to change, otherwise it won't work.
=== Images ===
* Webhook:
* Queue:
* Controller:
* Autoscaler:
* Activator:
* Net-istio webhook:
* Net-istio controller:
== KServe ==
=== Images ===
* KServe agent:
* Kserve controller:
* KServe storage-initializer:

Latest revision as of 21:52, 12 March 2022