You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Machine Learning/LiftWing/Inference Services/Production Image Development: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Kevin Bazira
(→‎Test: fixed typo)
imported>AikoChou
(Internal Wikitech links)
Line 1: Line 1:
= Summary =
= Summary =
This is a guide to developing model-servers and module images using WMF release infrastructure. For example purposes, we will be creating a production image for the <tt>draftquality</tt> model-server (a.k.a. predictor)
This is a guide to developing model-servers and module images using WMF release infrastructure. For example purposes, we will be creating a production image for the '''draftquality''' model-server (a.k.a. predictor).


= Blubber =
= Blubber =
We create Dockerfiles using an abstraction for container build configurations called [https://wikitech.wikimedia.org/wiki/Blubber Blubber]. If you want to publish a production-ready image to the [https://docker-registry.wikimedia.org/ WMF Docker Registry], you will need to develop a Blubberfile that can be used in the [https://wikitech.wikimedia.org/wiki/Deployment_pipeline Wikimedia Deployment Pipeline].
We create Dockerfiles using an abstraction for container build configurations called [[Blubber]]. If you want to publish a production-ready image to the [https://docker-registry.wikimedia.org/ WMF Docker Registry], you will need to develop a Blubberfile that can be used in the [[Deployment pipeline|Wikimedia Deployment Pipeline]].


== Developing a Blubberfile ==
== Developing a Blubberfile ==
Let&rsquo;s start our <tt>blubber.yaml</tt> file with some basic declarations. We want to use version v4 of the Blubber spec. Also we define our base image to build off of (<tt>buster</tt> in this case). Also we define our image to run insecurely in order to write to filesystem/cache during testing.
Let&rsquo;s start our [https://github.com/wikimedia/machinelearning-liftwing-inference-services/blob/main/.pipeline/draftquality/blubber.yaml blubber.yaml] file with some basic declarations. We want to use version v4 of the Blubber spec. Also we define our base image to build off of (<code>buster</code> in this case). Also we define our image to run insecurely in order to write to filesystem/cache during testing.


<syntaxhighlight lang="yaml">version: v4
<syntaxhighlight lang="yaml">version: v4
Line 14: Line 14:
</syntaxhighlight>
</syntaxhighlight>


Next we define our <tt>WORKDIR</tt> within the generated Dockerfile.
Next we define our working directory within the generated Dockerfile.


<syntaxhighlight lang="yaml">lives:
<syntaxhighlight lang="yaml">lives:
   in: /srv/draftquality
   in: /srv/draftquality
</syntaxhighlight>
</syntaxhighlight>
=== Variants ===
=== Variants ===
Blubber uses the concept of variants to create multi-stage builds. You can have any number of variants, although it is a good idea to have atleast a <tt>test</tt> variant and also a <tt>production</tt> variant. You can also create a <tt>build</tt> variant that shares the common configurations that are needed by other variants.
Blubber uses the concept of variants to create multi-stage builds. You can have any number of variants, although it is a good idea to have at least a <code>test</code> variant and also a <code>production</code> variant. You can also create a <code>build</code> variant that shares the common configurations that are needed by other variants.


==== Build ====
==== Build ====
Line 31: Line 29:
     python:
     python:
       version: python3.7
       version: python3.7
       requirements: [model-server/requirements.txt]
       requirements: [revscoring/draftquality/model-server/requirements.txt, python/requirements.txt]
</syntaxhighlight>
</syntaxhighlight>


Line 49: Line 47:
</syntaxhighlight>
</syntaxhighlight>


We will also need to run some commands to finish up installing all the required assets, which we will do using <tt>builder</tt> command.
We will also need to run some commands to finish up installing all the required assets, which we will do using '''builder''' command.


<syntaxhighlight lang="yaml">    builder:
<syntaxhighlight lang="yaml">    builder:
Line 56: Line 54:
         "nltk.downloader", "omw", "sentiwordnet", "stopwords", "wordnet"]
         "nltk.downloader", "omw", "sentiwordnet", "stopwords", "wordnet"]
</syntaxhighlight>
</syntaxhighlight>


==== Production ====
==== Production ====
Now let&rsquo;s create a <tt>production</tt> variant that relies on packages installed by the <tt>build</tt> variant. We will use the copies key and copy over the model-server source code from our local filesystem, the nltk data (stopwords) and also all other dependencies installed via pip.
Now let&rsquo;s create a <code>production</code> variant that relies on packages installed by the <code>build</code> variant. We will use the '''copies''' key and copy over the model-server source code and the files in the shared Python directory from our local filesystem, the nltk data (stopwords) and also all other dependencies installed via pip.


<syntaxhighlight lang="yaml"> production:
<syntaxhighlight lang="yaml"> production:
     copies:
     copies:
       - from: local
       - from: local
         source: model-server
         source: revscoring/draftquality/model-server
         destination: model-server
         destination: model-server
      - from: local
        source: python/*.py
        destination: model-server/
       - from: build
       - from: build
         source: /home/somebody/nltk_data
         source: /home/somebody/nltk_data
Line 74: Line 74:
</syntaxhighlight>
</syntaxhighlight>


We define the Python requirements for the production image
We define the Python requirements for the production image and which packages we wish to install from APT for image.


<syntaxhighlight lang="yaml">   python:
<syntaxhighlight lang="yaml">   python:
       version: python3.7
       version: python3.7
       use-system-flag: false
       use-system-flag: false
</syntaxhighlight>
</syntaxhighlight>


<syntaxhighlight lang="yaml">   apt:
<syntaxhighlight lang="yaml">   apt:
       packages:
       packages:
         - python3
         - python3
Line 134: Line 134:
<syntaxhighlight lang="yaml">    entrypoint: ["python3",  "model-server/model.py"]
<syntaxhighlight lang="yaml">    entrypoint: ["python3",  "model-server/model.py"]
</syntaxhighlight>
</syntaxhighlight>


==== Test ====
==== Test ====
The <tt>test</tt> variant is going to be similar to the other two however it is much more light-weight. This variant simply runs some tests via <tt>tox</tt> (which is configured in the tox.ini in app)
The <code>test</code> variant is going to be similar to the other two however it is much more light-weight. This variant simply runs some tests via '''tox''' (which is configured in the tox.ini in app)


Let&rsquo;s start by defining the packages we need from APT
Let&rsquo;s start by defining the packages we need from APT


<syntaxhighlight lang="yaml"> test:
<syntaxhighlight lang="yaml"> test:
     apt:
     apt:
       packages:
       packages:
Line 148: Line 147:
</syntaxhighlight>
</syntaxhighlight>


Next, let&rsquo;s copy over the source files from our local filesystem
Next, let's copy over the source files from our local filesystem


<syntaxhighlight lang="yaml">    copies:
<syntaxhighlight lang="yaml">    copies:
       - from: local
       - from: local
         source: model-server
         source: revscoring/draftquality/model-server
         destination: model-server
         destination: model-server
</syntaxhighlight>
</syntaxhighlight>
Line 160: Line 159:
<syntaxhighlight lang="yaml">    python:
<syntaxhighlight lang="yaml">    python:
       version: python3.7
       version: python3.7
       requirements: [model-server/requirements-test.txt]
       requirements: [revscoring/draftquality/model-server/requirements-test.txt]
       use-system-flag: false
       use-system-flag: false
</syntaxhighlight>
</syntaxhighlight>


Finally, let&rsquo;s invoke tox to run our tests when the image is run as a container:
Finally, let's invoke tox to run our tests when the image is run as a container:


<syntaxhighlight lang="yaml">    entrypoint: ["tox", "-c", "model-server/tox.ini"]
<syntaxhighlight lang="yaml">    entrypoint: ["tox", "-c", "model-server/tox.ini"]
</syntaxhighlight>
</syntaxhighlight>
The complete blubberfile for the draftquality model-server can be found [https://github.com/wikimedia/machinelearning-liftwing-inference-services/blob/main/.pipeline/draftquality/blubber.yaml here].




== Testing your Blubberfile ==
== Testing your Blubberfile ==
Now that we have created a Blubberfile, let&rsquo;s test it using the blubberoid service, to ensure that our config generates a working Dockerfile. For more info see: https://wikitech.wikimedia.org/wiki/Blubber/Download#Blubber_as_a_(micro)Service
Now that we have created a Blubberfile, let's test it using the [[Blubber/Download|Blubber]] service, to ensure that our config generates a working Dockerfile.
 
Simply start using this Bash function to get the blubber. This will ensure you're using the same version that is deployed for use by Wikimedia CI.


<syntaxhighlight lang="sh">curl -s "https://blubberoid.wikimedia.org/v1/production" -H 'content-type: application/yaml' --data-binary @".pipeline/revscoring/draftquality/blubber.yaml" >  Dockerfile
<syntaxhighlight lang="sh">blubber() {
  if [ $# -lt 2 ]; then
    echo 'Usage: blubber config.yaml variant'
    return 1
  fi
  curl -s -H 'content-type: application/yaml' --data-binary @"$1" https://blubberoid.wikimedia.org/v1/"$2"
}
</syntaxhighlight>
</syntaxhighlight>


In the above command, we are calling the blubberoid service and requesting a Dockerfile based on the &ldquo;production&rdquo; variant. We pipe the blubberoid response into a file called Dockerfile in our current working directory.
Then we can use the following command to create a Dockerfile based on the <code>production</code> variant. We pipe the blubber response into a file called Dockerfile in our current working directory.<syntaxhighlight lang="bash">
blubber .pipeline/draftquality/blubber.yaml production > Dockerfile
</syntaxhighlight>Next, we can build our image locally with the following command:


Next, we can build our image locally with the following command:
<syntaxhighlight lang="sh">cat Dockerfile | docker build -t accraze/draftquality-prod:v1 -f - .
</syntaxhighlight>


<syntaxhighlight lang="sh">cat Dockerfile | docker build -t accraze/draftquality-prod:v1 -f - revscoring/draftquality
Here, we pipe our newly-generated Dockerfile into the context of our Docker build, tag it and use the current directory as the build context (as specified by the <code>.</code>). After the build process completes, you should now have a production image that reflects the configuration defined in our Blubberfile.
</syntaxhighlight>


Here, we pipe our newly-generated Dockerfile into the context of our Docker build, tag it and point to the directory where the source files for the image are located (<tt>revscoring/draftquality</tt>). After the build process completes, you should now have a production images that reflects the configuration defined in our Blubberfile.




Line 190: Line 199:
Once you are happy with the image being generated from your Blubberfile, it&rsquo;s time to configure a pipeline to build the image, run tests and publish the production-ready images to the WMF Docker Registry.
Once you are happy with the image being generated from your Blubberfile, it&rsquo;s time to configure a pipeline to build the image, run tests and publish the production-ready images to the WMF Docker Registry.


In our inference-services repo, all pipelines are configured via PipelineLib in the <tt>.pipeline/config.yaml</tt> file. We will want to configure two pipelines for the above image. One to build our images then run tests, and one to publish our production image after the code change is merged.
In our inference-services repo, all pipelines are configured via PipelineLib in the <code>.pipeline/config.yaml</code> file. We will want to configure two pipelines for the above image. One to build our images then run tests, and one to publish our production image after the code change is merged.


<syntaxhighlight lang="yaml">
<syntaxhighlight lang="yaml">
   draftquality:
   draftquality:
    directory: revscoring/draftquality
     stages:
     stages:
       - name: run-test
       - name: run-test
Line 204: Line 212:
   draftquality-publish:
   draftquality-publish:
     blubberfile: draftquality/blubber.yaml
     blubberfile: draftquality/blubber.yaml
    directory: revscoring/draftquality
     stages:
     stages:
       - name: publish
       - name: publish

Revision as of 11:26, 17 August 2022

Summary

This is a guide to developing model-servers and module images using WMF release infrastructure. For example purposes, we will be creating a production image for the draftquality model-server (a.k.a. predictor).

Blubber

We create Dockerfiles using an abstraction for container build configurations called Blubber. If you want to publish a production-ready image to the WMF Docker Registry, you will need to develop a Blubberfile that can be used in the Wikimedia Deployment Pipeline.

Developing a Blubberfile

Let’s start our blubber.yaml file with some basic declarations. We want to use version v4 of the Blubber spec. Also we define our base image to build off of (buster in this case). Also we define our image to run insecurely in order to write to filesystem/cache during testing.

version: v4
base: docker-registry.wikimedia.org/buster:20220109
runs:
  insecurely: true

Next we define our working directory within the generated Dockerfile.

lives:
  in: /srv/draftquality

Variants

Blubber uses the concept of variants to create multi-stage builds. You can have any number of variants, although it is a good idea to have at least a test variant and also a production variant. You can also create a build variant that shares the common configurations that are needed by other variants.

Build

Let’s start with our build variant. We need to declare the Python version we want to run and also point to where the requirement files are located.

variants:
  build:
    python:
      version: python3.7
      requirements: [revscoring/draftquality/model-server/requirements.txt, python/requirements.txt]

Next, we need to specify which packages we wish to install from APT for image.

    apt:
      packages:
        - python3-pip
        - python3-dev
        - python3-setuptools
        - g++
        - git
        - gfortran
        - liblapack-dev
        - libopenblas-dev
        - libenchant1c2a

We will also need to run some commands to finish up installing all the required assets, which we will do using builder command.

    builder:
      # FIXME: path hack - see: https://phabricator.wikimedia.org/T267685
      command: ["PYTHONPATH=/opt/lib/python/site-packages", "python3.7", "-m",
        "nltk.downloader", "omw", "sentiwordnet", "stopwords", "wordnet"]

Production

Now let’s create a production variant that relies on packages installed by the build variant. We will use the copies key and copy over the model-server source code and the files in the shared Python directory from our local filesystem, the nltk data (stopwords) and also all other dependencies installed via pip.

 production:
    copies:
      - from: local
        source: revscoring/draftquality/model-server
        destination: model-server
      - from: local
        source: python/*.py
        destination: model-server/
      - from: build
        source: /home/somebody/nltk_data
        destination: /home/somebody/nltk_data
      - from: build
        source: /opt/lib/python/site-packages
        destination: /opt/lib/python/site-packages

We define the Python requirements for the production image and which packages we wish to install from APT for image.

   python:
      version: python3.7
      use-system-flag: false
   apt:
      packages:
        - python3
        - liblapack3
        - libopenblas-base
        - libenchant1c2a
        - aspell-ar
        - aspell-bn
        - aspell-el
        - hunspell-id
        - aspell-is
        - aspell-pl
        - aspell-ro
        - aspell-sv
        - aspell-ta
        - aspell-uk
        - myspell-cs
        - myspell-de-at
        - myspell-de-ch
        - myspell-de-de
        - myspell-es
        - myspell-et
        - myspell-fa
        - myspell-fr
        - myspell-he
        - myspell-hr
        - myspell-hu
        - myspell-lv
        - myspell-nb
        - myspell-nl
        - myspell-pt-pt
        - myspell-pt-br
        - myspell-ru
        - myspell-hr
        - hunspell-bs
        - hunspell-ca
        - hunspell-en-au
        - hunspell-en-us
        - hunspell-en-gb
        - hunspell-eu
        - hunspell-gl
        - hunspell-it
        - hunspell-hi
        - hunspell-sr
        - hunspell-vi
        - voikko-fi
        - wmf-certificates

Lastly, we define the entrypoint to our application when the image is run as a container.

    entrypoint: ["python3",  "model-server/model.py"]

Test

The test variant is going to be similar to the other two however it is much more light-weight. This variant simply runs some tests via tox (which is configured in the tox.ini in app)

Let’s start by defining the packages we need from APT

 test:
    apt:
      packages:
        - python3-pip
        - python3-setuptools

Next, let's copy over the source files from our local filesystem

    copies:
      - from: local
        source: revscoring/draftquality/model-server
        destination: model-server

Let’s define our Python requirements for testing:

    python:
      version: python3.7
      requirements: [revscoring/draftquality/model-server/requirements-test.txt]
      use-system-flag: false

Finally, let's invoke tox to run our tests when the image is run as a container:

    entrypoint: ["tox", "-c", "model-server/tox.ini"]

The complete blubberfile for the draftquality model-server can be found here.


Testing your Blubberfile

Now that we have created a Blubberfile, let's test it using the Blubber service, to ensure that our config generates a working Dockerfile.

Simply start using this Bash function to get the blubber. This will ensure you're using the same version that is deployed for use by Wikimedia CI.

blubber() {
  if [ $# -lt 2 ]; then
    echo 'Usage: blubber config.yaml variant'
    return 1
  fi
  curl -s -H 'content-type: application/yaml' --data-binary @"$1" https://blubberoid.wikimedia.org/v1/"$2"
}

Then we can use the following command to create a Dockerfile based on the production variant. We pipe the blubber response into a file called Dockerfile in our current working directory.

blubber .pipeline/draftquality/blubber.yaml production > Dockerfile

Next, we can build our image locally with the following command:

cat Dockerfile | docker build -t accraze/draftquality-prod:v1 -f - .

Here, we pipe our newly-generated Dockerfile into the context of our Docker build, tag it and use the current directory as the build context (as specified by the .). After the build process completes, you should now have a production image that reflects the configuration defined in our Blubberfile.


Pipeline Configuration

Once you are happy with the image being generated from your Blubberfile, it’s time to configure a pipeline to build the image, run tests and publish the production-ready images to the WMF Docker Registry.

In our inference-services repo, all pipelines are configured via PipelineLib in the .pipeline/config.yaml file. We will want to configure two pipelines for the above image. One to build our images then run tests, and one to publish our production image after the code change is merged.

  draftquality:
    stages:
      - name: run-test
        build: test
        run: true
      - name: production
        build: production

  draftquality-publish:
    blubberfile: draftquality/blubber.yaml
    stages:
      - name: publish
        build: production
        publish:
          image:
            name: '${setup.project}-draftquality'
            tags: [stable]