You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Machine Learning/LiftWing/Inference Services/Production Image Development: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Accraze
 
imported>Kevin Bazira
(→‎Test: fixed typo)
Line 137: Line 137:


==== Test ====
==== Test ====
The <tt>test</tt> variant is going to be similar to the other twok however it is much more light-weight. This variant simply runs some tests via <tt>tox</tt> (which is configured in the tox.ini in app)
The <tt>test</tt> variant is going to be similar to the other two however it is much more light-weight. This variant simply runs some tests via <tt>tox</tt> (which is configured in the tox.ini in app)


Let&rsquo;s start by defining the packages we need from APT
Let&rsquo;s start by defining the packages we need from APT

Revision as of 07:56, 9 March 2022

Summary

This is a guide to developing model-servers and module images using WMF release infrastructure. For example purposes, we will be creating a production image for the draftquality model-server (a.k.a. predictor)

Blubber

We create Dockerfiles using an abstraction for container build configurations called Blubber. If you want to publish a production-ready image to the WMF Docker Registry, you will need to develop a Blubberfile that can be used in the Wikimedia Deployment Pipeline.

Developing a Blubberfile

Let’s start our blubber.yaml file with some basic declarations. We want to use version v4 of the Blubber spec. Also we define our base image to build off of (buster in this case). Also we define our image to run insecurely in order to write to filesystem/cache during testing.

version: v4
base: docker-registry.wikimedia.org/buster:20220109
runs:
  insecurely: true

Next we define our WORKDIR within the generated Dockerfile.

lives:
  in: /srv/draftquality


Variants

Blubber uses the concept of variants to create multi-stage builds. You can have any number of variants, although it is a good idea to have atleast a test variant and also a production variant. You can also create a build variant that shares the common configurations that are needed by other variants.

Build

Let’s start with our build variant. We need to declare the Python version we want to run and also point to where the requirement files are located.

variants:
  build:
    python:
      version: python3.7
      requirements: [model-server/requirements.txt]

Next, we need to specify which packages we wish to install from APT for image.

    apt:
      packages:
        - python3-pip
        - python3-dev
        - python3-setuptools
        - g++
        - git
        - gfortran
        - liblapack-dev
        - libopenblas-dev
        - libenchant1c2a

We will also need to run some commands to finish up installing all the required assets, which we will do using builder command.

    builder:
      # FIXME: path hack - see: https://phabricator.wikimedia.org/T267685
      command: ["PYTHONPATH=/opt/lib/python/site-packages", "python3.7", "-m",
        "nltk.downloader", "omw", "sentiwordnet", "stopwords", "wordnet"]


Production

Now let’s create a production variant that relies on packages installed by the build variant. We will use the copies key and copy over the model-server source code from our local filesystem, the nltk data (stopwords) and also all other dependencies installed via pip.

  production:
    copies:
      - from: local
        source: model-server
        destination: model-server
      - from: build
        source: /home/somebody/nltk_data
        destination: /home/somebody/nltk_data
      - from: build
        source: /opt/lib/python/site-packages
        destination: /opt/lib/python/site-packages

We define the Python requirements for the production image

    python:
      version: python3.7
      use-system-flag: false
    apt:
      packages:
        - python3
        - liblapack3
        - libopenblas-base
        - libenchant1c2a
        - aspell-ar
        - aspell-bn
        - aspell-el
        - hunspell-id
        - aspell-is
        - aspell-pl
        - aspell-ro
        - aspell-sv
        - aspell-ta
        - aspell-uk
        - myspell-cs
        - myspell-de-at
        - myspell-de-ch
        - myspell-de-de
        - myspell-es
        - myspell-et
        - myspell-fa
        - myspell-fr
        - myspell-he
        - myspell-hr
        - myspell-hu
        - myspell-lv
        - myspell-nb
        - myspell-nl
        - myspell-pt-pt
        - myspell-pt-br
        - myspell-ru
        - myspell-hr
        - hunspell-bs
        - hunspell-ca
        - hunspell-en-au
        - hunspell-en-us
        - hunspell-en-gb
        - hunspell-eu
        - hunspell-gl
        - hunspell-it
        - hunspell-hi
        - hunspell-sr
        - hunspell-vi
        - voikko-fi
        - wmf-certificates

Lastly, we define the entrypoint to our application when the image is run as a container.

    entrypoint: ["python3",  "model-server/model.py"]


Test

The test variant is going to be similar to the other two however it is much more light-weight. This variant simply runs some tests via tox (which is configured in the tox.ini in app)

Let’s start by defining the packages we need from APT

  test:
    apt:
      packages:
        - python3-pip
        - python3-setuptools

Next, let’s copy over the source files from our local filesystem

    copies:
      - from: local
        source: model-server
        destination: model-server

Let’s define our Python requirements for testing:

    python:
      version: python3.7
      requirements: [model-server/requirements-test.txt]
      use-system-flag: false

Finally, let’s invoke tox to run our tests when the image is run as a container:

    entrypoint: ["tox", "-c", "model-server/tox.ini"]


Testing your Blubberfile

Now that we have created a Blubberfile, let’s test it using the blubberoid service, to ensure that our config generates a working Dockerfile. For more info see: https://wikitech.wikimedia.org/wiki/Blubber/Download#Blubber_as_a_(micro)Service

curl -s "https://blubberoid.wikimedia.org/v1/production" -H 'content-type: application/yaml' --data-binary @".pipeline/revscoring/draftquality/blubber.yaml" >  Dockerfile

In the above command, we are calling the blubberoid service and requesting a Dockerfile based on the “production” variant. We pipe the blubberoid response into a file called Dockerfile in our current working directory.

Next, we can build our image locally with the following command:

cat Dockerfile | docker build -t accraze/draftquality-prod:v1 -f - revscoring/draftquality

Here, we pipe our newly-generated Dockerfile into the context of our Docker build, tag it and point to the directory where the source files for the image are located (revscoring/draftquality). After the build process completes, you should now have a production images that reflects the configuration defined in our Blubberfile.


Pipeline Configuration

Once you are happy with the image being generated from your Blubberfile, it’s time to configure a pipeline to build the image, run tests and publish the production-ready images to the WMF Docker Registry.

In our inference-services repo, all pipelines are configured via PipelineLib in the .pipeline/config.yaml file. We will want to configure two pipelines for the above image. One to build our images then run tests, and one to publish our production image after the code change is merged.

  draftquality:
    directory: revscoring/draftquality
    stages:
      - name: run-test
        build: test
        run: true
      - name: production
        build: production

  draftquality-publish:
    blubberfile: draftquality/blubber.yaml
    directory: revscoring/draftquality
    stages:
      - name: publish
        build: production
        publish:
          image:
            name: '${setup.project}-draftquality'
            tags: [stable]