You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Debian Packaging: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Effie Mouzeli
No edit summary
imported>Effie Mouzeli
Line 1: Line 1:
{{TOC limit|4}}
== Rebuilding a package ==
== Rebuilding a package ==


Process to simple repackage a debian package and push it to wmf apt repo.  
Process to simple repackage a debian package and push it to wmf apt repo.


Example: repackage <code>python-libthumbor</code> for debian stretch
Example: repackage <code>python-libthumbor</code> for debian stretch


1. Login to <code>boron.eqiad.wmnet</code>
1. Login to <code>boron.eqiad.wmnet</code>:


2. As a user run the following to download <code>python-libthumbor</code>'s source package:
2. As a user run the following to download <code>python-libthumbor</code>'s source package:
  $ apt-get source libthumbor=1.3.2-0+wmf1
  '''$ apt-get source libthumbor=1.3.2-0+wmf1'''
3. We should now have 3 files and 1 directory in our working directory:
3. We should now have 3 files and 1 directory in our working directory:
   -rw-r--r--  1 user wikidev  15K Jul  6  2016 libthumbor_1.3.2.orig.tar.gz          -> source code of the upstream authors
   -rw-r--r--  1 user wikidev  15K Jul  6  2016 libthumbor_1.3.2.orig.tar.gz          -> source code of the upstream authors
Line 15: Line 17:
   drwxr-xr-x  6 user wikidev 4.0K Nov 20 14:11 libthumbor-1.3.2                      -> extracted source package
   drwxr-xr-x  6 user wikidev 4.0K Nov 20 14:11 libthumbor-1.3.2                      -> extracted source package
4. In the <code>libthumbor-1.3.2/debian</code> directory, we may find the following files:
4. In the <code>libthumbor-1.3.2/debian</code> directory, we may find the following files:
  '''changelog''' -> records all changes and defines the version of the package
  '''changelog''' -> records all changes and defines the version of the package
  '''compat''' -> a meta file used for automation of the package build
  '''compat''' -> a meta file used for automation of the package build
  '''control''' -> defines the binary package names and their relation to other packages (e.g. files needed to build and run the package)
  '''control''' -> defines the binary package names and their relation to other packages (e.g. files needed to build and run the package)
  '''copyright''' -> decribes the license and the copyright holders
  '''copyright''' -> decribes the license and the copyright holders
  '''docs''' -> this can be used to declare some files as documentation
  '''docs''' -> this can be used to declare some files as documentation
  '''patches''' -> a directory of code changes done to the original tarball
  '''patches''' -> a directory of code changes done to the original tarball
  '''rules''' -> defines the package build process
  '''rules''' -> defines the package build process
  '''source''' -> a directory defining some details of how the source package is defined
  '''source''' -> a directory defining some details of how the source package is defined
  '''watch''' -> a meta file which contains information where to look for new releases (i.e. where to download a newer source)
  '''watch''' -> a meta file which contains information where to look for new releases (i.e. where to download a newer source)
5. Go to <code>libthumbor-1.3.2</code>
5. Go to <code>libthumbor-1.3.2</code>:
  $ libthumbor-1.3.2
'''$ libthumbor-1.3.2'''
  $ dch -i
'''$ dch -i'''


6. We should see something like the following:
6. We should see something like the following:
<pre>
<pre>
  libthumbor (1.3.2-0+wmf1+stretch1) stretch-wikimedia; urgency=low
libthumbor (1.3.2-0+wmf1+stretch1) stretch-wikimedia; urgency=low
  * Rebuild for stretch
-- Effie Mouzeli <effie@wikimedia.org>  Tue, 20 Nov 2018 13:42:24 +0000
libthumbor (1.3.2-0+wmf1) unstable; urgency=low
  * Initial release. (Closes: #783577)
-- Gilles Dubuc <gilles@wikimedia.org>  Thu, 19 May 2016 15:30:16 +0000
</pre>
7. Add your new version string, make sure it is a higher version than the previous one.


  * Rebuild for stretch
8. Build!
  '''$ DIST=stretch pdebuild'''  -> (use packages from stretch repo)
or
'''$ DIST=stretch-wikimedia pdebuild'''  -> (include packages from stretch-wikimedia repo)
9. Find the build results:
user@boron:~'''$ ls  -la /var/cache/pbuilder/result/stretch-amd64/*thum*'''
-rw-r--r-- 1 user wikidev 5873 Nov 20 14:12 /var/cache/pbuilder/result/stretch-amd64/libthumbor_1.3.2-0+wmf1+stretch1_amd64.buildinfo
-rw-r--r-- 1 user wikidev 1797 Nov 20 14:12 /var/cache/pbuilder/result/stretch-amd64/libthumbor_1.3.2-0+wmf1+stretch1_amd64.changes
-rw-r--r-- 1 user wikidev 2500 Nov 20 14:12 /var/cache/pbuilder/result/stretch-amd64/libthumbor_1.3.2-0+wmf1+stretch1.debian.tar.xz
-rw-r--r-- 1 user wikidev 1260 Nov 20 14:12 /var/cache/pbuilder/result/stretch-amd64/libthumbor_1.3.2-0+wmf1+stretch1.dsc
-rw-r--r-- 1 user wikidev 7862 Nov 20 14:12 /var/cache/pbuilder/result/stretch-amd64/python-libthumbor_1.3.2-0+wmf1+stretch1_all.deb


  -- Effie Mouzeli <effie@wikimedia.org>  Tue, 20 Nov 2018 13:42:24 +0000
=== Download source packages from snapshot.debian.org ===
If you are looking  to build a different version than the one provided by a source package, you can have a look at [https://snapshot.debian.org Debian snapshots]. Everything you need will be downloaded for you by pointing <code>dget</code> to the relevant <code>.dsc</code> file:
'''$ dget https://snapshot.debian.org/archive/debian-debug/20180826T210249Z/pool/main/libr/librsvg/librsvg_2.40.20-3.dsc'''
 
== Upload to Wikimedia Repo ==
1. Login to <code> install1002.wikimedia.org</code>, create a directory and copy all files from boron
'''$ ssh install1002.wikimedia.org'''
'''$ mkdir libthumbor; cd libthumbor'''
'''$ rsync rsync://boron.eqiad.wmnet/pbuilder-result/stretch-amd64/*thum* .'''
2. Import!
'''$ sudo -H bash'''
'''$ reprepro -C main include stretch-wikimedia libthumbor_1.3.2-0+stretch+wmf1_amd64.changes'''
Note: We split the archives sometimes, i.e. we add packages to components instead of <code>main</code>


  libthumbor (1.3.2-0+wmf1) unstable; urgency=low
3. Do not forget to '''!log uploads in -operations IRC channel''' !


   * Initial release. (Closes: #783577)
4. ''Achievement Unlocked!''
=== Upload to a component directory ===
Sometimes we build packages for specific purposes thus we don't want them to be available under the <code>main</code> component.
To create a new component, you simply need to prepare a patch for <code>puppet/modules/aptrepo/distributions-wikimedia</code>. After the component is available on <code>install*</code> servers, you can upload to it by running:
  '''$ reprepro -C component/thumbor include stretch-wikimedia libthumbor_1.3.2-0+stretch+wmf1_amd64.changes'''


  -- Gilles Dubuc <gilles@wikimedia.org>  Thu, 19 May 2016 15:30:16 +0000
== Create a Debian patch ==
</pre>
Sometimes we might wish to add a fix to the software we are packaging, not included in the original source.
7. Add your new version string, make sure it is a higher version than the previous one


8. Build!
1. Check if your source package is using quilt:
  $ DIST=stretch pdebuild  -> (use packages from stretch repo)
   '''$ cat debian/source/format'''
  or
   3.0 (quilt)
  $ DIST=stretch-wikimedia pdebuild  -> (include packages from stretch-wikimedia repo)
9. Find the build results
  user@boron:~$ ls  -la /var/cache/pbuilder/result/stretch-amd64/*thum*
  -rw-r--r-- 1 user wikidev 5873 Nov 20 14:12 /var/cache/pbuilder/result/stretch-amd64/libthumbor_1.3.2-0+wmf1+stretch1_amd64.buildinfo
  -rw-r--r-- 1 user wikidev 1797 Nov 20 14:12 /var/cache/pbuilder/result/stretch-amd64/libthumbor_1.3.2-0+wmf1+stretch1_amd64.changes
   -rw-r--r-- 1 user wikidev 2500 Nov 20 14:12 /var/cache/pbuilder/result/stretch-amd64/libthumbor_1.3.2-0+wmf1+stretch1.debian.tar.xz
  -rw-r--r-- 1 user wikidev 1260 Nov 20 14:12 /var/cache/pbuilder/result/stretch-amd64/libthumbor_1.3.2-0+wmf1+stretch1.dsc
   -rw-r--r-- 1 user wikidev 7862 Nov 20 14:12 /var/cache/pbuilder/result/stretch-amd64/python-libthumbor_1.3.2-0+wmf1+stretch1_all.deb


== Upload to Wikimedia Repo ==
2. Make the edits you wish to do for your fix and run the following command:
1. Login to <code> install1002.wikimedia.org</code>, create a directory and copy
'''$ dpkg-source --commit''' 
  $ ssh install1002.wikimedia.org
librsvg-2.40.18/tests/crash.c
  $ mkdir libthumbor; cd libthumbor
Enter the desired patch name: something-with-sense.patch
  $ rsync rsync://boron.eqiad.wmnet/pbuilder-result/stretch-amd64/*thum* .
2. Import!
  $ sudo -H bash
  $ reprepro -C main include stretch-wikimedia libthumbor_1.3.2-0+stretch+wmf1_amd64.changes
Note: We split the archives sometimes


3. Do not forget to !log uploads in -operations IRC channel
3. Name your patch and use the spawned editor to make a description, for instance:
Description: add -exact in webp conversion
The -exact option was introduced in webp 0.5, which is only available on Stretch.
python-thumbor-wikimedia (2.2-1+deb9u1) stretch-wikimedia; urgency=low
  * Repackaging for stretch
Author: Effie Mouzeli <effie@wikimedia.org>
  ---


4. Achievement Unlocked.
4. Your patch will be saved under <code>debian/patches/foo.patch</code>. Build it with the usual way:
'''$ DIST=stretch pdebuild'''  -> (use packages from stretch repo)
or
'''$ DIST=stretch-wikimedia pdebuild'''  -> (include packages from stretch-wikimedia repo)


[[Category:Package management|*]]
[[Category:Package management|*]]

Revision as of 16:06, 21 December 2018

Rebuilding a package

Process to simple repackage a debian package and push it to wmf apt repo.

Example: repackage python-libthumbor for debian stretch

1. Login to boron.eqiad.wmnet:

2. As a user run the following to download python-libthumbor's source package:

$ apt-get source libthumbor=1.3.2-0+wmf1

3. We should now have 3 files and 1 directory in our working directory:

 -rw-r--r--  1 user wikidev  15K Jul  6  2016 libthumbor_1.3.2.orig.tar.gz          -> source code of the upstream authors
 -rw-r--r--  1 user wikidev 1.2K Jul  6  2016 libthumbor_1.3.2-0+wmf1.dsc           -> meta file describing which files are around
 -rw-r--r--  1 user wikidev 2.4K Jul  6  2016 libthumbor_1.3.2-0+wmf1.debian.tar.xz -> Debian-specific changes
 drwxr-xr-x  6 user wikidev 4.0K Nov 20 14:11 libthumbor-1.3.2                      -> extracted source package

4. In the libthumbor-1.3.2/debian directory, we may find the following files:

 changelog -> records all changes and defines the version of the package
 compat -> a meta file used for automation of the package build
 control -> defines the binary package names and their relation to other packages (e.g. files needed to build and run the package)
 copyright -> decribes the license and the copyright holders
 docs -> this can be used to declare some files as documentation
 patches -> a directory of code changes done to the original tarball
 rules -> defines the package build process
 source -> a directory defining some details of how the source package is defined
 watch -> a meta file which contains information where to look for new releases (i.e. where to download a newer source)

5. Go to libthumbor-1.3.2:

$ libthumbor-1.3.2
$ dch -i

6. We should see something like the following:

 libthumbor (1.3.2-0+wmf1+stretch1) stretch-wikimedia; urgency=low
 
  * Rebuild for stretch
 
 -- Effie Mouzeli <effie@wikimedia.org>  Tue, 20 Nov 2018 13:42:24 +0000
 
 libthumbor (1.3.2-0+wmf1) unstable; urgency=low
 
  * Initial release. (Closes: #783577)
 
 -- Gilles Dubuc <gilles@wikimedia.org>  Thu, 19 May 2016 15:30:16 +0000

7. Add your new version string, make sure it is a higher version than the previous one.

8. Build!

$ DIST=stretch pdebuild   -> (use packages from stretch repo)
or 
$ DIST=stretch-wikimedia pdebuild  -> (include packages from stretch-wikimedia repo)

9. Find the build results:

user@boron:~$ ls  -la /var/cache/pbuilder/result/stretch-amd64/*thum*
-rw-r--r-- 1 user wikidev 5873 Nov 20 14:12 /var/cache/pbuilder/result/stretch-amd64/libthumbor_1.3.2-0+wmf1+stretch1_amd64.buildinfo
-rw-r--r-- 1 user wikidev 1797 Nov 20 14:12 /var/cache/pbuilder/result/stretch-amd64/libthumbor_1.3.2-0+wmf1+stretch1_amd64.changes
-rw-r--r-- 1 user wikidev 2500 Nov 20 14:12 /var/cache/pbuilder/result/stretch-amd64/libthumbor_1.3.2-0+wmf1+stretch1.debian.tar.xz
-rw-r--r-- 1 user wikidev 1260 Nov 20 14:12 /var/cache/pbuilder/result/stretch-amd64/libthumbor_1.3.2-0+wmf1+stretch1.dsc
-rw-r--r-- 1 user wikidev 7862 Nov 20 14:12 /var/cache/pbuilder/result/stretch-amd64/python-libthumbor_1.3.2-0+wmf1+stretch1_all.deb

Download source packages from snapshot.debian.org

If you are looking to build a different version than the one provided by a source package, you can have a look at Debian snapshots. Everything you need will be downloaded for you by pointing dget to the relevant .dsc file:

$ dget https://snapshot.debian.org/archive/debian-debug/20180826T210249Z/pool/main/libr/librsvg/librsvg_2.40.20-3.dsc

Upload to Wikimedia Repo

1. Login to install1002.wikimedia.org, create a directory and copy all files from boron

$ ssh install1002.wikimedia.org
$ mkdir libthumbor; cd libthumbor
$ rsync rsync://boron.eqiad.wmnet/pbuilder-result/stretch-amd64/*thum* .

2. Import!

$ sudo -H bash
$ reprepro -C main include stretch-wikimedia libthumbor_1.3.2-0+stretch+wmf1_amd64.changes

Note: We split the archives sometimes, i.e. we add packages to components instead of main

3. Do not forget to !log uploads in -operations IRC channel !

4. Achievement Unlocked!

Upload to a component directory

Sometimes we build packages for specific purposes thus we don't want them to be available under the main component. To create a new component, you simply need to prepare a patch for puppet/modules/aptrepo/distributions-wikimedia. After the component is available on install* servers, you can upload to it by running:

 $ reprepro -C component/thumbor include stretch-wikimedia libthumbor_1.3.2-0+stretch+wmf1_amd64.changes

Create a Debian patch

Sometimes we might wish to add a fix to the software we are packaging, not included in the original source.

1. Check if your source package is using quilt:

 $ cat debian/source/format
 3.0 (quilt)

2. Make the edits you wish to do for your fix and run the following command:

$ dpkg-source --commit  
librsvg-2.40.18/tests/crash.c
Enter the desired patch name: something-with-sense.patch

3. Name your patch and use the spawned editor to make a description, for instance:

Description: add -exact in webp conversion
The -exact option was introduced in webp 0.5, which is only available on Stretch.

python-thumbor-wikimedia (2.2-1+deb9u1) stretch-wikimedia; urgency=low

  * Repackaging for stretch

Author: Effie Mouzeli <effie@wikimedia.org>
 ---

4. Your patch will be saved under debian/patches/foo.patch. Build it with the usual way:

$ DIST=stretch pdebuild   -> (use packages from stretch repo)
or 
$ DIST=stretch-wikimedia pdebuild  -> (include packages from stretch-wikimedia repo)