You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Difference between revisions of "Add a wiki"

From Wikitech-static
Jump to navigation Jump to search
imported>Alex Monk
imported>Alex Monk
(→‎Finish: wikidataclient changes per https://phabricator.wikimedia.org/T125501#2002776)
Line 87: Line 87:
| wikidata.dblist || All wikis running the Wikidata repo
| wikidata.dblist || All wikis running the Wikidata repo
|-
|-
| wikidataclient.dblist || All wikis running the Wikidata client
| wikidataclient.dblist || All wikis running the Wikidata client (most new language-project wikis should start off like this)
|-
|-
| wikimania.dblist || All Wikimania wikis
| wikimania.dblist || All Wikimania wikis
Line 132: Line 132:
* Run sync-file w/static/images/project-logos/path-to-logo.png
* Run sync-file w/static/images/project-logos/path-to-logo.png
* Run sync-file langlist (this must be done before the interwiki cache below)
* Run sync-file langlist (this must be done before the interwiki cache below)
* Unless it's a language project, update the [[meta:Interwiki map]] and run <tt>updateinterwikicache</tt>
* Unless it's a language project, update the [[meta:Interwiki map]]
* <tt>updateinterwikicache</tt>


=== RESTBase ===
=== RESTBase ===
Line 150: Line 151:
** Create the container for thumbnails. <tt>mwscript extensions/WikimediaMaintenance/filebackend/setZoneAccess.php databasename --backend=local-multiwrite --private</tt>
** Create the container for thumbnails. <tt>mwscript extensions/WikimediaMaintenance/filebackend/setZoneAccess.php databasename --backend=local-multiwrite --private</tt>


=== Wikidata ===
=== Labs DNS ===
====dblist====
Create a patch for ops/puppet to add the database to the correct list in modules/role/manifests/labs/dnsrecursor.pp
* Add to dblists/wikidataclient.dblist and sync-dir dblists (might need to touch/sync InitialiseSettings/CommonSettings)
 
====sites table====
* Add the new wiki to all site tables of current wikidataclient wikis and also wikidatawiki.
* Populate the sites table on the new wiki, which can be done with the populateSitesTable script in Wikibase.  It has to be run once for each site group (e.g. wikipedia, wikivoyage, ...), as well as for wikidata (wikidata.dblist, with --site-group wikipedia) and commons (--site-group wikipedia).
 
Both of these steps can be done with populateSitesTable:
 
<code>foreachwikiindblist wikidataclient.dblist extensions/Wikidata/extensions/Wikibase/lib/maintenance/populateSitesTable.php --force-protocol https</code>
 
In addition to updating the sites table, the script clears the sites cache (memcached) which is needed for things to work correctly.


==After you finished==
==After you finished==

Revision as of 17:55, 5 February 2016

This page will walk you through how to add a new wiki project. This includes new languages on sister projects, and wikis for committees, chapters etc.

Start

  • Tell in advance the Ops list, jcrespo (jynus) or Coren that this is happening so the storage layer can be prepared and checked (labs, backups, dumps). In particular, it needs to be made clear whether the wiki will be public - if so, ops will arrange for the wiki to be replicated to labs. If not, ops will need to add the wiki to $private_wikis in the puppet repository's manifests/realm.pp.

This can be fixed after the wiki creation, but it is more painful/involves custom queries.

DNS

  • First of all, the DNS entry for the new wiki has to be added. Make the following changes to your local checkout of operations/dns.git and submit to gerrit for review.
    • If it is a language project, add the language code to /templates/helpers/langs.tmpl in operations/dns repo. If it is not a new language, then it has probably been done already as these are shared by all the sister projects.
    • If it is a wiki in *.wikimedia.org domain (chapter wiki or special wiki), add it to /templates/wikimedia.org. Make sure to add a mobile entry if necessary.
  • Merge the change in gerrit and run authdns-update. Query the DNS servers to make sure it has been correctly deployed. See DNS#HOWTO for details.

Apache configuration

You will firstly have to login to our deployment host. As of sometime, this is the host tin, you will have to go via bast1001. Then gather all relevant information for the new project, including all configuration variables and entries. Each wiki will have different requirements (project name, namespaces, extensions to be enabled etc).

You now have to determine if this is a new language project (say something like Spanish Wikibooks) or a chapter wiki, or a special wiki (anything else, including private wikis). Then choose the correct part of this manual:

Language Project

  • Make sure the language has been approved by the Language committee at Requests for new languages on Meta-Wiki. Usually, the Maniphest task will contain a link to the approval page.
  • Make the following changes on your own checkout of operations/mediawiki-config.git:
  • Make sure that the language code appears in UniversalLanguageSelector's langdb (file data/langdb.yaml).
  • Add the language code to the ContentTranslation registry - cxserver repository, files config.dev.yaml and config.prod.yaml, in the source and target sections in each.
  • Go to #Finish

Finish

IMPORTANT: For Private Wikis

  • Private wiki databases must not be replicated to the labsdb MySQL instances!
    • Before creating a database for a new private wiki, make sure to add the db name to the puppet global array $private_wikis in manifests\realm.pp.
    • Deploy this config change with puppet and manually restart the Prelabsdb-db (Sanitarium) MySQL instance that will house this wiki's db (most likely s3).
    • If you need help with this, please ask a member of the Ops team for help. This is very important.

MediaWiki configuration

Database list Purpose
s1.dblist
s2.dblist
s3.dblist
s4.dblist
s5.dblist
s6.dblist
s7.dblist
Database lists of wikis in each MySQL database cluster
all.dblist All wikis should be listed in here
closed.dblist Any closed (no write access, full read access) wikis
deleted.dblist Wiki databases which MediaWiki is no longer configured to access
small.dblist
medium.dblist
large.dblist
Database lists of wikis arranged into their relevant size
flaggedrevs.dblist All wikis running the FlaggedRevs extension
securepollglobal.dblist $wgSecurePollCreateWikiGroups wikis: Board Election wikis
visualeditor-default.dblist All wikis where VisualEditor is enabled by default
commonsuploads.dblist All wikis which should have local uploading soft-disabled. Uploads go to Commons instead.
fishbowl.dblist All fishbowl (restricted write access, full read access) wikis
private.dblist All private (read and write restricted) wikis
special.dblist All special wikis
wikidata.dblist All wikis running the Wikidata repo
wikidataclient.dblist All wikis running the Wikidata client (most new language-project wikis should start off like this)
wikimania.dblist All Wikimania wikis
wikimedia.dblist All chapter wikis
wikibooks.dblist All Wikibooks wikis
wikinews.dblist All Wikinews wikis
wikipedia.dblist All Wikipedia wikis
wikiquote.dblist All Wikiquote wikis
wikisource.dblist All Wikisource wikis
wikiversity.dblist All Wikiversity wikis
wikivoyage.dblist All Wikivoyage wikis
wiktionary.dblist All Wiktionary wikis
  • Now it is time to actually create the database (this script also performs other tasks, such as notifying the 'newprojects' mailing list). There are two different ways to do this, depending on if you did Part A, B, or C above:
  • mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki languagecode projectname databasename domain
  • Part A is for standard language additions to projects, like adding a Spanish Wikinews, which will be my example.
  • mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki languagecode projectname databasename domain
  • EG: mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki es wikinews eswikinews es.wikinews.org to add a Spanish Wikinews.
  • Part B is for Chapters Wikis. My example will be adding a Finnish chapter wiki:
  • mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki languagecode projectname databasename domain
  • EG: mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki fi wikimedia fiwikimedia fi.wikimedia.org
  • Part C is for non-standard special wikis such as committees, chapters, and the like. My example will be strategy.wikimedia.org.
  • mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki languagecode projectname databasename domain
  • EG: mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki en wikimedia strategywiki strategy.wikimedia.org

(Note the 'aawiki' in these examples: aawiki is in s3, so this will create the database in s3, which is usually the right place for new small wikis.)

  • Merge the config change in Gerrit, and pull it onto tin
  • Check *.dblist files now contain the new wiki
  • Run sync-dir dblists to synchronize all database lists
  • Check wikiversions.json is sane
  • Run sync-wikiversions to synchronize the version number to use for this wiki
  • Run sync-file wmf-config/InitialiseSettings.php
  • Run sync-file w/static/images/project-logos/path-to-logo.png
  • Run sync-file langlist (this must be done before the interwiki cache below)
  • Unless it's a language project, update the meta:Interwiki map
  • updateinterwikicache

RESTBase

RESTBase is a service providing a RESTful API for the projects' wikis. To enable it to serve the new wiki as well, create a patch for ops/puppet adding its domain to RESTBase's configuration.

Parsoid

Parsoid has it's own copy of SiteMatrix, which needs updating using tools/fetch-sitematrix.js in mediawiki/services/parsoid.git.

Search

You shouldn't have to do much (maybe adjust shard sizes preemptively if it'll be big), the index should be automatically created by addWiki.php and all wikis now opt into Cirrus/Elastic by default. See Search/New#Adding_new_wikis for more information.

Swift

  • Public wikis:
    • Create the container for thumbnails. mwscript extensions/WikimediaMaintenance/filebackend/setZoneAccess.php databasename --backend=local-multiwrite
  • Private wikis:
    • Create the container for thumbnails. mwscript extensions/WikimediaMaintenance/filebackend/setZoneAccess.php databasename --backend=local-multiwrite --private

Labs DNS

Create a patch for ops/puppet to add the database to the correct list in modules/role/manifests/labs/dnsrecursor.pp

After you finished

If there's something to import (as is often the case in new Language wikis), someone will do so, following the process described at MetaWikipedia:incubator:Incubator:Importing from Incubator (logged at MetaWikipedia:incubator:Incubator:Site creation log).

See also Delete a wiki.