You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Uploading large files: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Dereckson
(Now 5 GB max, per rOMWCa6e5f230)
imported>Urbanecm
(→‎Tip and tricks: no longer possible)
Line 5: Line 5:
* the name of the user account for this first revision and upload
* the name of the user account for this first revision and upload


Per [https://phabricator.wikimedia.org/rOMWCa6e5f230be5c0d8f9da181f8509e4cf3062419ea rOMWCa6e5f230] the maximum file size is 5 GB.
MediaWiki currently doesn't support files greater than 4 GB (as size is stored as a 32 bits unsigned integer) while our swift backend storage is limited to 5 Gb. See [[phab:T191804]] and [[phab:T191802]] for discussion to extend this limit respectively to 5 GB and beyond.


== Step 1: download files ==
== Step 1: download files ==


Download the files to terbium (or if there's not enough space, tin) with http_proxy=webproxy.eqiad.wmnet:8080 (or https_proxy for https requests)
Download the files to [[mwmaint1001]] (or if there's not enough space, tin) with http_proxy=webproxy.eqiad.wmnet:8080 (or https_proxy for https requests)


(url-downloader.wikimedia.org:8080 has a limit just under 1GB - see [https://phabricator.wikimedia.org/T111941])
(url-downloader.wikimedia.org:8080 has a limit just under 1GB - see [https://phabricator.wikimedia.org/T111941])


<source lang="bash">
<source lang="bash">
curl -O -x webproxy.eqiad.wmnet:8080 <URL>
wget <URL>
</source>
</source>


Line 24: Line 24:
mwscript importImages.php --wiki=commonswiki --comment-ext=txt --user=USERNAME /tmp/uploads
mwscript importImages.php --wiki=commonswiki --comment-ext=txt --user=USERNAME /tmp/uploads
</source>
</source>
== Tip and tricks ==
=== Google Drive ===
It's also possible to download a file from Google Drive, but you need to transform the URL and ask curl to follow redirects:
<source lang="bash">
curl -L -O -x webproxy.eqiad.wmnet:8080 https://googledrive.com/host/0BxoqnOKDr5j5NWJHQ0RlM2VJcWM
</source>
You have to use <nowiki>https://googledrive.com/host/<the file ID></nowiki> as URL.

Revision as of 12:58, 19 August 2018

Requirement

You need:

  • the URL of the media file to upload
  • a text file with the first revision content
  • the name of the user account for this first revision and upload

MediaWiki currently doesn't support files greater than 4 GB (as size is stored as a 32 bits unsigned integer) while our swift backend storage is limited to 5 Gb. See phab:T191804 and phab:T191802 for discussion to extend this limit respectively to 5 GB and beyond.

Step 1: download files

Download the files to mwmaint1001 (or if there's not enough space, tin) with http_proxy=webproxy.eqiad.wmnet:8080 (or https_proxy for https requests)

(url-downloader.wikimedia.org:8080 has a limit just under 1GB - see [1])

wget <URL>

At this stage, it could be pertinent to check the hash of the file if known.

Step 2: import image to Commons

mwscript importImages.php --wiki=commonswiki --comment-ext=txt --user=USERNAME /tmp/uploads