You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Recent changes

Jump to navigation Jump to search

Track the most recent changes to the wiki on this page. See main namespace without noise.

Recent changes options Show last 50 | 100 | 250 | 500 changes in last 1 | 3 | 7 | 14 | 30 days
Hide registered users | Hide anonymous users | Hide my edits | Hide bots | Hide minor edits
Show new changes starting from 02:17, 29 June 2022
   
List of abbreviations:
N
This edit created a new page (also see list of new pages)
m
This is a minor edit
b
This edit was performed by a bot
(±123)
The page size changed by this number of bytes

28 June 2022

N    00:03  User:0xau007 diffhist +18 0xau007 (Hi statement)
 m   00:01  Map of database maintenance diffhist +1,055 Dexbot (Bot: Updating the report)

27 June 2022

     23:51  Server Admin Log‎‎ 2 changes history +92,721 [Stashbot‎ (2×)]
     
23:51 (cur | prev) +85,388 Stashbot (cmjohnson@cumin1001: START - Cookbook sre.hosts.provision for host mw1474.mgmt.eqiad.wmnet with reboot policy FORCED)
     
01:25 (cur | prev) +7,333 Stashbot (cmjohnson@cumin1001: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host dse-k8s-worker1008.mgmt.eqiad.wmnet with reboot policy FORCED)
     23:13  User talk:AndreaWest/WDQS Q and A diffhist +1,402 Jheald
     21:35  User talk:AndreaWest/Blazegraph Features and Capabilities diffhist +3,238 Jheald (+)
     21:19  Release Engineering/SAL diffhist +178 Stashbot (Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/809022)
     20:56  Deployments diffhist +1,050 Bartosz Dziewoński (→‎Wednesday, June 29: Phab links)
     19:39  User:AndreaWest/WDQS Testing‎‎ 2 changes history +2,959 [AndreaWest‎ (2×)]
     
19:39 (cur | prev) +707 AndreaWest (→‎Query Characteristics/Frequencies)
 m   
00:35 (cur | prev) +2,252 AndreaWest (→‎Wikidata Triples for Compliance Testing)
     19:18  User:AndreaWest/WDQS Testing/Explaining the Benchmark Queries Spreadsheet diffhist +2,538 AndreaWest (→‎Compliance)
 m   19:17  User:AndreaWest diffhist +26 AndreaWest
N    18:41  Nova Resource:Wikidata-reconciliation diffhist +128 Labslogbot (Auto update of instance info.)
N    18:33  Nova Resource:Canasta diffhist +112 Labslogbot (Auto update of instance info.)
N    18:22  User talk:Apladm diffhist +670 StrikerBot (Welcome to Toolforge!)
N    18:19  User talk:Gouvernathor diffhist +676 StrikerBot (Welcome to Toolforge!)
     18:14  Nova Resource:Tools/SAL diffhist +956 Stashbot (taavi: restart calico, appears to have got stuck after the ca replacement operation)
     18:13  Help talk:SSH Fingerprints diffhist +195 BCornwall (→‎Old OpenSSH mention can be removed: done)
     16:42  Nova Resource:Paws/SAL diffhist +153 Stashbot (Rook: Make renderer accessable in dev env #172 9f2f171ff2a519fa11a784e8236eaf337bb13d08 T308986)
N    14:33  Incidents/2022-06-16 MariaDB password leak diffhist +4,027 Marostegui (Scorecard)
N    13:00  Event Platform/Intake diffhist +483 Ottomata (Created page with "PLACEHOLDER: document event platform intake requirements * - initialization of default values * - set meta.dt as the ingestion time (current time is fine) * - set meta.stream to the proper value (if not set by caller) * - $schema as the schema that can validate the generated event (if not set by caller) * - set meta.id to a uuid if not set * - verification of schema vs stream (this requires looking up stream config) * - validation of the event against it...")
     12:21  Netbox diffhist +165 Ayounsi (→‎Production infrastructure)
N    01:09  Nova Resource:Tools.signatures/SAL diffhist +117 Stashbot (wm-bot: <anticomposite> remove old stale reports)
 m   00:29  User:AndreaWest/WDQS Testing/Running Iguana diffhist +9 AndreaWest
 m   00:28  User:AndreaWest/WDQS Testing/Running TFT diffhist +10 AndreaWest
 m   00:01  Map of database maintenance diffhist 0 Dexbot (Bot: Updating the report)

26 June 2022

Nm   19:46  User:Suryopurnomo diffhist +240 Suryopurnomo
     13:42  User:AndreaWest/Blazegraph Features and Capabilities diffhist +1,198 AndreaWest (→‎wikibase:label: Added text about using OPTIONAL with labels)
 m   00:02  Map of database maintenance diffhist 0 Dexbot (Bot: Updating the report)

25 June 2022

     18:17  Server Admin Log diffhist +2,028 Stashbot (jmm@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest1002.eqiad.wmnet)
     13:51  Image-suggestion diffhist 0 Aklapper (fix two typos)
     09:32  Nova Resource:Tools.bridgebot/SAL diffhist +92 Stashbot (wm-bot: <lucaswerkmeister> Double IRC messages to other bridges)
     06:34  Nova Resource:Wikisp/SAL diffhist +102 Stashbot (Aldrich: Apollo: Update zammad from 5.0 to 5.2 (T311332))
     03:25  Nova Resource:Cvn/SAL diffhist +83 Stashbot (AntiComposite: /cs flags #cvn-wp-en PhantomTech voiced)
     02:00  Deployments diffhist +85 Labdajiwa
 m   00:00  Map of database maintenance diffhist 0 Dexbot (Bot: Updating the report)

24 June 2022

     22:42  SecurePoll diffhist +2,979 Foks (→‎Tallying: rm)
N    22:24  Talk:SecurePoll diffhist +306 Foks (→‎Encryption: "This section has not been updated for the new situation after 2013": new section)
     20:52  Nova Resource:Deployment-prep/SAL diffhist +63 Stashbot (taavi: added `denisse` as a member)
     20:52  Release Engineering/SAL diffhist +61 Stashbot (taavi: added `denisse` as a member)
     20:48  Deployments diffhist +454 Matthias Mullie (→‎{{Deployment_day|date=2022-06-27}})
     19:35  Server Admin Log diffhist +54,579 Stashbot (dancy@deploy1002: backport aborted: (duration: 00m 12s))
N    19:29  Nova Resource:Sre-sandbox/SAL diffhist +95 Stashbot (ori: added self to project)
 m   19:08  MediaWiki at WMF diffhist 0 Mainframe98 (→‎Application resources: typo fixes)
 m   18:00  User:AndreaWest diffhist +161 AndreaWest (→‎Subpages)
N    17:59  User:AndreaWest/WDQS Testing/Explaining the Benchmark Queries Spreadsheet diffhist +6 AndreaWest (Created page with "Coming")
     17:58  User:AndreaWest/WDQS Testing diffhist +232 AndreaWest (→‎Query Characteristics/Frequencies)
N    16:28  User talk:Weeklyd3 diffhist +765 Weeklyd3 (→‎Welcome to Toolforge!: Reply)
     15:53  Nova Resource:Tools.quickcategories/SAL diffhist +143 Stashbot (wm-bot: <lucaswerkmeister> deployed 64df38cf96 (README.md update – pulled without webservice restart))
     15:49  Netbox diffhist +3,064 Ayounsi (add Redis mention)
N    15:48  Image-suggestion diffhist +1,426 Hnowlan (Created page with "'''image-suggestion''' is a service that provides read-only access to the Data Gateway Service. It is a relatively simple HTTP service running in Kubernetes. It was created by the Platform Engineering Team. The code lives in Gerrit. It is not to be confused with the git:mediawiki/services/im...")
 m   13:19  Help:Standalone puppetmaster/PuppetDB diffhist +115 Hashar (→‎Reconfigure the puppetmaster server)
N    12:51  User:James Rhoda diffhist +104 James Rhoda (Updated my user page)
     12:43  Nova Resource:Devtools/SAL diffhist +123 Stashbot (taavi: `os quota set devtools --ram 45056 --cores 22 --instances 9` # T311302)
     09:52  Ganeti diffhist +16 Aklapper (+{{ptag|Ganeti}})
     07:01  Nova Resource:Toolsbeta/SAL diffhist +522 Stashbot (wm-bot2: removing grid node toolsbeta-sgewebgrid-lighttpd-0901.toolsbeta.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko)
 m   00:01  Map of database maintenance diffhist −206 Dexbot (Bot: Updating the report)

23 June 2022

     21:47  User:AndreaWest/WDQS Testing diffhist +3,918 AndreaWest (→‎Wikidata Triples for Stress Testing)
     21:23  Server Admin Log‎‎ 2 changes history +69,133 [Stashbot‎ (2×)]
     
21:23 (cur | prev) +39,618 Stashbot (mutante: restbase-dev1006 has manually installed packages (wrk, maybe others))
     
00:35 (cur | prev) +29,515 Stashbot (brennen: end of phabricator maintenance window)
N    20:49  User:Weeklyd3 diffhist +41 Weeklyd3 (Interwiki link)
 m   20:39  SRE/Infrastructure naming conventions diffhist +380 Dzahn
     20:25  Deployments‎‎ 2 changes history +2,534 [Jdlrobson‎; DDesouza‎]
     
20:25 (cur | prev) +1,077 Jdlrobson
 m   
00:45 (cur | prev) +1,457 DDesouza (Schedule change)
     20:01  User:AndreaWest/WDQS Testing/Running Iguana diffhist −44 AndreaWest (→‎Code Modifications)
 m   18:18  Deployments/Holding the train diffhist +455 Brennen Bearnes (Minor tweaks to non-emergency para.)
     17:51  Nova Resource:Tools/SAL diffhist +1,622 Stashbot (wm-bot2: removing grid node tools-sgeexec-0941.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko)
     15:59  Release Engineering/SAL diffhist +87 Stashbot (taavi: reload zuul for https://gerrit.wikimedia.org/r/808021)
     15:10  Fundraising diffhist +5 Jgleeson (Update the overview image with a more recent photo, albeit still eight years old.)
 m   14:45  Kubernetes/Clusters/New diffhist −177 Herron (update prometheus hostnames after hw refresh)
     13:20  Nova Resource:Paws/SAL diffhist +140 Stashbot (Rook: remove deployhook token #171 T310382 74180f43cf6487f53b4812106ab8471aba2f8528)
 m   12:58  Splunk On-Call diffhist −1 LSobanski (→‎Viewing the business hours pager schedule:: Removed the colon from the section title to make linking easier (and in line with the communication sent out).)
     10:23  Nova Resource:Quarry/SAL diffhist +148 Stashbot (Rook: 807534: Introduce black formatting to quarry | https://gerrit.wikimedia.org/r/c/analytics/quarry/web/+/807534)
     10:00  Kubernetes/Clusters diffhist +229 Alexandros Kosiaris (→‎staging)
     07:43  User:Samtar diffhist +177 Samtar
     04:55  User talk:BryanDavis diffhist 0 Unknown user (sorry)
     01:23  User:AndreaWest diffhist −14 AndreaWest (Changed GitHub link)
N    00:48  User:DDesouza diffhist +6 DDesouza (Created page with "Hello.")
 m   00:00  Map of database maintenance diffhist −343 Dexbot (Bot: Updating the report)

22 June 2022

     23:56  Nova Resource:Suggestbot/SAL diffhist +138 Stashbot (Nettrom: shut down and deleted suggestbot-01 instance)
 m   21:21  User:AndreaWest/WDQS Testing/Testing the Testing diffhist +13 AndreaWest
     21:20  User:AndreaWest/WDQS Testing‎‎ 2 changes history +3,017 [AndreaWest‎ (2×)]
     
21:20 (cur | prev) +3,132 AndreaWest (→‎Workload Testing)
 m   
01:02 (cur | prev) −115 AndreaWest (→‎Testing Wikidata-Specific Updates and Queries)
     21:04  Nova Resource:Cvn/SAL diffhist +279 Stashbot (op873: <+CVNBot3> Added: LuchoCR is on es.wikipedia bot list, added by Operator873|CVN until the end of time ("Mass blockiing P2P-proxies with script"))
     21:04  User talk:Dbarratt diffhist +274 Lectrician1 (→‎GraphQL: new section)
     20:50  Nova Resource:Quarry/SAL diffhist +149 Stashbot (Rook: 806474: Get non-coincidental history entries. | https://gerrit.wikimedia.org/r/c/analytics/quarry/web/+/806474)
     20:25  Backport windows/Deployers diffhist 0 Stang (Undo revision 1990823 by Stang (talk))
 m   19:49  DNS/Netbox diffhist +1 BCornwall (→‎DNS records involved: fix misspelling)
     18:17  User:AndreaWest/Blazegraph Features and Capabilities‎‎ 2 changes history +5,561 [AndreaWest‎ (2×)]
     
18:17 (cur | prev) +5,777 AndreaWest (→‎wikibase:label: Clarified SPARQL functions, lang and langmatches)
     
00:46 (cur | prev) −216 AndreaWest (→‎wikibase:around and wikibase:box: Simplified the :box function)
     17:36  Service restarts diffhist +183 Sukhbir Singh (→‎Authoritative DNS: add note about authdns instructions)
     17:36  Release Engineering/SAL diffhist +1,151 Stashbot (taavi: gerrit: add tfellows to the extension-OpenBadges group per request in T308278)
 m   17:34  Apache Traffic Server diffhist +779 Sukhbir Singh (→‎Packaging: fix indentation)
N    17:29  User talk:KBach diffhist +669 StrikerBot (Welcome to Toolforge!)
N    17:26  User talk:Lars Willighagen diffhist +680 StrikerBot (Welcome to Toolforge!)
     17:20  Fundraising/Development tools/Fundraising-dev diffhist −16 Ejegg (→‎Setup and Administration)
     16:52  Nova Resource:Admin-monitoring/SAL diffhist +121 Stashbot (taavi: remove some failed nova-fullstack instances now that DNS problems are hopefully fixed)
     16:48  Nova Resource:Admin/SAL diffhist +278 Stashbot (taavi: restart designate-*.service on both cloudservices nodes)
     15:54  Nova Resource:Tools/SAL diffhist +524 Stashbot (wm-bot2: removing grid node tools-sgewebgrid-lighttpd-0917.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko)
N    14:14  User:Diskdance diffhist +330 Diskdance
     13:08  Kubernetes/cert-manager diffhist +201 Elukey (→‎Configuration)
N    12:55  User:Shyran diffhist +40 Shyran (GitHub page added)
     12:49  GitLab/Gitlab Runner/Trusted Runners diffhist +1,846 Jelto (refactor requesting access and using Trusted Runners)
 m   11:41  Etcd diffhist −1 Btullis (→‎Operations: Typo)
     11:34  GitLab/Gitlab Runner/Security Evaluation diffhist +531 Jelto (add section about allowed images)
     07:26  Splunk On-Call diffhist +103 MVernon (→‎Scheduling an override (out of office, on vacation, etc): note about devnull for business hours if not scheduled)
     07:17  MariaDB diffhist −11 Marostegui (Mholloway phab user is disabled.)
     01:18  Server Admin Log diffhist +15,474 Stashbot (mwdebug-deploy@deploy1002: helmfile [codfw] DONE helmfile.d/services/mwdebug: apply)
 m   00:01  Map of database maintenance diffhist +97 Dexbot (Bot: Updating the report)

21 June 2022

     22:45  Nova Resource:Tools.bridgebot/SAL diffhist +92 Stashbot (wm-bot: <lucaswerkmeister> Double IRC messages to other bridges)
     22:16  Deployments diffhist −14,716 Thcipriani
N    22:16  Deployments/Archive/2022/06 diffhist +27,794 Thcipriani (Created page with " ==Week of June 06== ==={{Deployment_day|date=2022-06-05}}=== {{Deployment calendar event card |when=2022-06-05 00:00 SF |length=24 |window=No deploys all day! See Deployments/Emergencies if things are broken. |who= |what=No Deploys }} ==={{Deployment_day|date=2022-06-06}}=== {{Deployment calendar event card |when=2022-06-06 00:00 SF |length=1 |window=UTC morning backport window<br/><small>'''Your patch may or ma...")
     22:01  Release Engineering/SAL diffhist +175 Stashbot (brennen: gitlab-runners: re-registering all shared runners)
N    17:10  Incidents/2022-06-21 asw-a2-codfw accidental power cycle diffhist +3,338 Jcrespo (→‎Summary)
N    17:07  File:Screenshot 20220621 185358.png diffhist +102 Jcrespo (codfw app server latency increase during power incident)
     16:41  Redis diffhist −3 Phuedx (Removed MainStash from Cluster redis_sessions section per T212129)
     16:00  Search/S3 Plugin Enable diffhist +377 Bking
     15:43  Nova Resource:Tools.toolschecker/SAL diffhist +70 Stashbot (taavi: start test-long-running-buster job)
     15:25  Analytics/Systems/EventLogging/Backfilling diffhist +1,707 Aqu (Backfilling sanitization example presentation)
     15:24  Event Platform/EventStreams diffhist −48 Krinkle (→‎API: prefer schema.wikimedia.org viewer)
     15:23  Nova Resource:Tools/SAL diffhist +524 Stashbot (wm-bot2: removing grid node tools-sgewebgrid-lighttpd-0914.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko)
Nm   14:41  Performance/Mobile Device Lab diffhist +316 Dave Pifke (Add link to OS documentation)
N    14:36  Performance/Mobile Device Lab/Raspberry Pi Image diffhist +2,649 Dave Pifke (Add initial documentation about building and running the image on a Raspberry Pi)
     14:17  Service restarts diffhist 0 BCornwall (→‎Haproxy: Fix HAProxy link)
     14:03  Check bacula.py diffhist +106 Jcrespo (→‎See also: +source code)
 m   13:01  Bacula diffhist +1 Valerio Bozzolan (→‎Levels: fix space)
     12:36  Nova Resource:Paws/SAL diffhist +148 Stashbot (Rook: T302164 #173 upgrading single user container b687ca6aedc745c63b2659124dca0dab01d38173)
     11:48  Nova Resource:Quarry/SAL diffhist +152 Stashbot (Rook: 806504: Show username on 404 page when logged in | https://gerrit.wikimedia.org/r/c/analytics/quarry/web/+/806504)
     11:37  User:Majavah diffhist −2 Majavah
 m   11:36  Talk:Main Page diffhist 0 Majavah (Reverted edits by Loslina (talk) to last revision by Majavah)
     11:05  Kubernetes/Clusters/New diffhist +317 Elukey (→‎Networking)
     09:52  Deployments/Yearly calendar/2022 diffhist +89 Jforrester (Mark "Inspiration Week" as the whole week, not just the Monday; link 20 June deploy log.)
     04:48  Nova Resource:Admin/SAL diffhist +230 Stashbot (andrewbogott: stopping nova-fullstack agent on cloudcontrol1003; it's going to page us otherwise and we're all AFK tomorrow)
     04:10  Nova Resource:Wikisource/Wikisource Export diffhist +63 Samwilson (→‎Creating a new instance: set CALIBRE_TEMP_DIR)
 m   00:35  User:AndreaWest/Blazegraph Features and Capabilities diffhist +111 AndreaWest (→‎wikibase:label)
 m   00:02  Map of database maintenance diffhist −989 Dexbot (Bot: Updating the report)

20 June 2022

     23:04  Nova Resource:Suggestbot/SAL diffhist +283 Stashbot (Nettrom: installed libmariadb-dev and python3.9-dev on suggestbot-02)
     22:10  User:AndreaWest/WDQS Testing diffhist −52 AndreaWest (→‎Testing Wikidata-Specific Updates and Queries)
     20:21  Deployments diffhist +588 Kosta Harlan (→‎{{Deployment_day|date=2022-06-21}})
 m   20:15  User:Ladsgroup diffhist −30 Nintendofan885 (Cleanup links)
     20:11  Wikimediastatus.net diffhist +68 Nintendofan885 (link to about page for Atlassian Statuspage)
     18:34  Infographics diffhist +770 Krinkle (→‎Individual software, services, and clusters)
N    18:30  Talk:Wikidata Query Service/Streaming Updater diffhist +260 So9q (→‎Add link to polling updater?: new section)
     16:30  Nova Resource:Deployment-prep/SAL diffhist +109 Stashbot (urbanecm: add sgimeno as a project member (Growth engineer with need for access))
     16:30  Release Engineering/SAL diffhist +384 Stashbot (urbanecm: add sgimeno as a project member (Growth engineer with need for access))
     07:14  Server Admin Log diffhist +308 Stashbot (SandraEbele: Started Airflow 3 Wikidata metrics jobs (Articleplaceholder, Reliability and SpecialEntityData metrics).)
     04:11  Nova Resource:Wikisource/Documentation diffhist +465 Samwilson (add grafana links)
N    01:53  User talk:ErebusOni diffhist +13 ErebusOni (Faerie's fate)
 m   00:00  Map of database maintenance diffhist −206 Dexbot (Bot: Updating the report)

19 June 2022

     16:28  Nova Resource:Toolsbeta/SAL diffhist +83 Stashbot (taavi: restart OOM'd puppetdb on toolsbeta-puppetdb-02)
     10:28  Server Admin Log diffhist +493 Stashbot (ayounsi@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db1132.eqiad.wmnet with reason: depooled)
     00:20  Netbox diffhist +14 Quiddity (syntax lang fixes)
 m   00:01  Map of database maintenance diffhist 0 Dexbot (Bot: Updating the report)

18 June 2022

 m   00:02  Map of database maintenance diffhist −194 Dexbot (Bot: Updating the report)

17 June 2022

     23:46  Global traffic routing diffhist −12 Krinkle (→‎Sites)
 m   23:42  Eqiad data center diffhist +11 Krinkle
 m   23:41  Eqsin data center diffhist −7 Krinkle ({{Infobox data centre}})
     23:41  Drmrs data center diffhist −8 Krinkle ({{Infobox data centre}})
 m   23:40  Esams data center diffhist −7 Krinkle ({{Infobox data centre}})
     23:40  Codfw data center diffhist −8 Krinkle ({{Infobox data centre}})
     23:39  Ulsfo data center diffhist −7 Krinkle ({{Infobox data centre}})
     23:39  Template:Infobox data centre inline diffhist −2,189 Krinkle (Krinkle moved page Template:Infobox data centre inline to Template:Infobox data centre)
Nm   23:39  Template:Infobox data centre diffhist +2,247 Krinkle (Krinkle moved page Template:Infobox data centre inline to Template:Infobox data centre)
     23:30  Template:ClusterMap1 diffhist −2,843 Krinkle (preserve history, but redirect to current version to avoid red links)
     23:30  Template:ClusterMap diffhist +100 Krinkle (Convert to high contrast hand drawn map, as proposed by User:Effie Mouzeli at Template:ClusterMap1 in May 2019)
     23:26  Template:ClusterMap/style.css diffhist +41 Krinkle
     23:06  Category:Servers diffhist +8 Krinkle
     23:05  Help:Glossary diffhist −7 Krinkle
     23:04  Kubernetes/Clusters diffhist −8 Krinkle
 m   23:04  Wikidough diffhist +4 Krinkle
 m   23:04  Cache servers diffhist +4 Krinkle
     23:03  Torrus.wikimedia.org diffhist −17 Krinkle
     23:03  Wikimedia infrastructure diffhist +8 Krinkle (as per Terminology and ops-l March 2022 thread.)
     23:03  Get involved diffhist −3 Krinkle (as per Terminology and ops-l March 2022 thread.)
     23:02  Wikitech-static diffhist +5 Krinkle
 m   23:02  Graphite.wikimedia.org diffhist +4 Krinkle (as per Terminology and ops-l March 2022 thread.)
 m   23:02  Terminology diffhist −9 Krinkle
     23:01  MediaWiki at WMF diffhist −220 Krinkle (→‎Infrastructure)
 m   22:58  Caching overview diffhist +1,035 Krinkle
 m   22:57  Traffic cache hardware diffhist +3 Krinkle
     22:57  Template:Navigation Wikimedia infrastructure diffhist +33 Krinkle
     22:05  Server Admin Log‎‎ 2 changes history +46,243 [Stashbot‎ (2×)]
     
22:05 (cur | prev) +16,273 Stashbot (AndyRussG: update payments-wiki revision 10304f69 -> ef53c82e)
     
01:43 (cur | prev) +29,970 Stashbot (pt1979@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs1017.eqiad.wmnet with reason: host reimage)
     22:04  HAProxy diffhist +356 Krinkle (Shift down one level for MariaDB docs, and hoist/elaborate section about edge caching, and update intro, + nav template)
     21:38  Talk:Kafka diffhist +350 BCornwall (→‎Confluent package link goes to empty dir: I updated the link)
     21:37  Requestctl diffhist +61 Krinkle
     21:36  Kafka diffhist +10 BCornwall (Change 404'd apt link in intro to current apt URL. Not future-proof as it's pointing to bullseye specifically but it's better than nothing)
     21:16  Incident response/Full report template diffhist +54 Krinkle
     21:14  Incident response/Lightweight report template diffhist +45 Krinkle
     21:09  Parsoid diffhist +145 C. Scott Ananian (→‎Prepare the vendor patch: add the gen_deploy_log.sh command.)
     21:07  Performance/Runbook/Measure backend performance diffhist +123 Krinkle (→‎See also: +mw:Manual:Profiling)
     20:04  Nova Resource:Codesearch/SAL diffhist +79 Stashbot (legoktm: added Krinkle as project admin, Gerrit +2)
     19:46  Purged diffhist −1 BCornwall (Fix broken grafana link)
 m   19:16  SRE/Dc-operations diffhist 0 JJMC89 (Reverted edits by Wandakazi mnyamane (talk) to last revision by Md Tanbir Islam)
     19:05  Wikimediastatus.net/User instructions diffhist +51 Krinkle
 m   18:12  Incidents/20120607-LastModifiedExtension diffhist −8 Krinkle
     17:53  Nova Resource:Admin/SAL diffhist +251 Stashbot (andrewbogott: switching to a new python-based health check for galera and haproxy. This may make things more stable, or it may not. T310664)
     17:47  Portal:Cloud VPS/Admin/Infrastructure projects diffhist −295 Majavah (→‎admin-monitoring: add)
     17:15  Release Engineering/SAL‎‎ 2 changes history +522 [Stashbot‎ (2×)]
     
17:15 (cur | prev) +104 Stashbot (ori: provisioned deployment-cache-text07 in deployment-prep to test query normalization via VCL)
     
01:08 (cur | prev) +418 Stashbot (TimStarling: on deployment-docker-cpjobqueue01 and deployment-docker-changeprop01 I redeployed the changeprop configuration, reverting the PHP 7.4 hack)
     15:55  User:SRodlund/PAWS (staging) diffhist −20 Krinkle (rm unknown single member category from draft. looks like it might be good for Category:Toolforge tools when published.)
     15:53  Template:Server diffhist −94 Krinkle
     15:51  Template:Server/status diffhist −115 Krinkle (No longer tracking online status, only "active" or "decom")
     15:51  Obsolete:Dataset1001 diffhist −12 Krinkle
 m   15:49  Incidents/2022-05-31 Analytics Hadoop failure diffhist +550 Btullis (→‎Summary: Updated impact summary)
N    15:49  Template:Server/status/active diffhist +118 Krinkle
     15:49  Template:Server/status/online diffhist −75 Krinkle (Krinkle moved page Template:Server/status/online to Template:Server/status/active: No longer tracking online status, only "active" or "decom")
     15:48  Storage2 diffhist −21 Krinkle (offline -> decom)
     15:47  Rutherfordium diffhist −2 Krinkle (offline -> decom)
     15:47  Mwlog1001 diffhist −2 Krinkle (offline -> decom)
     15:47  Bast3001 diffhist −68 Krinkle
 m   15:46  Mwmaint1002 diffhist −66 Krinkle
 m   15:46  Mwlog1002 diffhist −66 Krinkle
 m   15:46  Contint1001 diffhist −81 Krinkle
 m   15:46  Labweb1001 diffhist −48 Krinkle
     15:45  People1002 diffhist −19 Krinkle
     15:45  Phab1001 diffhist −19 Krinkle
     15:45  Mwdebug1003 diffhist −48 Krinkle
     15:45  People1003 diffhist −19 Krinkle
     15:45  Bast4001 diffhist −19 Krinkle
     15:45  Phab2001 diffhist −19 Krinkle
     14:53  Analytics/Systems/Airflow/Developer guide diffhist −58 Xcollazo (→‎SparkSQLOperator: Make link permalink.)
N    14:15  User:Samtar/common.js diffhist +808 Samtar (Created page with "mw.loader.load('//meta.wikimedia.org/w/index.php?action=raw&ctype=text/javascript&title=MediaWiki:MoreMenu.import.js'); mw.loader.load('//en.wikipedia.org/w/index.php?title=User:Mxn/CommentsInLocalTime.js&action=raw&oldid=1029979061&ctype=text/javascript'); mw.loader.load('//en.wikipedia.org/w/index.php?title=User:PleaseStand/userinfo.js&oldid=1017091544&action=raw&ctype=text/javascript'); mw.loader.load('//en.wikipedia.org/w/index.php?title=User:Enterprisey/diff-permali...")
     13:44  Netbox diffhist −142 Ayounsi
N    12:21  User:Uzume diffhist +173 Uzume (+)
     11:00  Incidents/2022-05-25 de.wikipedia.org diffhist +118 Alexandros Kosiaris (Update scorecard per action items from the Incident review ritual)
     10:52  Kubernetes/Kubernetes Workshop diffhist +8,458 Mbaoma (→‎Getting Started)
     10:24  CAS-SSO/Administration diffhist +203 Muehlenhoff (→‎Updating the CAS Debian package: Add a pointer to docs for how to rebase to a new CAS upstream release)
     10:03  Kubernetes/Kubernetes Workshop/Load Testing diffhist +526 Mbaoma
N    09:53  File:50 users from Mumbai.png diffhist +20 Mbaoma
N    09:52  File:50 users from São Paulo.png diffhist +24 Mbaoma
N    09:50  File:K8s Test results.png diffhist +16 Mbaoma
     08:35  Incidents/2022-05-24 Failed Apache restart diffhist −1,047 Alexandros Kosiaris
     02:15  User:AndreaWest/Blazegraph Features and Capabilities diffhist +562 AndreaWest (→‎wikibase:label)
     01:06  Changeprop diffhist +1,186 Tim Starling (→‎Deploying: unholy sshpipe automation)
 m   00:00  Map of database maintenance diffhist −670 Dexbot (Bot: Updating the report)

16 June 2022

     22:32  Memcached for MediaWiki diffhist +38 Krinkle (→‎Magic numbers)
     22:03  Deployments/Yearly calendar/2022 diffhist +76 Thcipriani
     21:42  Email System Revamp diffhist +151 JHathaway
     21:16  Nova Resource:Tools.lexeme-forms/SAL diffhist +101 Stashbot (wm-bot: <lucaswerkmeister> deployed 466976ba49 (l10n updates))
     21:13  Nova Resource:Quarry/SAL diffhist +193 Stashbot (Rook: 806271: Prettify User not found page | https://gerrit.wikimedia.org/r/c/analytics/quarry/web/+/806271 8002e6f06c64568441d7fc5ddc70ea2525a4c6fb)
     20:19  Deployments diffhist +11,267 DannyS712 (→‎{{Deployment_day|date=2022-06-16}}: +1)
Nm   17:31  Incidents/2022-05-31 Analytics Hadoop failure diffhist +5,629 Krinkle (Krinkle moved page Incidents/2022-05-31 Analytics Data Lake - Hadoop Namenode failure to Incidents/2022-05-31 Analytics Hadoop failure)
     17:31  Incidents/2022-05-31 Analytics Data Lake - Hadoop Namenode failure diffhist −4,719 Krinkle (Krinkle moved page Incidents/2022-05-31 Analytics Data Lake - Hadoop Namenode failure to Incidents/2022-05-31 Analytics Hadoop failure)
     17:30  Netbox diffhist +542 Ayounsi (add nbshell)
N    17:13  User talk:Dexbot diffhist +33 Nintendofan885 (redirect bot talk to owner)
N    17:13  User:Dexbot diffhist +122 Nintendofan885 (create)
     16:52  Incidents/2022-06-14 overload varnish / haproxy diffhist −2,243 Krinkle (Changed redirect target from Incidents/2022-06-10 overload varnish / haproxy to Incidents/2022-06-10 overload varnish haproxy)
Nm   16:51  Incidents/2022-06-10 overload varnish haproxy diffhist +2,302 Krinkle (Krinkle moved page Incidents/2022-06-10 overload varnish / haproxy to Incidents/2022-06-10 overload varnish haproxy)
N    16:51  Incidents/2022-06-10 overload varnish / haproxy diffhist +59 Krinkle (Krinkle moved page Incidents/2022-06-10 overload varnish / haproxy to Incidents/2022-06-10 overload varnish haproxy)
     14:18  Nova Resource:Metricsinfra/SAL diffhist +92 Stashbot (taavi: add 'gitlab-runners' project to list of scraped projects)
N    11:00  Kubernetes/Kubernetes Workshop/Using WMF Kubernetes to run your services diffhist +24,668 Mbaoma
     10:24  MediaWiki at WMF diffhist −176 Krinkle (→‎{{Anchor|Versioned resources|Unversioned resources}}Static files)
     09:49  VXLAN-EVPN Network Testing - Sonic on Dell switches diffhist −102 Cathal Mooney (→‎Test Topology)
     09:02  Kubernetes/Kubernetes Workshop/Kubernetes at the Wikimedia Foundation (WMF) diffhist +8,916 Mbaoma
     05:54  IP and AS allocations diffhist −915 Ayounsi (moved AS numbers to Netbox)
     00:16  User talk:Btullis diffhist 0 Btullis
 m   00:01  Map of database maintenance diffhist +382 Dexbot (Bot: Updating the report)

15 June 2022

     23:12  User:Quiddity/common.js diffhist −90 Quiddity (rm hiddencats expando)
     22:48  Server Admin Log diffhist +61,049 Stashbot (marostegui@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1184 (T310011)', diff saved to https://phabricator.wikimedia.org/P29867 and previous config saved to /var/cache/conftool/dbconfig/20220615-224845-marostegui.json)
     22:39  Release Engineering/SAL diffhist +259 Stashbot (brennen: phabricator: tagged release/2022-06-15/1 (T310742))
N    22:12  Kubernetes/Kubernetes Workshop/Manageability and Security diffhist +10,319 Mbaoma
N    22:11  Kubernetes/Kubernetes Workshop/Kubernetes at the Wikimedia Foundation (WMF) diffhist +341 Mbaoma (Created page with " == Next Module == Module 10: == <div style="text-align: right; direction: ltr; margin-left: 1em;"> Previous Module == <div style="text-align: right; direction: ltr; margin-left: 1em;"> Module 8: Manageability and Security <div> Category:Kubernetes Category: How-To")
N    21:47  Kubernetes/Kubernetes Workshop/Kubernetes Package Manager diffhist +6,166 Mbaoma (→‎Next Module)
     21:20  Kubernetes/Kubernetes Workshop/Building a production ready cluster diffhist +15,214 Mbaoma (→‎Next Module)
N    21:05  File:UI of Add Rule.png diffhist +14 Mbaoma
N    21:00  File:UI of create a proxy.png diffhist +20 Mbaoma
N    20:33  File:Modified Kubernetes Architecture Chart.png diffhist +38 Mbaoma
     20:30  Nova Resource:Devtools/SAL diffhist +317 Stashbot (mutante: - created gitlab-runner-1002 - applied puppet role - attached cinder volume "docker" - running puppet again)
 m   19:55  User:AndreaWest/WDQS Testing diffhist +390 AndreaWest (→‎Testing Wikidata-Specific Updates and Queries)
     19:09  Incidents/2022-05-09 confctl diffhist +567 Krinkle (document cause and why it only affected Codfw)
     18:58  Nova Resource:Tools.wikibugs/SAL diffhist +59 Stashbot (taavi: full shutdown + restart)
     18:49  Nova Resource:Cvn/SAL diffhist +140 Stashbot (AntiComposite: /cs flags #cvn-wp-en Zppix voiced)
     18:41  Incidents/2022-05-24 Failed Apache restart diffhist +1,120 Krinkle
 m   18:40  Server Admin Log/Archive 45 diffhist −1 Krinkle
     18:40  PyBal diffhist −12 Krinkle
     18:15  Incidents/2022-05-31 Analytics Data Lake - Hadoop Namenode failure diffhist +415 Ottomata
     18:05  Incident response/Report editintro diffhist +7 Krinkle (use messagebox styling, updated oudated links)
     18:03  Nova Resource:Paws/SAL diffhist +222 Stashbot (Rook: removing unnecessary yaml #167 f4733023a1b63e48d12f99da5344af5513aea245)
     18:02  Incident status diffhist +81 Krinkle (Change input field such that date is within the input, for easier modification and creating of yesterday/backdated reports)
     17:58  Incidents/2022-05-26 Database hardware failure diffhist +27 Krinkle
N    17:58  M1 diffhist +35 Krinkle (Redirected page to MariaDB#Miscellaneous)
 m   17:56  Monitoring/strongswan diffhist +6 BCornwall (Pedantic ipsec spelling)
     17:56  Incidents/2022-05-25 de.wikipedia.org diffhist +92 Krinkle
     17:32  Provenance diffhist +142 Pcoombe (→‎Currently reserved values: + fundraising TY banners)
     17:19  Incidents/2022-05-09 exim-bdat-errors diffhist +212 Krinkle
     17:12  Incident response/Full report template diffhist −44 Krinkle (→‎Actionables)
     17:12  Incident response/Lightweight report template diffhist −44 Krinkle (→‎Actionables)
N    17:11  Incidents/2022-05-21 varnish cache busting diffhist +2,370 Krinkle
     17:08  Incidents/2022-05-05 Wikimedia full site outage diffhist +79 Krinkle (https://www.wikimediastatus.net/incidents/xzmd6vwvvgmx)
     17:02  Incidents/2022-05-21 - varnish cache busting diffhist −2,321 Krinkle (Krinkle moved page Incidents/2022-05-21 - varnish cache busting to Incidents/2022-05-21 varnish cache busting)
     16:50  Incidents/2022-05-02 deployment diffhist −95 Krinkle (mw:Inclusive language)
     16:49  Bacula diffhist +12 Krinkle
     16:36  Incidents/2022-05-01 etcd diffhist +215 Krinkle (clarify, and make explicit that public services were not affected (I think?))
     16:32  Main Page diffhist 0 Krinkle
     15:08  Scap/Release diffhist −2,424 Jaime Nuche (Edited for clarity)
     14:06  Deployments‎‎ 2 changes history +1,621 [Thcipriani‎; Awight‎]
 m   
14:06 (cur | prev) +393 Awight (→‎{{Deployment_day|date=2022-06-15}}: mark patches as deployed)
     
00:06 (cur | prev) +1,228 Thcipriani
     13:28  User:SRodlund diffhist +15 Aklapper (+{{historical}})
     12:14  Netbox diffhist +3,318 Ayounsi (→‎Monitoring)
     11:34  Nova Resource:Admin/SAL diffhist +86 Stashbot (taavi: restart neutron-linuxbridge-agent on cloudvirt1022)
 m   00:02  Map of database maintenance diffhist +750 Dexbot (Bot: Updating the report)

14 June 2022

     23:52  Server Admin Log‎‎ 2 changes history +90,039 [Stashbot‎ (2×)]
     
23:52 (cur | prev) +44,141 Stashbot (mutante: gitlab-runner1001/1002 - clean revert not possible, icinga alerting about failed buildkitd service, manually deleting systemd unit and trying to clean up T308271)
     
00:36 (cur | prev) +45,898 Stashbot (marostegui@cumin1001: dbctl commit (dc=all): 'Repooling after maintenance db1147 (T310011)', diff saved to https://phabricator.wikimedia.org/P29701 and previous config saved to /var/cache/conftool/dbconfig/20220614-003608-marostegui.json)
     23:29  Nova Resource:Devtools/SAL diffhist +225 Stashbot (mutante: - creating instance gitlab-runner-1001 since we did not have a test machine for gitlab-runners but need one to test things like gerrit:791655 before hitting prod T308271)
     22:24  Nova Resource:Tools.lexeme-forms/SAL diffhist +101 Stashbot (wm-bot: <lucaswerkmeister> deployed b0143851e0 (l10n updates))
     21:59  Fundraising/Team processes/Onboarding diffhist +84 Wfan (→‎Accounts & Privileges to get)
     21:48  User:AndreaWest/WDQS Testing diffhist +21 AndreaWest (→‎Testing Wikidata-Specific Updates and Queries: Simplified classification of the query patterns)
     21:30  Release Engineering/SAL‎‎ 2 changes history +924 [Stashbot‎ (2×)]
     
21:30 (cur | prev) +529 Stashbot (TheresNoTime: clear out stuck `beta-scap-sync-world` jobs (repeatedly per each queued `beta-mediawiki-config-update-eqiad` job), queued jobs now running. monitored for until each job had run successfully. jobs up to date)
     
00:42 (cur | prev) +395 Stashbot (TimStarling: on deployment-deploy03 removed helm2, as was done in production)
     20:46  Netbox diffhist +3,797 Ayounsi (refactoring, still WIP)
N    20:18  Kubernetes/Kubernetes Workshop/Refactoring code using Kubernetes features diffhist +12,925 Mbaoma
N    20:17  Kubernetes/Kubernetes Workshop/Building a production ready cluster diffhist +410 Mbaoma (Created page with "== Overview == At the end of this module, you should be able to: * Use kubespray to set up a cluster. * Set up a cluster on Google’s Kubernetes Service (GKE). == Next Module == Module 7: == <div style="text-align: right; direction: ltr; margin-left: 1em;"> Previous Module == <div style="text-align: right; direction: ltr; margin-left: 1em;"> Module 5: <div> Category:Kubernetes Category: How-To")
N    19:25  Kubernetes/Kubernetes Workshop/Autoscaling in Kubernetes diffhist +1,973 Mbaoma
 m   18:51  Maps/v2/Metrics diffhist 0 RLazarus
N    18:31  Fundraising/Cluster/Payments staging server diffhist +1,438 Jgreen (→‎Deploying to staging)
     18:23  Kubernetes/Kubernetes Workshop/Setting up Infrastructure as Code (IaC) in Kubernetes diffhist +15,742 Mbaoma
     16:26  Nova Resource:Admin/SAL diffhist +1,288 Stashbot (wm-bot2: OSDs (['cloudcephosd1001', 'cloudcephosd1002', 'cloudcephosd1003', 'cloudcephosd1004', 'cloudcephosd1005', 'cloudcephosd1006', 'cloudcephosd1007', 'cloudcephosd1008', 'cloudcephosd1009', 'cloudcephosd1010', 'cloudcephosd1011', 'cloudcephosd1012', 'cloudcephosd1013', 'cloudcephosd1014', 'cloudcephosd1015', 'cloudcephosd1016', 'cloudcephosd1017', 'cloudcephosd1018', 'cloudcephosd1019', 'cloudcephosd1020', 'cloudcephosd1021', 'cl)
     16:08  Fundraising diffhist +73 Cstone (→‎E Cluster layout, deployment, codebases)
     15:07  Analytics/Systems/DataHub/Upgrading diffhist +423 Milimetric
     13:40  User:Huji diffhist +37 Huji
     13:11  Nova Resource:Paws/SAL diffhist +123 Stashbot (Rook: Removing leftover db-proxy bits #170 8795bf7ded5eb2406a7dce70d8bd4450f6b2511d)
N    12:57  Incidents/2022-06-14 overload varnish / haproxy diffhist +2,302 Filippo Giunchedi
     08:06  Portal:Cloud VPS/Admin/Runbooks/Check for VMs leaked by the nova-fullstack test diffhist 0 David Caro (→‎No active metadata service found)
     06:03  User:Marostegui diffhist +3 Marostegui
     03:01  User:AndreaWest/WDQS Testing/Running Iguana diffhist +3,675 AndreaWest (→‎Code Modifications)
 m   00:37  Map of database maintenance diffhist +521 Dexbot (Bot: Updating the report)

13 June 2022

     23:21  Deployments diffhist +688 Bartosz Dziewoński (→‎Tuesday, June 14: +)
     20:31  User:AndreaWest/WDQS Testing diffhist +4,790 AndreaWest (→‎Wikidata Triples for Stress Testing)
 m   20:30  User:AndreaWest/WDQS Testing/Running Iguana diffhist +1 AndreaWest (→‎Testing Overview)
     20:07  Search/adding-nodes-to-deployment-prep diffhist +29 Ori (→‎Puppet Master (puppet server))
     16:56  Performance/Runbook/Measure backend performance diffhist +1,977 Krinkle (→‎Benchmarking: Copy from mw:PHP optimisation tips#Benchmarking, as written by @Tim Starling on 2 September 2021‎.)
     16:22  MediaWiki talk:Edittools diffhist +120 Krinkle (→‎Charinsert: Reply)
     16:21  MediaWiki:Edittools diffhist +430 Krinkle
     12:53  Mailman diffhist +135 Slyngshede (→‎Create a mailing list)
     12:48  Talk:Puppet coding diffhist +104 Jbond (→‎Moving this under Puppet and splitting up the content: Reply)
     12:33  Remove a message from mailing list archive diffhist +2 Ladsgroup
     12:23  Management Interfaces diffhist +398 MVernon (→‎Is remote IPMI enabled?: Add note on an HP peculiarity (and how to fix it))
N    11:40  Kubernetes/Kubernetes Workshop/Setting up Infrastructure as Code (IaC) in Kubernetes diffhist +673 Mbaoma
N    11:36  Kubernetes/Kubernetes Workshop/Build a service application on Kubernetes diffhist +7,103 Mbaoma
     11:14  Nova Resource:Admin/SAL diffhist +876 Stashbot (wm-bot2: Finished rebooting node cloudcephosd1021.eqiad.wmnet (T309789) - cookbook ran by dcaro@vulcanus)
     10:43  Kubernetes/Clusters/New diffhist +7 Klausman (→‎Istio)
N    10:24  Kubernetes/Kubernetes Workshop/Set up a batch application on Kubernetes diffhist +8,045 Mbaoma (→‎Previous Module)
     09:42  SRE/Infrastructure Foundations/Team diffhist +318 Jobo
     09:32  Help:Cloud VPS Instances diffhist +3,525 Mbaoma
N    09:09  File:Launch Instance - SSH.png diffhist +21 Mbaoma
N    08:37  File:Launch Instance - Step 7b.png diffhist +25 Mbaoma
N    08:36  File:Launch Instance - Step 7.png diffhist +24 Mbaoma
N    08:35  File:Launch Instance - Step 6.png diffhist +24 Mbaoma
N    08:35  File:Launch Instance - Step 5.png diffhist +24 Mbaoma
N    08:31  File:Launch Instance - Step 3.png diffhist +24 Mbaoma
N    08:28  File:Launch Instance - Step 2b.png diffhist +25 Mbaoma
N    08:27  File:Launch Instance - Step 2.png diffhist +24 Mbaoma
N    08:26  File:Launch Instance - Step 1.png diffhist +24 Mbaoma
N    08:24  File:Instances 2.png diffhist +45 Mbaoma
N    07:56  File:Instances.png diffhist +37 Mbaoma
N    07:49  File:2FA.png diffhist +25 Mbaoma
 m   00:02  Map of database maintenance diffhist −240 Dexbot (Bot: Updating the report)

12 June 2022

     21:13  Release Engineering/SAL diffhist +96 Stashbot (Reedy: Reloading Zuul to deploy https://gerrit.wikimedia.org/r/804777)
     18:31  Server Admin Log‎‎ 2 changes history +8,610 [Stashbot‎ (2×)]
     
18:31 (cur | prev) +4,306 Stashbot (andrew@cumin1001: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host clouddumps1002.wikimedia.org with OS bullseye)
     
01:46 (cur | prev) +4,304 Stashbot (andrew@cumin1001: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddumps1001.wikimedia.org with reason: host reimage)
     15:57  Help talk:Access to Cloud VPS instances with PuTTY and WinSCP diffhist +351 Ameisenigel (→‎Access Cloud VPS project with PuTTY/WinSCP: new section)
 m   00:01  Map of database maintenance diffhist 0 Dexbot (Bot: Updating the report)

11 June 2022

     23:12  User:Resident Mario diffhist 0 Quiddity (rvself, outdated lint list)
 m   23:08  Analytics/Systems/AQS/Scaling/LoadTesting diffhist +6 Quiddity (close tag)
 m   22:59  Php.ini diffhist +9 Quiddity (fix dbl rdr)
 m   22:59  Wmerrors