You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Recent changes

Jump to navigation Jump to search

Track the most recent changes to the wiki on this page. See main namespace without noise.

Recent changes options Show last 50 | 100 | 250 | 500 changes in last 1 | 3 | 7 | 14 | 30 days
Hide registered users | Hide anonymous users | Hide my edits | Show bots | Hide minor edits
Show new changes starting from 22:30, 21 October 2021
   
List of abbreviations:
N
This edit created a new page (also see list of new pages)
m
This is a minor edit
b
This edit was performed by a bot
(±123)
The page size changed by this number of bytes

21 October 2021

     00:39  Deployments diffhist +12,404 Thcipriani
     00:06  Server Admin Log diffhist +14,742 Stashbot mwdebug-deploy@deploy1002: helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' .

20 October 2021

     22:14  Nova Resource:Tools.magnustools/SAL diffhist +101 Stashbot wm-bot: <root> Removed 2 empty records from public_html/js_toolinfo.json
     21:55  Nova Resource:Tools.gerrit-patch-uploader/SAL diffhist +173 Stashbot wm-bot: <legoktm> updated toolinfo.json with https://github.com/valhallasw/gerrit-patch-uploader/commit/0aa5bbe17271bc79c77e6b0b453a9573a41adf80
     21:48  Nova Resource:Tools.dicompte/SAL diffhist +121 Stashbot wm-bot: <root> Fixed syntax errors in toolinfo.json so that Toolhub will index this project.
     21:06  Nova Resource:Admin/SAL diffhist +99 Stashbot andrewbogott: creating cloudinfra-nfs project T293936
N    21:06  Nova Resource:Cloudinfra-nfs diffhist +119 Labslogbot Auto update of instance info.
     21:03  Nova Resource:Tools.sal/SAL diffhist +101 Stashbot wm-bot: <bd808> Updated to 7c753d1 (T289739)
     20:17  Nova Resource:Tools.stashbot/SAL diffhist +83 Stashbot wm-bot: <bd808> Updated stashbot to e54e6da
 m   19:57  RPKI diffhist +875 Dzahn →‎Wikimedia servers
 m   19:24  Logstash diffhist −2,519 Cwhite →‎Deployment
N    19:23  Incident documentation/2021-09-26 appserver latency diffhist +947 Krinkle Created page with "{{irdoc|status=review}} ==Summary== Increased db load for enwiki (s1) resulted in slower responses, which in turn resulted in overall php-fpm worker limits being reached and thus affecting requests for all wikis. For requests above the limit, the error was "''upstream connect error or disconnect/reset before headers. reset reason: overflow''". '''Impact''': For about 15 minutes, backend appservers were slower or unable to respond for all wikis. This mainly affected logg..."
Nm   19:14  Incident documentation/2019-07-16 docker registry diffhist +5,018 Krinkle Krinkle moved page Incident documentation/20190716-docker-registry to Incident documentation/2019-07-16 docker registry
     19:14  Incident documentation/20190716-docker-registry diffhist −4,955 Krinkle Krinkle moved page Incident documentation/20190716-docker-registry to Incident documentation/2019-07-16 docker registry
Nm   19:14  Incident documentation/2019-07-23 logstash diffhist +3,266 Krinkle Krinkle moved page Incident documentation/20190723-logstash to Incident documentation/2019-07-23 logstash
     19:14  Incident documentation/20190723-logstash diffhist −3,210 Krinkle Krinkle moved page Incident documentation/20190723-logstash to Incident documentation/2019-07-23 logstash
     19:14  Template:Irdoc diffhist +4 Krinkle
N    19:13  Incident documentation/2021-09-18 appserver latency diffhist +617 Krinkle
     19:04  Incident documentation/2021-09-06 Wikifeeds diffhist +114 Krinkle
     18:41  Incident documentation/2021-09-13 cirrussearch restart diffhist −73 Krinkle
     18:39  Wikifeeds diffhist +22 Krinkle →‎Service level indicators/objectives (SLIs/SLOs)[1]
N    18:31  Incident documentation/2021-09-04 appserver latency diffhist +1,333 Krinkle
N    18:17  File:4-Sep-2021-latency-quantile.png diffhist +58 Krinkle
N    18:16  File:4-Sep-2021-latency.png diffhist +48 Krinkle
N    18:09  File:4-Sep-2021-http-status.png diffhist +52 Krinkle
     16:07  User:Elukey/MachineLearning/Deploy diffhist +4,340 Elukey →‎How to add a new helmfile config
     15:41  Nova Resource:Tools/SAL diffhist +276 Stashbot majavah: removing toollabs-webservice from grid exec and master nodes where it's not needed and not managed by puppet
     15:36  Nova Resource:Tools.quarry/SAL diffhist +97 Stashbot mdipietro: please don't use this logging queue, use "quarry" instead
     15:36  Nova Resource:Tools.paws/SAL diffhist +95 Stashbot mdipietro: please don't use this logging queue, use "paws" instead
     15:14  User:Nhatminh01 diffhist −1 Nhatminh01 Changed redirect target from m:User:Nhatminh01 to m:User:Thingofme
     13:45  Wikimedia Cloud Services team/EnhancementProposals/ceph client refactor diffhist +3,657 David Caro →‎abstraction
     12:15  Nova Resource:Toolsbeta/SAL diffhist +119 Stashbot majavah: upload toolforge-webservice 0.78 to stretch,buster,bullsye-toolsbeta repositories
     11:45  SRE/Clinic Duty diffhist +22 Faidon Liambotis →‎Schedule: Change clinic duty for 2021-11-01
 m   09:43  Analytics/Systems/Cluster/Hadoop/Alerts diffhist +259 Btullis →‎Paging: Added details about the audit log
     01:31  Server Admin Log diffhist +27,590 Stashbot mwdebug-deploy@deploy1002: helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' .

19 October 2021

     23:54  Deployments diffhist −10,731 Tks4Fish →‎{{Deployment_day|date=2021-10-19}}: +1
Nm   22:44  User:Cwhite/OpenSearch diffhist +2,825 Cwhite
     22:24  Release Engineering/SAL diffhist +542 Stashbot brennen: gitlab-test: deleted unused gitlab-runner-test
     20:56  Nova Resource:Tools.notwikilambda/SAL diffhist +480 Stashbot wm-bot: <lucaswerkmeister> previous log messages all related to T293828
     20:19  Analytics/Data Lake diffhist +168 Neil P. Quinn-WMF Add info on floating-point number types in Spark and Presto
     18:02  Memcached for MediaWiki diffhist +7 Effie Mouzeli →‎Service
     17:39  Nova Resource:Tools.bridgebot/SAL diffhist +111 Stashbot wm-bot: <bd808> Enabled keepnick module for ZNC and flushed configuration to disk.
 m   16:52  SRE/Infrastructure naming conventions diffhist +17 RLazarus
     16:18  Heterogeneous deployment diffhist +24 Jforrester →‎Change on deployment.eqiad.wmnet then commit: This shouldn't be done frequently, as it bypasses CI.
 m   16:02  Cumin diffhist +705 Volans →‎Host selection: Add link to the global grammar for mixed cases
     15:54  Nova Resource:Toolhub/SAL diffhist +76 Stashbot bd808: Update demo server to 617dea7
     15:43  Event Platform/Instrumentation How To diffhist +3,384 Ottomata →‎Overriding event stream config settings
     15:36  User:Elukey/MachineLearning/Deploy diffhist +14,991 Elukey →‎Test your model after deployment
N    15:35  User:Razzi/superset 1.3.1 errors diffhist +1,943 Razzi Created page with "<pre> Creating a new presto database view, pasting in presto://an-coord1001.eqiad.wmnet:8281/analytics_hive?protocol=https Testing connection yields ERROR: (builtins.NoneType) None (Background on this error at: http://sqlalche.me/e/13/dbapi) This makes sense because we haven't configured security. Here's the config from production: { "metadata_params": {}, "engine_params": { "connect_args": { "KerberosConfigPath": "/etc/krb5.conf", "KerberosKeyt..."
     13:52  Event Platform/Stream Configuration diffhist +35 Ottomata →‎stream
     13:51  Event Platform/EventStreams/Administration diffhist −436 Ottomata →‎Configuration
     13:48  Event Platform/EventGate/Administration diffhist +116 Ottomata →‎EventStreamConfig change
     12:43  Graphite diffhist +762 Filippo Giunchedi →‎Operations manual
N    11:55  User:Khr2003 diffhist +621 Khr2003 Created page with "{{User info | full name = Abdul Al-Hasany | about me = Full Stack PHP programmer. I enjoy working with open source and explore new projects. | about my work = * Major contributions to: ** [https://videocuttool.wmflabs.org/ VideoCutTools] * WordPress plugins: ** [https://wordpress.org/plugins/editor-block-outline/ Editor Block Outline] ** [https://wordpress.org/plugins/nashaat-activity-log/ Nashaat Activity Log] | contact me = * Twitter: [https://twitter.com/KalimahApps]..."
 m   11:28  Analytics/Systems/Cluster/Hadoop/Alerts diffhist +3,891 Btullis →‎HDFS Capacity Remaining: Added severity levels.
     10:37  Wikimedia Cloud Services team/EnhancementProposals/ceph client refactor diffhist +534 David Caro
N    09:13  User talk:GiFontenelle diffhist +679 StrikerBot Welcome to Toolforge!
     08:06  Nova Resource:Puppet-diffs/Documentation diffhist +825 Hashar add commands for integration and deployment-prep syncing
 m   04:38  Analytics/Data Lake/Traffic/Webrequest diffhist −1 Effeietsanders typo
     00:38  Server Admin Log diffhist +11,211 Stashbot ryankemper@cumin1001: START - Cookbook sre.wdqs.data-transfer

18 October 2021

     22:59  Analytics/Systems/Superset diffhist +32 Razzi →‎Upgrading: Add back necessary `export PYTHONPATH=/etc/superset `
     21:43  SRE/Clinic Duty/Access requests diffhist +523 Dzahn (click "members"->"add members",
     21:41  SRE/LDAP diffhist +262 Dzahn (click 'members'->'add members')
     20:46  Nova Resource:Tools.morfeusz/SAL diffhist +86 Stashbot wm-bot: <peterbowman> Implement concraft-pl disambiguator
     19:46  Event Platform/Stream Configuration diffhist +80 Ottomata →‎Common Settings Documentation
     19:46  Event Platform/Instrumentation How To diffhist +132 Ottomata →‎Stream Configuration
     19:41  Deployments diffhist +809 Bartosz Dziewoński →‎Tuesday, October 19: +
     19:21  Nova Resource:Admin/SAL diffhist +252 Stashbot andrewbogott: also ticked the 'admin' box on wikitech for majavah T292827
N    18:42  User talk:Lawandlasun diffhist +678 StrikerBot Welcome to Toolforge!
N    17:40  User:AKhatun/Wikidata Subgraph Analysis diffhist +460 AKhatun Form article structure
N    17:10  Wikimedia Cloud Services team/EnhancementProposals/ceph client refactor diffhist +2,664 Arturo Borrero Gonzalez →‎Problem statement: typos
     16:48  Help:Cloud Services Introduction diffhist −1 Aklapper remove "from Wikimedia Hackathon" details that I don't care about; add year to know how recent those external docs are
     16:26  Release Engineering/SAL diffhist +407 Stashbot Lucas_WMDE: killed one beta-scap-sync-world job that seemed stuck for a long time
     14:35  PAWS/PAWS and Pywikibot diffhist −154 Aklapper Use internal link
     14:13  Graphite diffhist +397 Filippo Giunchedi →‎Deleting metrics
     12:37  Nova Resource:Tools.lexeme-forms/SAL diffhist +101 Stashbot wm-bot: <lucaswerkmeister> deployed eae6c8d594 (l10n updates)

17 October 2021

N    22:14  User talk:Kaniivel diffhist +675 StrikerBot Welcome to Toolforge!
N    22:13  User talk:Debugger Byte diffhist +680 StrikerBot Welcome to Toolforge!
N    22:12  User talk:WQuarshie diffhist +676 StrikerBot Welcome to Toolforge!
     16:40  Main Page diffhist +6 Majavah fix
     12:51  Nova Resource:Tools.pbbot/SAL diffhist +116 Stashbot wm-bot: <peterbowman> Display redlinks in SandboxRedirects servlet (e542bc5)
     00:48  Deployments diffhist +103 Juan90264 →‎{{Deployment_day|date=2021-10-18}}: +1 config Gerrit [731231]
N    00:15  User talk:Mgblabs diffhist +96 Mgblabs User-Talk-Page ~~~~ {} *
N    00:14  User:Mgblabs diffhist +95 Mgblabs MyUserPage ~~~~ {} *

16 October 2021

 m   17:15  Analytics/Team/Onboarding diffhist +1 Lucas Werkmeister →‎E-mail lists: fix typo
     16:47  Release Engineering/SAL diffhist +346 Stashbot Lucas_WMDE: root@deployment-cache-upload06:~# touch /srv/trafficserver/tls/etc/ssl_multicert.config && systemctl reload trafficserver-tls.service # T293070, based on T271808#6739578
     16:14  Main Page diffhist −43 Majavah we use "Cloud VPS" everywhere else, no need to put the full "Virtual Private Server" in here
     14:53  Nova Resource:Tools.lexeme-forms/SAL diffhist +600 Stashbot wm-bot: <lucaswerkmeister> deployed 1903c3d0eb (don’t show duplicate warning errors)
     11:52  Help:Labs labs labs diffhist +152 Majavah →‎Toolforge: expand a bit
 m   10:55  User:Mainframe98 diffhist 0 Mainframe98
     08:59  MariaDB/troubleshooting diffhist +310 Jcrespo →‎Overload due to long running queries
     08:56  MariaDB diffhist +25 Jcrespo →‎Long running queries
 m   08:53  Db-kill diffhist +115 Jcrespo how it complements query killer
     07:47  Nova Resource:Toolsbeta/SAL diffhist +118 Stashbot majavah: deployed cert-manager and wave as a test for automating T292238
 m   07:31  Wikimedia site requests diffhist +5 Ideophagous added missing word
     05:39  Memcached for MediaWiki diffhist −280 Effie Mouzeli →‎Service
     05:27  Memcached for MediaWiki/mcrouter diffhist +79 Effie Mouzeli
     03:56  Server Admin Log diffhist +266 Stashbot ryankemper@cumin1001: END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0)

15 October 2021

     23:48  Server Admin Log‎‎ 2 changes history +24,161 [Stashbot‎ (2×)]
     
23:48 (cur | prev) +5,052 Stashbot ryankemper@cumin1001: START - Cookbook sre.wdqs.data-transfer
     
00:09 (cur | prev) +19,109 Stashbot ryankemper@cumin1001: END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0)
     22:44  User:Magnus Manske/hay directory diffhist −674 BryanDavis BryanDavis changed the content model of the page User:Magnus Manske/hay directory from "JSON" to "wikitext": Using JSON content type restricts editing to interface admins?
 m   22:26  How to deploy code diffhist +1 Ammarpad →‎ResourceLoader and l10n messages
     22:18  User talk:Bstorm diffhist +310 BryanDavis →‎I'm goating to miss you on the team: new WikiLove message
     21:29  Logstash/Common Logging Schema diffhist +5 BryanDavis →‎Required Fields: BOLD the version number match requirement, prettier link
     21:00  User:Bstorm diffhist −659 Bstorm Removing a few things...Will remove the meta links when they stop existing
     20:14  Db-kill diffhist +115 Legoktm fix display title
     20:13  MariaDB diffhist −549 Legoktm →‎Long running queries: actually just link to db-kill, which already exists!
     20:11  Query killer diffhist +2,430 Legoktm may not work properly under high load, link to db-kill for emergency manual intervention
     17:51  User:SDineshKumar/Learning&Dev diffhist +437 SDineshKumar added a priority task.
     17:50  Deployments/Train/Ideas diffhist +204 20after4 link mw:Wikimedia_Release_Engineering_Team/Project/Train2.0
     16:14  Release Engineering/SAL diffhist +149 Stashbot brennen: gitlab.wikimedia.org: apply max artifact size of 350M and artifact expiry of 2 weeks for T292372
     15:01  Nova Resource:Tools/SAL diffhist +129 Stashbot arturo: add updated ingress-nginx docker image in the registry (v1.0.1) for T293472
     14:33  Portal:Toolforge/Admin diffhist +30 BryanDavis →‎Deleting a tool (obsolete by-hand method): remove section title
     14:33  Portal:Toolforge/Admin/New grid engine exec host diffhist +34 David Caro →‎Debian Stretch/Son of Grid Engine grid
     10:44  Puppet/Pontoon diffhist −12 Filippo Giunchedi →‎Server bootstrap
     10:24  LibreNMS diffhist +360 Filippo Giunchedi →‎Upgrade LibreNMS
     09:43  WMDE/Wikidata/Alerts diffhist +399 Michael Große →‎Number of Rows in wb_changes table
     08:14  Deployments diffhist +183 Juan90264 →‎{{Deployment_day|date=2021-10-18}}: +1 config Gerrit [730744]
 m   04:01  Ticket.wikimedia.org diffhist −2 MABot Bot: Fixing broken redirect to moved target page VRT System
 m   04:00  Out-of-band management diffhist +16 MABot Bot: Fixing double redirect to Out-of-band network
 m   04:00  Incident documentation/20170607-WikiScrape diffhist −4 MABot Bot: Fixing double redirect to Incident documentation/2017-06-07 scrape

14 October 2021

N    23:32  Deployments/Train/Ideas diffhist +1,805 20after4 some train process ideas
     23:31  Deployments diffhist +1,330 Juan90264 →‎{{Deployment_day|date=2021-10-14}}: +2 config Gerrit [730736] and [730737]
     18:25  Analytics/Data access diffhist +431 Ottomata →‎Team specific (they do not grant access to PII data on Hadoop, for that see analytics-privatedata-users)
N    17:45  User:Elukey/MachineLearning/Deploy diffhist +3,047 Elukey →‎Helm at the WMF
     17:29  Changeprop diffhist −1,717 Hnowlan →‎To Kubernetes: No need for duplicate/outdated k8s docs!
     17:24  Portal:Toolforge/Admin/Kubernetes diffhist +308 Majavah →‎Custom admission controllers
     16:13  Release Engineering/SAL diffhist +215 Stashbot addshore: reloaded zuul for zuul: mw-ext-MailGun will need PHP 7.3+ T236396
     15:07  GitLab/Gitlab Runner diffhist +1,681 Jelto
     15:02  Nova Resource:Toolsbeta/SAL diffhist +470 Stashbot wm-bot: Joining grid node toolsbeta-sgewebgen-09-1.toolsbeta.eqiad1.wikimedia.cloud to the toolsbeta cluster - cookbook ran by dcaro@vulcanus
     14:28  Orchestrator diffhist +2,019 Kormat →‎Upgrading orchestrator: trimm trailing garbage
     14:24  SRE/Data Persistence/Backups/Runbooks diffhist +332 LSobanski
 m   14:08  SRE/Clinic Duty diffhist −179 Volans →‎Maintain the 'ops-maintenance' mails and calendar: Remind to mark calendar events as Free
     13:24  User:Jbond/debuging diffhist +77 Jbond
     12:31  Puppet/Pontoon diffhist +99 Filippo Giunchedi →‎Quickstart (five minutes)
     12:28  Nova Resource:Admin/SAL diffhist +149 Stashbot arturo: [codfw1dev] add DB grants for cloudbackup2002.codfw.wmnet IP address to the cinder DB (T292546)
     11:36  Backport windows/Deployers diffhist +14 Lucas Werkmeister (WMDE) →‎General advice: update window names (UTC morning/evening/late)
 m   08:44  Analytics/Systems/Jupyter diffhist −126 Btullis Replaced the section on Jupyter server logs
     06:41  SRE/LDAP/Groups diffhist +93 Aklapper Link to puppet code
     01:41  Server Admin Log diffhist +17,092 Stashbot ejegg: updated payments-wiki from b329d2dea2 to 19d18c1852
     00:02  CAS-SSO/Administration diffhist 0 Krinkle →‎Rebooting

13 October 2021

     23:31  Nova Resource:Tools.lexeme-forms/SAL diffhist +261 Stashbot wm-bot: <lucaswerkmeister> fully restarted webservice (stop/start) to avoid label issues
     23:29  Nova Resource:Tools.pagepile-visual-filter/SAL diffhist +223 Stashbot wm-bot: <lucaswerkmeister> deployed 66e06c58c5 (remove type ignore comments), updated dependencies including Flask 2.0.2, fully restarted webservice (stop/start) to avoid label issues
     23:28  Nova Resource:Tools.ranker/SAL diffhist +223 Stashbot wm-bot: <lucaswerkmeister> deployed 3fd9ab9f20 (remove type ignore comments), updated dependencies including Flask 2.0.2, fully restarted webservice (stop/start) to avoid label issues
     23:23  Nova Resource:Toolhub/SAL diffhist +134 Stashbot bd808: Updated Demo server to 563f977
     23:12  Release Engineering/SAL diffhist +359 Stashbot bd808: Killed a couple of deployment-prep update jobs in Jerkins that were hung on "waiting for executor" with 4 open executors.
     23:09  Deployments‎‎ 2 changes history +1,384 [Juan90264‎; Dylsss‎]
     
23:09 (cur | prev) +936 Juan90264 →‎{{Deployment_day|date=2021-10-13}}: +1 config Gerrit [730575]
     
00:25 (cur | prev) +448 Dylsss →‎{{Deployment_day|date=2021-10-14}}: {{gerrit|730243}}
     23:05  Portal:Toolforge/Admin diffhist +1,888 BryanDavis →‎Deleting a tool: Link to phab:T170355
N    21:26  File:Tool restore process.png diffhist +138 Andrew Bogott Diagram showing flow process when a disabled tool is restored
N    21:25  File:Tool disable process.png diffhist +129 Andrew Bogott Flow diagram of what happens when a tool is disabled
     20:10  PAWS/Admin diffhist −179 Bstorm →‎Database: update for trove
N    19:29  User:Triciaburmeister diffhist +287 Triciaburmeister Add page content
     18:22  Nova Resource:Tools.quickcategories/SAL diffhist +160 Stashbot wm-bot: <lucaswerkmeister> deployed 7a5d6823e2 (remove type ignore comments), updated dependencies including Flask 2.0.2
 m   15:52  Wikimedia Cloud Services team/EnhancementProposals/Toolforge Buildpack Implementation diffhist +1 BryanDavis →‎To make this useful with current Toolforge and webservice: spelling
     15:50  User:Jbond/debuging diffhist +216 Jbond →‎Check the pooled state
     14:35  Orchestrator diffhist +3,107 Kormat →‎Operations: split out packaging from operations
N    14:22  User:Kormat/Orchestrator diffhist +637 Kormat →‎Operations: does it work with only one section?
     13:52  MediaWiki talk:Titleblacklist diffhist +216 Nhatminh01 →‎Adding: new section
 m   13:34  Analytics/Systems/Kerberos/UserGuide diffhist +251 Btullis →‎Authenticate via Kerberos: Updated with details of the extended max lifetime of a Kerberos ticket.
     13:13  SRE/Data Persistence/Databases/Runbooks diffhist +191 LSobanski
     12:42  GitLab/Gitlab Runner diffhist +1,123 Jelto
     12:37  Portal:Toolforge/Admin/New grid engine exec host diffhist +102 David Caro →‎Host setup
     12:13  Analytics/Systems/Jupyter diffhist +622 Btullis →‎Viewing JupyterHub logs: Added details of the Kibana dashboard.
     12:09  Memcached for MediaWiki diffhist +27 Effie Mouzeli →‎Service
N    11:38  User:Notsniwiast diffhist +2 Notsniwiast creation
     11:18  Nova Resource:Toolsbeta/SAL diffhist +434 Stashbot wm-bot: Added a new grid webgrid generic node toolsbeta-sgewebgen-09-1.toolsbeta.eqiad1.wikimedia.cloud to the pool (T292465) - cookbook ran by dcaro@vulcanus
     10:46  Nova Resource:Admin/SAL diffhist +105 Stashbot arturo: updating python3-neutron across the fleet (T292936)
N    10:44  Maps/v2/Troubleshooting diffhist +124 Effie Mouzeli
     10:42  Maps diffhist +95 Effie Mouzeli
N    10:37  Db-kill diffhist +3,673 Jcrespo →‎Usage: where it is installed
 m   10:22  Alertmanager diffhist +531 Volans →‎How do I access the API?: Add more info
N    10:12  User talk:Notsniwiast diffhist +678 StrikerBot Welcome to Toolforge!
N    10:11  User talk:Yi7cuGhe diffhist +675 StrikerBot Welcome to Toolforge!
N    10:08  User talk:Yuanga diffhist +673 StrikerBot Welcome to Toolforge!
 m   10:05  MariaDB/troubleshooting diffhist +1,436 Jcrespo →‎Overload due to long running queries
     09:57  Nova Resource:Twl/SAL diffhist +96 Stashbot arturo: bump RAM quota from 16GB to 32GB (T292100)
     08:42  Kubernetes/Service ports diffhist −22 Elukey Delete inference entry (since we'll use the istio ingress gw ports)
 m   01:27  Analytics/Systems/Cluster diffhist +12 Quiddity →‎FAQ: lang="bash"
     01:16  Selling to Wikimedia Foundation/Pointer diffhist −3 Quiddity s/entertain/accept/
     00:38  Server Admin Log diffhist +15,911 Stashbot ejegg: updated payments-wiki from 030b11da1a to b329d2dea2

12 October 2021

     23:13  Wikimedia Cloud Services team/EnhancementProposals/Toolforge Buildpack Implementation diffhist +3,760 Bstorm →‎To make this useful with current Toolforge and webservice: add a link to the tekton dashboard setup
 m   21:58  Selling to Wikimedia Foundation/Pointer diffhist +70 Krinkle
     19:11  Release Engineering/SAL diffhist +197 Stashbot hashar: Building Docker images for https://gerrit.wikimedia.org/r/730279
     18:02  MediaWiki:Sitenotice diffhist +418 BryanDavis Announce meta:Coolest Tool Award 2021 nominations
 m   17:31  Server Lifecycle/Reimage diffhist +93 Volans →‎Reimage operations: Add step now covered by the cookbook
 m   17:25  Server Lifecycle diffhist −106 Volans →‎Automatic Installation: Removed step now handled by the reimage cookbook
     16:10  Nova Resource:Toolsbeta/SAL diffhist +442 Stashbot wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
 m   15:11  Anycast diffhist +935 Sukhbir Singh update notes to declare IPv6 support
     14:59  Analytics/Data Lake/Traffic/Webrequest diffhist +22 Neil P. Quinn-WMF Fix referrer value used for a blank referrer ("-", not "none"). Add link.
 m   14:49  Heterogeneous deployment/Train deploys diffhist −45 Ahmon Dancy Removed an obsolete note
     14:39  Kafka/Administration diffhist +3,511 Elukey →‎Alter topic partition number
 m   14:34  Analytics/Systems/Cluster/Spark diffhist +1 Neil P. Quinn-WMF →‎Executors: Fix typo
N    13:27  Monitoring/VarnishTrafficDrop diffhist +2,353 Ema →‎Things to do: link the DDoS playbook
     12:44  VRT System diffhist +101 Alexandros Kosiaris
N    11:58  File:Ulsfo-traffic-spike.png diffhist +27 Ema
N    11:54  File:Esams-traffic-drop.png diffhist +26 Ema
N    10:52  File:Typical Email workflow in VRT System.jpg diffhist +36 Alexandros Kosiaris
     09:06  Nova Resource:Admin/SAL diffhist +200 Stashbot dcaro: upgrading eqiad cloudnet hosts neutron packages (T292936)
     04:12  Volunteer NDA diffhist 0 Krinkle
     00:11  Server Admin Log diffhist +3,728 Stashbot eileen: civicrm revision changed from 598b59b0ee to 96090e4bd2, config revision is 85277466ed

11 October 2021

     23:29  Analytics/Systems/Cluster diffhist +122 Neil P. Quinn-WMF Add command for calculating size of HDFS directory
     19:26  Deployments diffhist +131 Cparle →‎{{Deployment_day|date=2021-10-12}}
     18:18  Incident documentation/2021-10-08 network provider diffhist +169 CDanis →‎Actionables
     15:35  Release Engineering/SAL diffhist +314 Stashbot hashar: gerrit: marked integration/composer read-only. We are no more using it and the few use cases left are tracked at https://phabricator.wikimedia.org/T249949
     15:24  Nova Resource:Toolsbeta/SAL diffhist +365 Stashbot wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
     14:32  Nova Resource:Tools.video2commons/SAL diffhist +64 Stashbot chicocvenancio: trying python3 port
     13:58  Nova Resource:Tools.video2commons-test/SAL diffhist +121 Stashbot chicocvenancio: checking out PR#138 and running with python3.9 type T292355
     12:20  User:Minato826 diffhist +199 Minato826 Removing redirect to deletion requested page
     12:16  User:Anoop Rao diffhist −997 Minato826 delete|author request-i think this was accidentally created by me
     12:14  Nova Resource:Tools.lexeme-forms/SAL diffhist +101 Stashbot wm-bot: <lucaswerkmeister> deployed fb32d04132 (l10n updates)
     11:45  Wikimedia Cloud Services team/EnhancementProposals/cinder backups diffhist −1 Arturo Borrero Gonzalez →‎Storage size: typo in bolding syntax
     11:42  Wikimedia Cloud Services team/EnhancementProposals/cloudswift diffhist +276 Arturo Borrero Gonzalez →‎Additional notes: mention codfw PoC
     11:07  UID diffhist +1,089 Jbond
     10:59  MariaDB/Provisioning a host diffhist −152 LSobanski →‎Replacing a failed host: Updating the reimage instructions
     10:19  Portal:Toolforge/Admin/Kubernetes/Networking and ingress diffhist +50 Majavah →‎nginx-ingress
 m   10:17  Server Lifecycle/Reimage diffhist +977 Volans →‎What to do if...: Add section for Icinga configuration correctness
 m   09:34  Kubernetes diffhist +145 JMeybohm Add some more file paths
 m   09:20  Dumps/Dumpsdata hosts diffhist +6 Volans →‎Reimaging an old host: Switch wmf-auto-reimage to the reimage cookbook
     09:20  MariaDB/Upgrading a section diffhist +141 Jcrespo →‎Upgrade procedure: adding automatic downtime cookbook
 m   09:19  Ganeti diffhist −48 Volans →‎Create a VM: Switch wmf-auto-reimage to the reimage cookbook
 m   09:18  Puppet diffhist +21 Volans →‎puppet agent: Switch wmf-auto-reimage to the reimage cookbook
 m   09:17  Cache servers diffhist −9 Volans →‎Add host to new cluster: Switch wmf-auto-reimage to the reimage cookbook
 m   09:16  SRE/Infrastructure Foundations/Ownership diffhist −964 Volans Switch wmf-auto-reimage to the reimage cookbook
     09:15  Management Interfaces diffhist +10 Jcrespo →‎Is there any overriding for next boot?: update deprecated reimage script
 m   09:14  SRE/SRE tooling diffhist −143 Volans Switch wmf-auto-reimage to the reimage cookbook
     08:19  Server Lifecycle diffhist −3,723 Volans Updated documentation for reimage for the new process
     07:32  Kafka/Administration diffhist +477 Elukey →‎Rebalance topic partitions to new brokers
 m   06:57  User:SDineshKumar/Learning&Dev diffhist +372 SDineshKumar midday publish
     02:54  User talk:Krinkle diffhist +199 Krinkle →‎Low priority Navpop JS update request: Reply
     02:54  MediaWiki:Gadget-popups.js diffhist +5,150 Krinkle Update from en.wikipedia.org, per @Quiddity, diff 1928512.

10 October 2021

     23:47  Category:Toolforge tools diffhist +107 Quiddity update box-summary a bit, merging Toolhub into the top list
     22:25  Server Lifecycle/Reimage diffhist +12 Quiddity →‎Example snippet: lang=text
     22:19  User talk:Krinkle diffhist +316 Quiddity →‎Low priority Navpop JS update request: new section
     21:45  MediaWiki:Loginprompt diffhist +7 Krinkle more signal to noise
     21:00  Deployments diffhist +23 Huji →‎{{Deployment_day|date=2021-10-12}}
 m   16:10  User:SDineshKumar/Learning&Dev diffhist +2,585 SDineshKumar minor - add pholio link
     11:20  Nova Resource:Tools.lexeme-forms/SAL diffhist +111 Stashbot wm-bot: <lucaswerkmeister> deployed bf2834c472 (improve error handling)

9 October 2021

 m   21:04  Deployments diffhist +987 Rafael →‎{{Deployment_day|date=2021-10-13}}: fix
     16:28  SRE/LDAP/Groups diffhist 0 Majavah →‎NDA group: fix link
Nm   07:57  User:SDineshKumar/Learning&Dev diffhist +4,411 SDineshKumar review edit - minor
 m   07:46  User:SDineshKumar diffhist +161 SDineshKumar self tracker
     05:01  Server Admin Log‎‎ 2 changes history +9,032 [Stashbot‎ (2×)]
     
05:01 (cur | prev) +224 Stashbot jiji@deploy1002: helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' .
     
01:32 (cur | prev) +8,808 Stashbot ryankemper@cumin1001: END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) restart without plugin upgrade (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic restart - ryankemper@cumin1001 - T292814
N    02:00  Deployments/Archive/2021/10 diffhist +12,477 DeploymentCalendarTool Add last week
     00:18  Nova Resource:Wikisp/SAL diffhist +88 Stashbot DeusExMachina: Deleted mars-01 per T292884

8 October 2021

     23:00  Nova Resource:Tools.gerrit-reviewer-bot/SAL diffhist +153 Stashbot wm-bot: <root> root forced the deletion of job 913897
     21:17  Release Engineering/SAL diffhist +494 Stashbot hashar: Purging Docker images on all CI agents
     20:54  Template:Gitweb/doc diffhist −10 BryanDavis fix example example
     20:53  Toolhub.wikimedia.org diffhist +12 BryanDavis →‎Testing deployment: fix gerrit link
     20:52  Template:Gitweb diffhist +16 BryanDavis Use /r/plugins/gitiles/ rather than /g/
 m   19:23  Geolocation diffhist +4 Pcoombe link to Varnish
     17:58  Network monitoring diffhist +1,208 CDanis →‎NEL alerts
     17:30  Network Error Logging diffhist +1,179 CDanis →‎intro
     16:57  HTTPS diffhist +1,225 BBlack Add section about Certificate issuance practices
     16:05  Analytics/Data Lake/Traffic/BotDetection diffhist +14 Nettrom →‎Code: update to reference the shaded JARs
     15:10  Bacula diffhist +585 Jcrespo →‎Restore from a non-existent host (missing private key)
N    14:09  VRT System diffhist +18,781 Alexandros Kosiaris Bring up to date various places in the doc
     14:05  Performance/Metrics diffhist +136 Krinkle
     12:47  Mail diffhist +1 Alexandros Kosiaris VRT System
     12:36  WMDE/Wikidata/Alerts diffhist +1,340 Michael Große →‎Change Dispatching
     12:31  Znuny diffhist −2 Alexandros Kosiaris Changed redirect target from VRT Platform to VRT System
     12:31  OTRS diffhist −2 Alexandros Kosiaris Changed redirect target from VRT Platform to VRT System
     08:59  Event Platform/EventGate diffhist +52 Hashar →‎External links
     08:54  Apache Traffic Server diffhist +473 Ema →‎Terminology
     08:49  Wikimedia Cloud Services team/EnhancementProposals/GridEngine plans and timeline diffhist +89 David Caro →‎Reasoning
     08:01  Analytics/Systems/Cluster/AMD GPU diffhist +1,599 Elukey →‎Experimental
     07:52  IP and AS allocations diffhist +80 Ayounsi Add anycast community
N    06:47  Incident documentation/2021-10-08 network provider diffhist +2,244 RhinosF1 →‎Summary: change said -> a. Said doesn't make sense as it hasn't been said above.
N    06:40  Incident documentation/2021-10-08 eqiad network provider diffhist +64 Ayounsi Ayounsi moved page Incident documentation/2021-10-08 eqiad network provider to Incident documentation/2021-10-08 network provider
N    06:24  File:Screenshot 2021-10-08 at 08-23-35 NEL (Network Error Logging) - Elastic.png diffhist +45 Ayounsi
 m   02:06  Backport windows/Deployers diffhist −46 Krinkle →‎Purging: ain't nobody got time to shift that quote
     00:07  Server Admin Log diffhist +11,887 Stashbot tgr_: deploy window over

7 October 2021

     23:11  Deployments diffhist +9,527 Gergő Tisza →‎Thursday, October 7
     22:12  Release Engineering/SAL‎‎ 2 changes history +831 [Stashbot‎ (2×)]
     
22:12 (cur | prev) +704 Stashbot James_F: Zuul: [design/codex] Provide initial CI, node14-browser suite
     
01:06 (cur | prev) +127 Stashbot Reedy: unstick beta ci jobs
     20:11  SRE/Mailing Lists diffhist −24 Krinkle
     20:11  Volunteer NDA diffhist −13 Krinkle →‎Get support
     20:00  Canary diffhist +33 Quiddity standard disambig intro
     17:58  Nova Resource:Cvn/SAL diffhist +71 Stashbot Krinkle: killall cvn.cvnservice @ cvn-app8
 m   15:04  Gerrit/Upgrade diffhist +270 Hashar ssh command to list plugins
     14:21  Nova Resource:Toolsbeta/SAL diffhist +940 Stashbot wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
     14:12  Apache Traffic Server diffhist +168 Ema →‎Cheatsheet
N    13:10  Wikimedia Cloud Services team/EnhancementProposals/cinder backups diffhist +4,640 David Caro →‎Option 2: use backy2
 m   12:16  Server Lifecycle/Reimage diffhist +654 Volans Add warning
 m   12:01  Spicerack/Cookbooks diffhist +166 Volans Add link to logs
     11:04  Kubernetes diffhist +976 Giuseppe Lavagetto →‎Add a new service: Amend the section to reflect the last refactorings; try to make the text clearer with code examples rather than example commits.
     09:25  Portal:Toolforge/Admin/Kubernetes/Deploying diffhist −1,116 Majavah →‎ingress setup
     09:13  Nova Resource:Tools/SAL diffhist +247 Stashbot majavah: disabling settings api, now that all pod presets are gone T279106
     07:35  Swift/How To diffhist +774 Filippo Giunchedi →‎Replacing a disk without touching the rings
     07:35  Nova Resource:Project-proxy/SAL diffhist +93 Stashbot arturo: manually rebase labs/private.git, it had merge conflicts
     07:22  Alertmanager diffhist +834 Filippo Giunchedi →‎Create alerts
N    02:29  Module:Documentation/doc diffhist +1,851 Nhatminh01 Created page with "{{Module rating|release}} <!-- Add categories where indicated at the bottom of this page and interwikis at Wikidata --> {{Uses TemplateStyles|Module:Documentation/styles.css}} {{Lua|Module:Documentation/config|Module:Documentation/i18n|Module:Arguments|Module:Message box|Module:TNT}} This module displays a blue box containing documentation for templates, Lua modules, or other pages. The {{tlx|docu..."
     02:28  Template:Documentation diffhist −1,081 Nhatminh01 Replaced content with "{{#invoke:documentation|main|_content={{ {{#invoke:documentation|contentTitle}}}}}}"
N    02:28  Template:Documentation/doc diffhist +1,052 Nhatminh01
     00:16  Nova Resource:Tools.bridgebot/SAL diffhist +660 Stashbot wm-bot: <bd808> Bot is now connected to libera.chat via a ZNC bouncer (T292640)
     00:11  Server Admin Log diffhist +11,864 Stashbot mutante: [grafana2001:~] $ sudo systemctl start rsync-var-lib-grafana because of "PROBLEM - Check systemd state on grafana2001 is CRITICAL: CRITICAL - degraded" because of some race condition where a file vanished during sync

6 October 2021

     23:21  Deployments diffhist −13,254 Jforrester →‎Week of September 27: Archive manually (bot broke?)
Nm   22:33  Server Lifecycle/Reimage diffhist +8,543 Volans Add netbox data import from puppetdb step
N    20:41  User talk:Conny Kawohl diffhist +678 StrikerBot Welcome to Toolforge!
N    20:41  User talk:Qwerfjkl diffhist +674 StrikerBot Welcome to Toolforge!
N    20:41  User talk:CristinaGava94 diffhist +680 StrikerBot Welcome to Toolforge!
N    20:40  User talk:CrissGava diffhist +675 StrikerBot Welcome to Toolforge!
N    20:40  User talk:Jlillebo1 diffhist +675 StrikerBot Welcome to Toolforge!
     20:24  Nova Resource:Tools.notwikilambda/SAL diffhist +181 Stashbot wm-bot: <lucaswerkmeister> mount only function-orchestrator source into pod (8731f2ff6e), working around apparent “toolforge: tool” issue
     19:52  Nova Resource:Tools.ranker/SAL diffhist +100 Stashbot wm-bot: <lucaswerkmeister> deployed 3540e2a083 (🌈 navbar)
 m   18:41  Cumin diffhist +356 Volans Add the direct backend
     18:19  Template:Infrastructure Foundations/Navigation diffhist +52 Quiddity add missing link to match the image's link (it goes to an empty page though, ping User:Jobo FYI) (and the image-links are temporarily broken due to phab:T292657)
N    17:51  Machine Learning diffhist +125 Calbon
     16:07  Nova Resource:Toolhub/SAL diffhist +77 Stashbot bd808: Updated demo server to 91e33ad
     16:03  Help:IRC diffhist +40 BryanDavis Add Help:Cloud Services communication footer
     14:25  Analytics/Systems/Cluster/AMD GPU diffhist +952 Elukey →‎Use tensorflow
     10:36  Nova Resource:Toolsbeta/SAL diffhist +595 Stashbot wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
     09:47  Nova Resource:Cloudinfra/SAL diffhist +99 Stashbot dcaro: upgraded cloudmetrics to grafana 7.5 (T292614)
     06:46  Nova Resource:Tools/SAL diffhist +154 Stashbot majavah: taavi@toolserver-proxy-01:~$ sudo systemctl restart apache2.service # see if it helps with toolserver.org ssl alerts
     03:17  Deployments/Train diffhist +65 Vahurzpu →‎Wednesday: Note a new big group 1 wiki since phab:T286664
 m   01:59  Ganeti diffhist −30 Quiddity lang=bash
     01:39  Server Admin Log diffhist +10,381 Stashbot legoktm: legoktm@mwmaint1002:~$ echo "https://en.wikiversity.org/static/images/mobile/copyright/wikiversity.svg" |mwscript purgeList.php

5 October 2021

     21:47  Release Engineering/SAL diffhist +361 Stashbot brennen: updating dev-images docker-pkg files on primary contint for https://gitlab.wikimedia.org/releng/dev-images/-/merge_requests/7
     20:54  Heterogeneous deployment/Train deploys diffhist +48 Brennen Bearnes →‎Rollback: Formatting tweak for commit amending section
 m   20:36  SRE/Clinic Duty/Access requests diffhist 0 Neil P. Quinn-WMF Copyedit section headings
Nm   19:15  Out-of-band network diffhist +2,381 Legoktm Legoktm moved page OOB to Out-of-band network: better page title
N    19:15  OOB diffhist +33 Legoktm Legoktm moved page OOB to Out-of-band network: better page title
     15:40  Deployments diffhist +714 Dave Pifke →‎{{Deployment_day|date=2021-10-05}}
     15:33  Nova Resource:Toolhub/SAL diffhist +77 Stashbot bd808: Updated demo server to 2950dc6
N    15:23  Wikimedia Cloud Services team/EnhancementProposals/cloudswift diffhist +4,386 BryanDavis →‎How it works: link to https://docs.ceph.com/en/latest/radosgw/
N    15:13  Out-of-band management diffhist +17 BryanDavis SEO redir
     15:07  Ganeti diffhist +347 Jbond →‎Reinstall / Reimage a VM
     14:53  Portal:Cloud VPS/Infrastructure diffhist +41 Arturo Borrero Gonzalez →‎DNS: add designate link
     14:52  Portal:Cloud VPS/Admin/DNS diffhist −136 Arturo Borrero Gonzalez relocate designate diagram to https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/DNS/Designate
     14:52  Portal:Cloud VPS/Admin/DNS/Designate diffhist +58 Arturo Borrero Gonzalez add DNS & designate diagram
     14:46  Wikimedia Cloud Services team/EnhancementProposals/Toolforge Buildpack Implementation diffhist +11 David Caro →‎Namespacing
     14:41  Analytics/Systems/Cluster/AMD GPU diffhist −426 Elukey →‎Upgrade the Debian packages
     14:05  GitLab/Gitlab Runner diffhist +398 Jelto
     13:26  Alertmanager diffhist +34 MVernon →‎Create alerts: note CamelCase naming requirement.
     12:03  Nova Resource:Deployment-prep/SAL diffhist +119 Stashbot majavah: root@deployment-cache-text06:~# systemctl restart sssd # T286502
N    10:46  File:Cloudswitf-swift API exposed to cloud realm.drawio.png diffhist +63 Arturo Borrero Gonzalez
     09:39  Nova Resource:Admin/SAL diffhist +152 Stashbot arturo: [codfw1dev] cleaning up manila stuff from openstack (db, endpoints, tenant, VMs, and such) T291257
     09:26  Puppet/Pontoon diffhist +1,034 Filippo Giunchedi →‎Howto: add rebase section
 m   06:46  Incident documentation/2021-09-29 eqiad-kubernetes diffhist +5 JMeybohm →‎Summary
     00:48  Nova Resource:Wikitextexp/SAL diffhist +193 Stashbot bd808: Remove self (BryanDavis) from project after helping resolve T292264

4 October 2021

     23:30  Server Admin Log diffhist +9,952 Stashbot foks: resetting some emails used for abuse by a globally-banned user
     23:04  Nova Resource:Toolhub/SAL diffhist +79 Stashbot bd808: Updated demo instance to 675480a
     22:40  Deployments/Yearly calendar diffhist +9 Jforrester No worries about deploys in the first week of September, it was just the train that was cancelled.
     22:16  Deployments diffhist +1,016 Jdrewniak →‎{{Deployment_day|date=2021-10-04}}
     21:22  GitLab diffhist −123 Brennen Bearnes Update current state: Only one instance in gitlab-test, and production instance is open to all users.
     21:22  Nova Resource:Wikistats/SAL diffhist +106 Stashbot mutante: /usr/local/bin/wikistats/import_miraheze.sh T292369
     21:19  GitLab/Upgrade diffhist +467 Brennen Bearnes Stub gitlab-runner instructions.
     21:09  Release Engineering/SAL diffhist +536 Stashbot brennen: finished upgrading shared runners (1002 - 1005) to gitlab-runner 14.3.2 (T292256)
N    20:02  Talk:Wikimedia Cloud Services team/EnhancementProposals/GridEngine plans and timeline diffhist +1,075 BryanDavis →‎Possibility of new grid engine project for tools that cannot migrate: new section
     19:42  Nova Resource:Tools.lexeme-forms/SAL diffhist +101 Stashbot wm-bot: <lucaswerkmeister> deployed 1697521bf5 (l10n updates)
     18:13  Parsoid diffhist +3 Sbailey
 m   18:01  Provenance diffhist +7 Jforrester Fix link to T261949 to actually point to T261949 and not the front page.
     17:25  Nova Resource:Tools.admin/SAL diffhist +84 Stashbot wm-bot: <bd808> Restarted to pick up 942ec3e
     17:14  Operating system upgrade policy diffhist −251 Neil P. Quinn-WMF Combine problem statement and background section. Further changes now that this is a policy.
     17:07  Nova Resource:Toolsbeta/SAL diffhist +427 Stashbot bstorm: reboot everything T291406
     16:35  Nova Resource:Cloudstore/SAL diffhist +130 Stashbot bstorm: deleting cloudstore-nfs-01 as that was the old instance for "nfs-01" T291406
     15:33  Nova Resource:Tools.wmopbot/SAL diffhist +131 Stashbot wm-bot: <root> Ran `jstart -j y -mem 1g -once wmopbot.sh` from .bash_history in attempt to restart bot
     14:57  Redis diffhist +28 Phuedx →‎Cluster redis_sessions
 m   14:49  Wikidata Query Service/Runbook diffhist +1,082 DCausse →‎Free allocators decrease rapidly
 m   13:46  SRE/Clinic Duty diffhist 0 Kormat →‎Maintain the 'ops-