Page MenuHomePhabricator
Create Task
Maniphest T256822

[EPIC] Upgrade MediaWiki-Vagrant to Debian Buster
Closed, ResolvedPublic
Actions

Description

The Vagrant base image is currently Stretch (Debian 9), which is EOL in 2020-07-06 (although the Debian LTS project supports it until 2022 summer). Wikimedia production and CI (which Vagrant is meant to be compatible with) are moving to Buster (Debian 10; T247045: Migrate all of production metal and VMs to Buster or later, T252432: Drop MediaWiki testing in stretch and instead test only in buster), MediaWiki-Vagrant should too.

See also:

It's preferable to just recreate your Vagrant boxes from scratch (cattle not pets) but if you really need to upgrade an existing Stretch-based box, here are the steps:

  • dump DB with mysqldump --add-drop-database --databases `mysql -rNe 'show databases' | ack -v '^(mysql|information_schema|performance_schema)$'` > /vagrant/dump.sql
  • update Vagrant repo
  • run vagrant destroy -f; vagrant up
  • restore dump with mysql < /vagrant/dump.sql
  • clear caches, e-g- by running vagrant reload
  • regenerate all data wasn't stored in the DB:
    • if the cirrussearch role is enabled, run mwscript /vagrant/mediawiki/extensions/CirrusSearch/maintenance/ForceSearchIndex.php

Previous: T181353: [EPIC] Migrate MediaWiki-Vagrant base image to Debian Stretch
Next: T319167: [EPIC] Upgrade MediaWiki-Vagrant to Debian Bullseye

Details

SubjectRepoBranchLines +/-
Buster migration: Link to upgrading help pagemediawiki/vagrantmaster+2 -0
Update various PHP extensions to 7.4mediawiki/vagrantmaster+7 -18
Add --allow-releaseinfo-change to apt updatemediawiki/vagrantmaster+1 -1
Fix wikidiff2 package namemediawiki/vagrantmaster+2 -2
Switch to PHP 7.4mediawiki/vagrantmaster+50 -49
buster: Update uwsgi-plugin-rack-ruby versionmediawiki/vagrantmaster+1 -1
Merge remote-tracking branch 'origin/master' into buster-migrationmediawiki/vagrantbuster-migration+910 -637
Switch to PHP 7.4mediawiki/vagrantmaster+50 -49
buster: Update uwsgi-plugin-rack-ruby versionmediawiki/vagrantbuster-migration+1 -1
Customize query in gerrit

Related Objects
Search...

Event Timeline

Tgr created this task.Jun 30 2020, 10:11 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 30 2020, 10:11 PM
Tgr mentioned this in T164126: Rotate MediaWiki-Vagrant log files to prevent logs from eating up all disk space.Jun 30 2020, 10:14 PM
jgleeson subscribed.Jun 30 2020, 10:45 PM
Ejegg subscribed.Jul 1 2020, 5:43 PM
bd808 subscribed.Jul 14 2020, 11:42 PM

I led the Precise -> Trusty -> Jessie -> Stretch migrations for MediaWiki-Vagrant. I would be happy to see someone else take that lead role in a Stretch -> Buster migration. See T181353: [EPIC] Migrate MediaWiki-Vagrant base image to Debian Stretch for the last tracking task of this type.

bd808 renamed this task from Upgrade MediaWiki-Vagrant to Debian Buster to [EPIC] Upgrade MediaWiki-Vagrant to Debian Buster.Jul 14 2020, 11:43 PM
bd808 triaged this task as Medium priority.
bd808 added a project: Epic.
bd808 updated the task description. (Show Details)
Tgr added a project: User-Tgr.Jul 15 2020, 11:05 AM

I'm interested in theory, would have to dig myself out from below a mountain of other tasks first, though.

bd808 updated the task description. (Show Details)Jul 15 2020, 11:45 PM
jgleeson added a comment.EditedJul 16 2020, 3:31 PM

I'm also interested in helping out with this. Over in fundraising-tech land, we have a few mw-vagrant users and with our production stack recently moving to buster I'm keen to see what it would take to move the base vagrant to buster.

bd808 updated the task description. (Show Details)Oct 10 2020, 2:48 PM
Reedy updated the task description. (Show Details)Oct 26 2020, 1:45 AM
Tgr mentioned this in T266490: Add Link engineering: Vagrant role.Oct 29 2020, 3:38 AM
Tgr mentioned this in T266737: Set up Python 3.7 on MediaWiki-Vagrant.Oct 29 2020, 4:13 AM
Reedy mentioned this in T268273: Missing content_model rows after importing templates and modules.Nov 20 2020, 5:01 AM
Reedy mentioned this in T268338: Cannot run supported MW release branches on MediaWiki-Vagrant.Nov 20 2020, 2:01 PM
Reedy added a parent task: T268338: Cannot run supported MW release branches on MediaWiki-Vagrant.
bd808 added a comment.Jan 4 2021, 9:51 PM

T245757: Upgrade MediaWiki clusters to Debian Buster (debian 10) seems to be at the stage of upgrading prod MW servers to Buster, so in theory this task is ready to move forward.

bd808 moved this task from Backlog to In Progress on the MediaWiki-Vagrant board.Jan 10 2021, 6:12 PM
$ git push --set-upstream origin buster-migration
Total 0 (delta 0), reused 0 (delta 0)
remote: Processing changes: refs: 1, done
To ssh://gerrit/mediawiki/vagrant
 * [new branch]        buster-migration -> buster-migration
Branch 'buster-migration' set up to track remote branch 'buster-migration' from 'origin'.
gerritbot added a comment.Jan 10 2021, 11:47 PM

Change 655298 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[mediawiki/vagrant@buster-migration] buster: Update uwsgi-plugin-rack-ruby version

https://gerrit.wikimedia.org/r/655298

gerritbot added a project: Patch-For-Review.Jan 10 2021, 11:47 PM
gerritbot added a comment.Jan 11 2021, 1:10 AM

Change 655298 merged by jenkins-bot:
[mediawiki/vagrant@buster-migration] buster: Update uwsgi-plugin-rack-ruby version

https://gerrit.wikimedia.org/r/655298

Maintenance_bot removed a project: Patch-For-Review.Jan 11 2021, 1:10 AM
Reedy closed subtask T271664: "E: Package 'python-imaging' has no installation candidate" on Buster as Resolved.Apr 9 2021, 2:28 AM
Reedy mentioned this in T234240: Phan should work out of the box on MediaWiki-Vagrant.Apr 15 2021, 1:38 AM
Reedy added a parent task: T234240: Phan should work out of the box on MediaWiki-Vagrant.
Tgr mentioned this in T292324: mediawiki-vagrant fails provisioning (because of missing root cert).Oct 2 2021, 9:19 AM
Tgr mentioned this in T298326: apt.wikimedia.org packages php and php7.2 point to different versions of PHP 7.2.Dec 26 2021, 6:53 PM
Stang subscribed.Jun 19 2022, 1:34 PM
Mainframe98 subscribed.Sep 30 2022, 9:12 AM

Should the migration to PHP 7.4 be part of buster-migration or part of stretch? With the changes to MediaWiki, master now requires PHP 7.4 (T261872). Buster provides 7.3 and would be outdated already.

Mainframe98 added a comment.Sep 30 2022, 3:15 PM
In T256822#8274862, @Mainframe98 wrote:

Should the migration to PHP 7.4 be part of buster-migration or part of stretch? With the changes to MediaWiki, master now requires PHP 7.4 (T261872). Buster provides 7.3 and would be outdated already.

The reason this is important is because there are no stretch packages for PHP 7.4. This means that at this point, stretch is broken with MediaWiki master (REL1_ branches already were broken) and buster is too until it is upgraded to PHP 7.4. Changing stretch to use a custom repository (say Ondrej's) is a wasted effort when everyone should move to a buster-based box soon. (and bullseye later...)

I can't find PHP 7.4 in stretch-backports and Ondrej's PPA does not provide for stretch anymore, so that leaves us with manually compiling the packages. For an outdated distro, that's not worth the effort.

Mainframe98 mentioned this in T319061: Remove libphutil clone as part of arcanist initialization.Sep 30 2022, 6:01 PM
gerritbot added a comment.Sep 30 2022, 6:18 PM

Change 837165 had a related patch set uploaded (by Mainframe98; author: Mainframe98):

[mediawiki/vagrant@buster-migration] Merge remote-tracking branch 'origin/master' into buster-migration

https://gerrit.wikimedia.org/r/837165

gerritbot added a project: Patch-For-Review.Sep 30 2022, 6:18 PM
gerritbot added a comment.Sep 30 2022, 6:32 PM

Change 837166 had a related patch set uploaded (by Mainframe98; author: Mainframe98):

[mediawiki/vagrant@master] Switch to PHP 7.4

https://gerrit.wikimedia.org/r/837166

gerritbot added a comment.Oct 2 2022, 7:12 PM

Change 837166 abandoned by Mainframe98:

[mediawiki/vagrant@master] Switch to PHP 7.4

Reason:

This patch is targeting the wrong branch and needs to be redone after buster-migration has been rebased on top of master

https://gerrit.wikimedia.org/r/837166

Mainframe98 mentioned this in T319167: [EPIC] Upgrade MediaWiki-Vagrant to Debian Bullseye.Oct 3 2022, 8:42 AM
Mainframe98 added a parent task: T319167: [EPIC] Upgrade MediaWiki-Vagrant to Debian Bullseye.
gerritbot added a comment.Oct 5 2022, 3:24 PM

Change 837165 abandoned by Mainframe98:

[mediawiki/vagrant@buster-migration] Merge remote-tracking branch 'origin/master' into buster-migration

Reason:

Rebasing buster-migration on master and then merging the branch locally is less messy

https://gerrit.wikimedia.org/r/837165

Maintenance_bot removed a project: Patch-For-Review.Oct 5 2022, 3:30 PM
gerritbot added a comment.Oct 16 2022, 3:21 PM

Change 842963 had a related patch set uploaded (by Mainframe98; author: Bryan Davis):

[mediawiki/vagrant@master] buster: Update uwsgi-plugin-rack-ruby version

https://gerrit.wikimedia.org/r/842963

gerritbot added a project: Patch-For-Review.Oct 16 2022, 3:21 PM

Change 842964 had a related patch set uploaded (by Mainframe98; author: Mainframe98):

[mediawiki/vagrant@master] Switch to PHP 7.4

https://gerrit.wikimedia.org/r/842964

Tgr updated the task description. (Show Details)Oct 25 2022, 12:54 AM
kostajh subscribed.Oct 26 2022, 12:31 PM

Just noting that T319432: Migrate WMF production from PHP 7.4 to PHP 8.1 is in progress, not sure if that affects how you go about updating Vagrant for PHP 7.4, but wanted to mention it in case it has some bearing in the approach you take.

gerritbot added a comment.Oct 26 2022, 5:42 PM

Change 842963 merged by jenkins-bot:

[mediawiki/vagrant@master] buster: Update uwsgi-plugin-rack-ruby version

https://gerrit.wikimedia.org/r/842963

Tgr added a comment.Oct 26 2022, 5:46 PM

Ideally we'd match the production setup (which also means doing T319167: [EPIC] Upgrade MediaWiki-Vagrant to Debian Bullseye, probably). I'd expect the production migration to take a while, though.

Tgr closed subtask T271649: Make single wiki + Virtualbox provider work with a Buster base image as Resolved.Oct 26 2022, 6:20 PM
gerritbot added a comment.Oct 27 2022, 3:41 PM

Change 842964 merged by jenkins-bot:

[mediawiki/vagrant@master] Switch to PHP 7.4

https://gerrit.wikimedia.org/r/842964

Maintenance_bot removed a project: Patch-For-Review.Oct 27 2022, 4:30 PM
Tgr closed subtask T319430: Address deprecations and warnings in buster initial box provision as Resolved.Oct 29 2022, 12:14 AM
gerritbot added a comment.Oct 29 2022, 1:56 AM

Change 850614 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/vagrant@master] Fix wikidiff2 package name

https://gerrit.wikimedia.org/r/850614

gerritbot added a project: Patch-For-Review.Oct 29 2022, 1:56 AM
gerritbot added a comment.Oct 29 2022, 3:11 AM

Change 850614 merged by jenkins-bot:

[mediawiki/vagrant@master] Fix wikidiff2 package name

https://gerrit.wikimedia.org/r/850614

Tgr mentioned this in T321968: Rebuild Wikispore Vagrant box on Buster.Oct 29 2022, 4:19 AM
Tgr added a subtask: T321984: Make MediaWiki-Vagrant LXC provider work with a Buster base image.Oct 29 2022, 7:02 PM
gerritbot added a comment.Oct 29 2022, 7:48 PM

Change 850639 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/vagrant@master] Add --allow-releaseinfo-change to apt update

https://gerrit.wikimedia.org/r/850639

gerritbot added a comment.Oct 29 2022, 7:51 PM

Change 850639 merged by jenkins-bot:

[mediawiki/vagrant@master] Add --allow-releaseinfo-change to apt update

https://gerrit.wikimedia.org/r/850639

gerritbot added a comment.Nov 1 2022, 2:38 AM

Change 851177 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/vagrant@master] Update various PHP extensions to 7.4

https://gerrit.wikimedia.org/r/851177

gerritbot added a comment.Nov 1 2022, 5:23 AM

Change 851177 merged by jenkins-bot:

[mediawiki/vagrant@master] Update various PHP extensions to 7.4

https://gerrit.wikimedia.org/r/851177

Tgr added a comment.Nov 1 2022, 5:59 AM

Steps to upgrade an existing box:

  • dump DB with mysqldump --add-drop-database --databases `mysql -rNe 'show databases' | ack -v '^(mysql|information_schema|performance_schema)$'` > /vagrant/dump.sql
  • update Vagrant repo
  • run vagrant destroy -f; vagrant up
  • restore dump with mysql < /vagrant/dump.sql
  • clear caches, e-g- by running vagrant reload to
  • regenerate all data wasn't stored in the DB:
    • if the cirrussearch role is enabled, run mwscript /vagrant/mediawiki/extensions/CirrusSearch/maintenance/ForceSearchIndex.php
Tgr closed subtask T321984: Make MediaWiki-Vagrant LXC provider work with a Buster base image as Resolved.Nov 4 2022, 10:44 PM
Tgr updated the task description. (Show Details)Nov 8 2022, 6:21 AM
gerritbot added a comment.Jun 15 2023, 12:11 PM

Change 930609 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/vagrant@master] Buster migration: Link to upgrading help page

https://gerrit.wikimedia.org/r/930609

Tgr added a comment.Jun 15 2023, 12:33 PM

List of errors from a recent upgrade:

  1. ATP cache write error:
==> default: W: chown to _apt:root of directory /vagrant/cache/apt/partial failed - SetupAPTPartialDirectory (1: Operation not permitted)
==> default: W: chown to root:root of file /vagrant/cache/apt/partial/logstash_1%3a7.1.1-1_all.deb failed - 201::URIDone (1: Operation not permitted)

(T183150: /vagrant/cache/apt permissions warnings with Stretch sandboxed apt settings - long-standing issue, I don't think it causes any problems, just distracting)

  1. libruby2.5 downgrade
==> default: The following packages will be DOWNGRADED:
==> default:   libruby2.5
==> default: 1 upgraded, 4 newly installed, 1 downgraded, 0 to remove and 102 not upgraded.
==> default: E: Packages were downgraded and -y was used without --allow-downgrades.
==> default: Error: /Stage[main]/Packages::Ruby_dev/Package[ruby-dev]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install ruby-dev' returned 100: Reading package lists...
  1. Logstash OOM
==> default: Setting up logstash (1:7.1.1-1) ...
==> default: Using provided startup.options file: /etc/logstash/startup.options
==> default: OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000d4cc0000, 724828160, 0) failed; error='Cannot allocate memory' (errno=12)
==> default: /usr/share/logstash/bin/system-install: line 88: #: command not found
==> default: Unable to install system startup script for Logstash.
==> default: chmod: cannot access '/etc/default/logstash': No such file or directory
==> default: dpkg: error processing package logstash (--configure):
==> default:  installed logstash package post-installation script subprocess returned error exit status 1

This then somehow breaks all other apt package installs to, they all have the same

==> default: Using provided startup.options file: /etc/logstash/startup.options
==> default: OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000d4cc0000, 724828160, 0) failed; error='Cannot allocate memory' (errno=12)

lines + dpkg error.

gerritbot added a comment.Jun 15 2023, 12:37 PM

Change 930609 merged by jenkins-bot:

[mediawiki/vagrant@master] Buster migration: Link to upgrading help page

https://gerrit.wikimedia.org/r/930609

Maintenance_bot removed a project: Patch-For-Review.Jun 15 2023, 1:10 PM
hashar closed this task as Resolved.Apr 15 2024, 10:31 AM
hashar subscribed.

Looks like the main one to swtich was 2d46b067418cdeebdad384d612338598d386fb7b T271649

All sub tasks have been marked resolved and there is an upgrade it written in November 2022 above T256822#8358845

So that looks like it is resolved.

Tgr removed a parent task: T319167: [EPIC] Upgrade MediaWiki-Vagrant to Debian Bullseye.May 26 2024, 9:43 AM
Tgr added a subtask: T322450: Build MediaWiki-Vagrant LXC Buster base image.May 26 2024, 9:47 AM
Tgr updated the task description. (Show Details)May 26 2024, 9:49 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy