Page MenuHomePhabricator

Transition to MathML rendering as default
Open, Needs TriagePublic

Description

I just reinstalled my firefox and was shocked about the formula display without the native MathML plugin:

2021-01-01 (2).png (335×1 px, 105 KB)

Not only that the images stick out, but also the German Umlauts look horrible. I think it is a pity that only a few people seem to use the native MathML plugin of @fredw.

@fredw do you have an estimation of when MathML in Chrome will be ready? Given the demo I saw, I think it would be a good time now, to update descriptions on how people can try MathML rendering even today and have some smaller group testing before we eventually change defaults and deliver fallback images really as a fallback.

Plan

Plan as of Aug 2024, from a meeting between @Physikerwelt, @MSantos, @ssastry, @Krinkle, and @Jdlrobson.

Background: The impetus is T338429: Prepare Mathoid for RESTbase sunsetting

Phase 1:

Phase 1 - Follows-up:

Phase 2A: Production

  • Communication to wikitech-l about plan.
  • Enable native mathml by default on group0 and test wikis in production (August 2024).
  • Final pre-release QA round by Web Team
  • Communication to tech news inviting people to try the user preference and/or to sign up as pilot wiki.
  • Enable native mathml by default, gradually on all wikis in production (Oct-Nov-Dec 2024).
  • Communication to tech news when it's the default everywhere, and upcoming removal of mathoid option, highlighting the MathJax user preference for those who prefer the old look.
  • Remove the mathoid option from production (when?).

Phase 2B: Upcoming MediaWiki 1.43 Release

  • Enable native mathml by default in Math extension for MediaWiki 1.43 release, and disable mathoid support by default so that new/upgraded installs use the native mathml exclusively, unless a sysadmin specifically adds it back in their $wgMathValidMode configuration (deadline: Nov 2024)

Phase 2C: MediaWiki 1.39 LTS

  • Communication to mediawiki-announce-l/mediawiki-l about the plan to backport native mathml to 1.39 LTS and to sunset Mathoid in 2025.
  • Backport native mathml to 1.39 (when: a few weeks after first pilot wikis have it in production, to make sure we avoid most bugs in the stable release).
  • Set default configuration for Math extension to native mathml
  • Remove mathoid support code from 1.39 (thus the only options in 1.39 are native mathml, latex, and maybe mathjax)
  • Backup: If adoption of the next 1.39 patch ends up too slow, the backup plan is to evaluate T369809: Implement generic MW-API endpoints to replace the math endpoints of restbase

Details

Other Assignee
Krinkle

Related Objects

StatusSubtypeAssignedTask
OpenPhysikerwelt
StalledNone
Resolvedtaavi
ResolvedPhysikerwelt
ResolvedPhysikerwelt
OpenNone
ResolvedBUG REPORTPhysikerwelt
OpenBUG REPORTPhysikerwelt
ResolvedBUG REPORTPhysikerwelt
ResolvedBUG REPORTPhysikerwelt
DeclinedBUG REPORTNone
ResolvedBUG REPORTPhysikerwelt
ResolvedBUG REPORTPhysikerwelt
ResolvedBUG REPORTPhysikerwelt
ResolvedBUG REPORTPhysikerwelt
OpenBUG REPORTPhysikerwelt
ResolvedBUG REPORTPhysikerwelt
OpenBUG REPORTPhysikerwelt
ResolvedBUG REPORTPhysikerwelt
ResolvedBUG REPORTPhysikerwelt
ResolvedBUG REPORTPhysikerwelt
ResolvedPhysikerwelt
OpenNone
ResolvedBUG REPORTPhysikerwelt
OpenBUG REPORTNone
OpenBUG REPORTNone
OpenBUG REPORTPhysikerwelt
OpenBUG REPORTNone
DuplicateBUG REPORTPhysikerwelt
ResolvedBUG REPORTPhysikerwelt

Event Timeline

I'm quite happy with the Firefox native MathML rendering. :3

Krinkle updated the task description. (Show Details)
Krinkle added subscribers: MSantos, Jdlrobson, ssastry, Krinkle.

Backport native mathml to 1.39

First, 1.39 will be EOL in November 2025 so let 1.39 EOL naturally will only delay sunset of Mathoid for several months.
Second, backporting a feature to older EOL seems not a practice before. For example we did not backport RenameUser to MediaWiki (Core) 1.39 LTS.
Third, removing Mathoid support from 1.39 seems more problematic - Mathoid is currently not yet even officially deprecated, let alone EOL.

sunset Mathoid in 2025

A comparible practice is when WMF switch Parsoid to PHP, the original JS version of Parsoid remains supported until the LTS version is end of life.

Note by default Math uses WMF's Restbase endpoint but that endpoint has not been officially deprecated yet. It is however fine to discontinue it when all pre-1.43 versions are EOL in November 2025, though a formal deprecation announcement must be made.

Note sunset of Mathoid will mean users using Chrome<63 can not see math formulas properly, since native MathML is only supported in Chrome 109 and Chrome<63 users do not get JavaScript (to run MathJax). However since we may drop grade C support for Chrome<70 as part of T367821: Discovery: Deprecation of TLS 1.2, it is not much a concern.

Also note (though not directly related to this task) in Chrome 51-66, users will not be able to log in to Wikimedia wikis since T344791.

Note sunset of Mathoid will mean users using Chrome<63 can not see math formulas properly, since native MathML is only supported in Chrome 109 and Chrome<63 users do not get JavaScript (to run MathJax). However since we may drop grade C support for Chrome<70 as part of T367821: Discovery: Deprecation of TLS 1.2, it is not much a concern.

Also note (though not directly related to this task) in Chrome 51-66, users will not be able to log in to Wikimedia wikis since T344791.

Thanks for bringing this up. I know some mathematicians use old FF (chrome does not seem to be that common) versions, and most Wikipedia will still be accessible without math formulae, to be honest. Thus, it's much less severe than not allowing to connect. Therefore, I would prefer to discuss those topics independetly.

For the first Tech News checkbox in Phase2a, please let me know approximately when this will be ready for announcement, and what details need to be included in the entry (including which link(s) to use). I.e. Do editors need to do anything specific for content in preparation, or read any related documentation, or provide any specific feedback, or keep an eye out for any specific types of bugs? Thanks!

Krinkle reassigned this task from Krinkle to Physikerwelt.
Krinkle updated the task description. (Show Details)
Krinkle updated Other Assignee, added: Physikerwelt.
Krinkle updated Other Assignee, added: Krinkle; removed: Physikerwelt.

The plan did not seem to involve communication to the mathematical editing community, who tend not to be avid readers of wikitech-l. I've now added a topic at WikiProject Mathematics of the enwiki.

https://en.wikipedia.org/wiki/Wikipedia_talk:WikiProject_Mathematics#Transition_to_MathML_rendering_as_default

Thank you that's great. There has not been any communication so far. Could you also share your post also to the community group? https://meta.wikimedia.org/wiki/Wikimedia_Community_User_Group_Math

Is there going to be a preferred place to discuss this? Already there has been a negative reaction as some of the torture tests https://www-archive.mozilla.org/projects/mathml/demo/texvsmml.xhtml fail badly.

Is there going to be a preferred place to discuss this? Already there has been a negative reaction as some of the torture tests https://www-archive.mozilla.org/projects/mathml/demo/texvsmml.xhtml fail badly.

I think the mailing list https://meta.wikimedia.org/wiki/Wikimedia_Community_User_Group_Math#Contact_information would be good.

This seems to be an old test, I am not sure if it uses the latest MathML core specification. I see the following in my browser from a newer version of that test

Screenshot 2024-09-18 at 16-29-53 Math-ML Torture test.png (4×716 px, 265 KB)

which looks quite good, I think.
https://pshihn.github.io/math-ml/examples/torture.html

@NSoiffer Do you know a good link for a current MathML core test?

Krinkle updated the task description. (Show Details)

@Krinkle I don't really see anything in the checklist about communicating to the actual end users the mathematical (and chemical) editing community on our main wiki's, rather than a bunch of mailing lists they don't read.

Should there be any sort of User Acceptance Test to ensure the product has an acceptable level of quality, before this is made default. Lets just say some editors are not happy with its current state

RESTbase will be switched off regardless – the alternative is simply not acceptable at the moment, so basically you are continuing to threaten to overnight make all of technical Wikipedia look ugly, illegible, and amateur, without any backup plan.

https://en.wikipedia.org/w/index.php?title=Wikipedia_talk:WikiProject_Mathematics&diff=prev&oldid=1249548295

And there are a whole host of issues raised, only a few of these have made it into bug reports yet. https://en.wikipedia.org/w/index.php?title=Wikipedia_talk:WikiProject_Mathematics&diff=prev&oldid=1246817156

On the Final pre-release QA round by Web Team item, how versed are the QA Web Team, on the fine details of mathematical typography? Is there a way we can supply them with information on what we feel is important?

Just to clarify, the Deploy opt-in user preference for MathJax mode. option is the client side renderer. There are a number of issues with that T375238, a few are blockers, but hopefully these are fixed.

For the first Tech News checkbox in Phase2a, please let me know approximately when this will be ready for announcement, and what details need to be included […]

We're currently waiting for the final QA round, and we have a number of early bug reports above to work through. Right now we're not yet pressed for time with inviting more users or entire pilot wikis. We can start drafting a message though. The main take-aways would be:

@Krinkle I don't really see anything in the checklist about communicating to the actual end users the mathematical (and chemical) editing community on our main wiki's, rather than a bunch of mailing lists they don't read.

As an active editor and volunteer for over ten years myself, I prefer it when the Foundation does software development in public. As such, the draft is available for everyone to see. The user preference has been live since July 2024. There are known issues being worked through every week since then. I made the call to not announce it widely until we're more confident in it ourselves. There is no set deadline for Mathoid yet, and it's not enabled anywhere by default (beyond beta cluster and group0 testing). As you can see above, there is no shortage of feedback at the moment so we're not in a rush to get wider feedback or enablement yet. It would just waste people's time reporting duplicate/known bugs, and make it less likely for them to try again later to find other issues.

Tech News has been around since 2013, and is sent both on-wiki and via mailing lists to self-appointed embassadors in the community. Indeed, most end-users are not directly subscribed to that. Short of requiring that all users are developers, or that only users are developers, this is inevitable I think. Instead, we rely on the indirection of Tech News to connect these dots. For example, I'm a volunteer on Commons and nlwiki. When I read this in Tech News or on Wikitech-ambassadors, I would bring this to the relevant math/chemistry project page there. Likewise, yourself and others active here in Phabricator can take this to places like WikiProject Mathematics on enwiki. I see you've done that already, which is great.

We generally encourage community members that are tech-savvy to do this outreach directly, exaclty as you did, and liason bug reports back to here. I don't mind doing further outreach myself as well on a handful of suggested talk pages. I speak Dutch, English, and a bit of German :)

Should there be any sort of User Acceptance Test to ensure the product has an acceptable level of quality, before this is made default.

Sure. Acceptence is based on known test cases passing without glitches, and bug reports like those above. I'd consider the following kind of bugs as blockers:

  • glitches where symbols appear that shouldn't.
  • glitches where symbols don't appear that should.
  • glitches where symbols appear in an obviousy incorrect place.

The goal is not pixel identical rendering to MathJax, so the exact font size and spacing may vary. This is in part a feature in that these aspects can now be customised with CSS via user styles, and are decided by individual web browsers. We do have some connections at browser vendors (Mozilla/Google/Apple) so if there are bugs caused by the HTML>screen rendering in browsers, rather than the LaTeX>MathML HTML rendering in MediaWiki, we can help escalate upstream bug reports as well. Free freel to share links to https://bugs.webkit.org/, https://bugzilla.mozilla.org/ or Chromium's issue tracker.

While I've written or edited an odd formula here and there myself, my editing activities are mostly in other subjects. When in doubt, I trust @Physikerwelt assesment of bug reports and community feedback over my own judgement.

On the Final pre-release QA round by Web Team item, how versed are the QA Web Team, on the fine details of mathematical typography? Is there a way we can supply them with information on what we feel is important?

I don't know what criteria or process the Web Team prefer to use for their QA round. If you have a few cases in mind, feel free to suggest them here, that can't hurt. Note though that the QA round informs when we can add the next pilot wikis. It is not directly tied to becoming the default everywhere, and thus is not a substitute for bug reports and the community assessment described above.

Screen Shot 2024-10-09 at 7.23.57 AM.png (936×1 px, 107 KB)

The above is what the MathML rollout page looks like in my browser. Leaving aside the math font being significantly too large in the top "SVG" example, as you can see, in the MathML version:

  • there are many subtle spacing flaws (for example the vertical spacing is wrong in the fraction x/2)
  • the superscript 2 uses the wrong font (it uses a scaled-down 2 from a font intended for regular size instead of a properly optically scaled 2 intended for superscript size)
  • the two lines at the bottom have been smushed together
  • the strikethroughs of the 'y' letters are much too small (and in this font, y is not a legible character to cross out because the strikethrough overlaps the leg of the y)
  • the renderer does not understand the explicit alignment directions of the source markup (which asked for left-aligned, center-aligned, and right-aligned) – though this is a poor example because people should be using LaTeX \align instead of \array for this specific thing.

Any page which is intended to be convincing would be much better if it drew a substantial number of complicated examples directly from technical English Wikipedia pages, instead of making up a few contrived and unrepresentative examples.

The goal is not pixel identical rendering to MathJax, so the exact font size and spacing may vary. This is in part a feature in that these aspects can now be customised with CSS via user styles, and are decided by individual web browsers.

This is a terrible idea. There are plenty of manual tweaks required to get math rendering looking good in edge cases or gaps in the default rendering, but which tweak is required differs depending on which font is used. Additionally many fonts just have bad built-in metrics, not to mention poorly distinguished symbols, etc. This entire endeavor is not going to work unless you pick a single specific set of math fonts and ship it down to every reader, so that authors are assured that readers are seeing a nearly identical result. The plan of giving every reader and every device different fonts is not an acceptable way forward for math rendering in English Wikipedia.

In response to the "plan" at the top of this page:

Enable native mathml by default, gradually on all wikis in production (Oct-Nov-Dec 2024).

This is completely non-viable on this timeline. It's nowhere close to ready in its current state, and making it the default on English Wikipedia within the next 2 months would be tremendously harmful and disruptive. It's frankly not clear that MathML would ever be a viable default. (Edit: I took a look at https://en.wikipedia.beta.wmflabs.org/wiki/Help:MathTestNative and quickly lost count of how many layout bugs there are, as my browser renders this page. Literally dozens of distinct problems.)

Client-side MathJax is a more plausible default, but it needs (a) consistent math fonts shipped down to readers, and (b) significantly more testing and feedback to be ready.

The current "SVG" (i.e. server-side MathJax rendering SVGs) is the only option that is currently ready for production use on English Wikipedia.

Thanks @SalixAlba for creating https://www.mediawiki.org/wiki/Extension:Math/Native_MathML/Reported_Cases

I've made a few updates there, reporting here for awareness:

  • It now includes both MW MathML (new) and MathJax MathML (normally hidden). This'll make it easier to narrow down whether differences are due to what MathML/HTML we produce (in PHP) vs something that affects both and maybe something we can improve in upstream browsers or via CSS tweaks.
  • Both MathML versions now get the same preferred larger font-size as set by enwiki/Common.css
BeforeAfter
Screenshot 2024-10-23 at 01.20.19.png (1×834 px, 121 KB)
Screenshot 2024-10-23 at 01.20.40.png (1×832 px, 120 KB)

Since last week, several bugs have been fixed by @Physikerwelt, @Andreg-p and others.

I'll mention two examples from prod vs current master in Beta Cluster:

https://en.wikipedia.beta.wmflabs.org/wiki/Extension:Math/Native_MathML/Reported_Cases

TaskBeforeAfter
T375907
Screenshot 2024-10-23 at 01.27.11.png (570×836 px, 57 KB)
Screenshot 2024-10-23 at 01.27.12.png (575×825 px, 56 KB)
T375317
Screenshot 2024-10-23 at 01.29.13.png (681×828 px, 50 KB)
Screenshot 2024-10-23 at 01.29.16.png (671×826 px, 50 KB)

In every browser on my computer, looking at https://en.wikipedia.beta.wmflabs.org/wiki/Extension:Math/Native_MathML/Reported_Cases the MathML examples are entirely unacceptable. The MathJax SVG has some issues but mostly looks okay. It's plausible that the problems with the MathML rendering is just down to using bad/broken fonts with insufficient character support, bad glyph shapes, poor metrics/kerning, etc. It's never ever going to work to expect every Wikipedia reader to bring their own math font.

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy