Tracking task for known incompatibilities between the new and old rule parser, before we can re-enabling the new parser.
Previous issues of site performance, citing AbuseFilter as contributing factor or root cause:
- T172447: Investigate 2017-08-02 Save Timing regression (+40-60%)
- T138550: 1.28.0-wmf.7 save time regression
As the parser will be working, unit tests for it should be enabled (see https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/AbuseFilter/+/453996/)
Plan for finishing AbuseFilterCachingParser
- Fix "Short-circuit evaluation with multiple comparisons". – T218906, https://gerrit.wikimedia.org/r/498127
- Fix "Short circuit with assignments . –" T214674 (only if the fix is easy; to be discussed...)
- Fix and re-enable unit tests for CachingParser. – https://gerrit.wikimedia.org/r/454557
- Deprecate "wild" abusefilter syntax. – T156096
- Introduce "AFPData::DNONE" concept. – https://gerrit.wikimedia.org/r/512135, https://gerrit.wikimedia.org/r/478424
- Investigate and fix AbuseFilterCachingParser bug with "if" conditions. – T152281
- Improve parameters handling - https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/AbuseFilter/+/531140/
- Allow ifs without else and reject empty conditions (because CachingParser doesn't support empty if). – T230727
- Speed up hoisting for the CachingParser – T230982
- Report errors inside short-circuited blocks when checking syntax. – T232498
- Change the CachingParser to throw an AFPUserVisibleException instead of a TypeError for empty operands. – T156096, T153251
- Fix any on-wiki filters that use "wild" abusefilter syntax, including trailing commas in function calls (T153251) – T156096
- This would be way easier with https://meta.wikimedia.org/wiki/Requests_for_comment/Creating_abusefilter-manager_global_group
- Right before enabling the new parser, re-check logstash and ensure no empty operands are logged – T156096, T153251
- Enable CachingParser in production according to the plan below
- Use CachingParser as default in extension.json - Not before completing the switch in production, ensuring that we have no regressions, and completing the deprecations above; tentatively proposing MW 1.35.
Plan for deploying AbuseFilterCachingParser
(See T156095#5434572 for a preview)
- Establish base line metric for amount of time spent in AbuseFilter during saving of edits. For example, the time spent in AbuseFilter within Backend Save Timing (mean avg, median, p75; for three different days in a given week), as well as the total amount of time (median, p75, for three different days).
- Enable on Beta Cluster and group0 wikis and monitor impact on the above metrics as well as the general Save Timing and Backend Save Timing dashboards in Grafana.
- If no performance degradation (T156095#5425912), enable on group1.
- If no performance degradation, enable on group2.
Nice-to-haves
- Fix AbuseFilterCachingParser bug causing stack overflow. – T148660 (not a blocker, per T148660#5411970)
Other perf work on AbuseFilter
- Reduce the frequency at which we store send data to the cache.
- Move AbuseFilter logic from static methods to a new AbuseFilterRunner class. – https://gerrit.wikimedia.org/r/478232
- Centralise perf data resets. – https://gerrit.wikimedia.org/r/498725 (and underlying stack of more changes).
- T53294: Overhaul internal profiling system
- T219092: Runtime profiling shouldn't include time taken to parse wikitext
