Wikipedia:Wikipedia Signpost/2016-12-22/Technology report
 Article display preview: | This is a draft of a potential Signpost article, and should not be interpreted as a finished piece. Its content is subject to review by the editorial team and ultimately by JPxG, the editor in chief. Please do not link to this draft as it is unfinished and the URL will change upon publication. If you would like to contribute and are familiar with the requirements of a Signpost article, feel free to be bold in making improvements!
| ||||
Labs improvements impact 2016 Tool Labs Survey results
In 2015, the Wikimedia Labs team ran a survey of Tool Labs developers to further understand how they felt about the service, what they liked, and what they wanted to see improved. This year, Bryan Davis, a senior software engineer at the Wikimedia Foundation providing Tool Labs support as part of Community Tech, ran the survey again, and announced the results on the Labs-l mailing list in late November.
The survey showed that developer’s opinion of reliability of the Tool Labs platform has increased to 87% from 64% from last year, and is likely the main thing users of tools hosted on Tool Labs care about. Davis attributed this to infrastructure work done by members of the Labs Operations team.
He specifically highlighted Chase Pettet and Madhumitha Viswanathan’s work on improving the underlying Network File System (NFS) servers for performance and stability, Andrew Bogott’s work on the even more underlying OpenStack cloud that powers the server’s that run Tool Labs. YuviPanda is leading the effort to migrate from the unsupported Open Grid Engine to the newer and more stable Kubernetes platform, which has led to cleaning up and improving the webservice code.
“Moving to a codebase that is about 20 years newer and supported by an active community can do a lot to improve things,” said Davis.
Davis did note that there may be some recency bias as the period before the 2016 survey was relatively stable, while there was instability in the months before the 2015 one.
The Labs team monitors uptime as well, explained Davis. For the months of October and November, the combined availability (all services being up) was 99.669%, or about two and a half hours of downtime per month on average. They are currently aiming for 99.9% availability, or 45 minutes of downtime per month.
Davis also shared some of the planned improvements in 2017 that should help with stability and reliability.
“We have a new database cluster coming online soon,” said Davis. “Rather than just setting up the same old system on bigger hardware the DBA team has taken a deep look at the problem of replication with filtering and made configuration and architecture changes to improve the whole stack.”
Another goal is to improve the OpenStack networking system by upgrading it to “Neutron”. This would allow the Labs team to distribute servers better in the datacenter, ensuring that a power outage for a single row of servers won’t take down all of Labs (as it currently would).
And as mentioned earlier, work on transitioning away from OGE will continue. Davis is currently working on evaluation criteria for the OGE replacement system, and expects to do actual testing next quarter.
Tool Labs developers also found the support to have improved over the past year, increasing from 71% to 89%. Davis credited the community for coming together and helping each other more, citing the number of people answering questions on IRC and improving documentation on wikis.
He also saw a lot of room for improvement, noting that documentation was the most mentioned problem in the free form comment section of the survey.
“We have pretty good coverage of highly technical topics on Wikitech, but there are very few start to finish tutorials on how to create an account, upload your code, and see your app running,” said Davis. “It would be nice to see "my first X" tutorials for different basic projects (web service, editing bot, IRC bot, etc) for various languages.”
Usage of the three main services of Tool Labs, LabsDB, cron jobs, and web services, were all down compared to last year, according to survey responses. But the number of respondents who don’t actively maintain any tools went up from 16% to 22%. Davis had a few hypotheses as to why, such as people switching to Quarry for database queries instead of direct access, or people signing up, finding it difficult, and then losing interest.
“I'd love to hear more from the Tool Labs community on the 'why' here,” said Davis.
Davis expects the survey results to affect planning for the Labs team in 2017-2018 and longer term.
“The single biggest take away for us early on is that the documentation needs to be improved,” said Pettet. “We need to update it, curate it, and make sure our users understand it.” He added that it was heartening that users were seeing the benefits of all of their efforts working on uptime and availability.
“I've got this vision that I think a few others share of a world where making a technical contribution to the movement has a really low barrier of entry,” said Davis. “Ideally manipulating the data we have about what is happening in the wikis should be as easy as editing an article. Sure there will always be a few rules and local conventions that you need to follow, but you shouldn't have to learn a lot of new technology before you can get some work done.” He pointed to YuviPanda’s talk on “Stealing some of Wikimedia's Principles to Democratize Programming” that discusses the same kind of world, using the popular Quarry and relatively new PAWS tools as examples of lowering barriers to technical tasks.
“We are never going to get everyone to freely share in the sum of all knowledge if there are arbitrary silos that large numbers of people are locked out of just because they don't use the right computer operating system or understand the technical difference between an array and a vector,” said Davis.
In brief
New user scripts to customise your Wikipedia experience
- Page Curation[1] (source) by User:Lourdes – Adds a "Page Curation" link to the top toolbar, primarily designed for new page reviewers.
- TFA History Link[2] (source) by User:Lourdes – Adds a "TFA History" link to the top toolbar, primarily designed for Featured Article contributors, reviewers and recent changes patrollers.
- centralAuthLink[3] (source) by User:The Voidwalker – Adds links to the Central Auth for a user from in their: userspace, contributions, or logs.
- lastEdit[4] (source) by User:Opencooper – Shows the last person to edit the page and the relative time of the edit under the page title.
Newly approved bot tasks
- DatBot (task 3) – Reports users who have either tripped 5 filters quickly in a list, a filter that is in an "immediate report" list, or a very suspicious username-related filter.
- MusikBot II (approval) – Maintains the AWB CheckPage and counts the number of users with access.
- Bender the Bot (task 3, task 4) – replace
http://withhttps://for YouTube, and for selected domains. - Monkbot (task 11) – Fix cs1|2 author/editor parameters in articles listed in Category:CS1 maint: Multiple names: authors list and Category:CS1 maint: Multiple names: editors list.
- Dexbot (task 9) – Adding quotation marks to ref names containing special characters.
- PrimeBOT (task 5) – Add autosubst coding to wrapper templates in order to replace them.
- WugBot (approval) – Reads WP:GAN and posts the oldest nominations on Wikipedia:Good article nominations/Report.
- ZackBot (task 5) – Rename deprecated parameters in transclusions of {{Infobox NFL biography}}
Latest tech news from the Wikimedia technical community: 2016 #49 & #50. Please tell other users about these changes. Not all changes will affect you. Translations are available on Meta.
- Recent changes
- You can choose to see users from specific user groups in Special:ActiveUsers. (Phabricator task T116354)
- Everyone can now see Special:UserRights. Previously only those who could change user rights could. Other users got an error message. (Phabricator task T27319)
- ORES can now show how likely an edit is to be damaging to the wiki with different colours. This only works for languages that have trained ORES to recognize damaging edits. (Phabricator task T144922)
- You will now see categories with 0 pages in Special:Categories. Previously you did not see empty categories there. (Phabricator task T12915)
- Users who have Yahoo email addresses could not use Special:EmailUser to send emails. This has now been fixed. Emails will now come from a @wikimedia.org address. Users who get an email from you will still reply to your email address and be able to see it. (Phabricator task T66795)
- You can now see how many categories and pages there are in the categories in Special:TrackingCategories. This is to help you find pages that could need attention. (Gerrit code review)
- Markup colours for reviewed and pending revisions in the page history and recent changes and logs now match Wikimedia standard colours. The "You have a new message on your talk page" notification will have a slightly different colour. (Wikitech email list)
- Problems
- Because of work on cross-wiki watchlists global renaming is not working. The plan is to turn it on again on 16 December. Global renaming was turned off for a while in late November and early December as well. (Phabricator task T148242)
- Future changes
- The 2016 Community Wishlist Survey will decide what the Community Tech team will work on next year. Voting for wishes on the survey page concluded on 12 December. You can see what has happened to last year's wishes on the 2015 results page.
Installation code
- ^ Copy the following code, click here, then paste:
importScript( 'User:Lourdes/PageCuration.js' ); // Backlink: User:Lourdes/PageCuration.js - ^ Copy the following code, click here, then paste:
importScript( 'User:Lourdes/TFAhistorylink.js' ); // Backlink: User:Lourdes/TFAhistorylink.js - ^ Copy the following code, click here, then paste:
importScript( 'User:The Voidwalker/centralAuthLink.js' ); // Backlink: User:The Voidwalker/centralAuthLink.js - ^ Copy the following code, click here, then paste:
importScript( 'User:Opencooper/lastEdit.js' ); // Backlink: User:Opencooper/lastEdit.js
Discuss this story
Very nice overview of tools and labs. Highly appreciated. —TheDJ (talk • contribs) 10:36, 22 December 2016 (UTC)[reply]