Wikipedia talk:New pages patrol/Coordination/2022 WMF letter

Archives

Reply from WMF product staff (and invitation to continue the discussion!)

Note: I originally posted the below as a comment on this Signpost article about the open letter, where it has gained some discussion. I was then encouraged to also post it here so that the people interested in the letter would be more likely to see it. Thank you for reading, and let's continue the conversation!

Hello everyone – I’m Marshall Miller; I’m a Group Product Manager at WMF, and I’m one of the people at WMF who has spent time working on and thinking about New Page Patrol on English Wikipedia. I’ve gathered up notes from the rest of the team and checked in with Selena Deckelmann (the WMF's new Chief Product & Technology Officer), so that we can respond to the open letter from the New Pages Reviewers, and this Signpost “In focus” piece. Thank you all for working hard on the difficult task of reviewing pages, and to those of you who have made improvements to the software by submitting patches, and I hope we’ll have good conversations going forward.

We definitely agree that New Pages Patrol's work is important: keeping out newly-created articles that are bad-faith, self-promotion, or simply not ready for inclusion in the encyclopedia. We can see that there's lots of support in the community for improvements to lighten the workload for New Pages Patrol's hard-working reviewers, and so we'd like to invite New Page Reviewers, and whoever else is interested in the PageTriage software and new page processes, to a meeting to talk more about the specific needs and work together to improve the process (more information at the end of this reply).

For those that may be newer to this topic, we’d like to share some information about WMF's support for the PageTriage extension:

• The PageTriage extension was developed by the WMF's original Features Engineering team in 2012, in partnership with English Wikipedia editors. Over the years, the New Page Reviewers have built extensive workflows and processes around the original tools.
• In 2017-2018, at NPP’s request, the Community Tech team ran the ACTRIAL research study to evaluate the impact of restricting page creation to registered users. As a result of that research, the restriction was made permanent.
• Following that study in 2018, in response to community requests, the Growth team made some improvements to the extension by adding quality assessments and copyright violation scores for each page, and allowing people to filter based on those qualities. This work also made part of the toolset usable by the Articles for Creation process.
• In the 2019 Community Wishlist Survey, User:Insertcleverphrasehere posted a proposal: Page Curation and New Pages Feed improvements. New Page Reviewers and supporters came out in force to vote for that proposal, and it ended up as the #1 proposal for the year. (Here are the results.) Because of the high number of votes, the Community Tech team worked for more than six months making improvements to PageTriage, completing 13 different wishes that were prioritized by the NPP members.
• There have been two Community Wishlist Surveys since, in 2020 and 2021, but those years didn’t have a Community Wishlist proposal about PageTriage.
• In the recording for the 'Conversation with the Trustees', we wanted to clarify that the trustees said that New Pages Patrol and Articles for Creation workflows are community developed, not that the PageTriage extension is.
• There are some more details about the WMF's support for PageTriage in a previous response posted before the letter was published.

We understand that the people who worked on the open letter would prefer ongoing dedicated resources rather than relying on the Community Wishlist. As we continue this discussion going forward, we just wanted to say that the Community Wishlist remains an opportunity. Given the success of the NPP proposal in 2019, and the strong show of support from the signatories of the open letter, it's likely that a proposal to the next Wishlist Survey in January 2023 would be successful, and would result in more improvements to PageTriage.

But thinking more broadly, (like we mentioned above) we'd like to talk further about what the group considers to be the top priorities for improving the extension. When we invest in tools and features, we want to build things that work well for as many people and communities as we can – we’re trying to think about the reviewers, the new editors, edit-a-thon organizers, and the different languages and wikis around the world. We will be inviting anyone interested to join us for a conversation where we can hear about your priorities and get perspectives from other wikis and parts of our communities that are interested in new pages work. We will work to find a convenient time in the coming weeks where we can all meet to discuss and will follow up at NPP's talk page with an invitation. Please feel free to share it with anyone who may be interested in the discussion.

We're looking forward to continuing the discussion, and also please let us know if this reply should be posted in other places! MMiller (WMF) (talk) 18:08, 3 October 2022 (UTC)

Hi Marshall. Thank you for reposting here, it's nice that we finally have an official response from the WMF and/or the Board of Trustees that our reviewers can follow and join in, and I know from our previous collaborations that you and our team can work very well together. These issues are of such importance that we are still expecting a reply from Ms Iskander herself or at least from CPTO Ms Denckelmann with at least a short word of assurance from them that they are personally aware and have instructed their managers to take this on. I, and I'm sure the lead NPP coords MB and Novem Linguae will be pleased to have a live meeting with you that will be a genuine multi-way dialogue rather than the one-sided lecture our colleague received from the BoT.

The goal is not only to make urgent upgrades to PageTriage to make it more streamlined and comprehensive but, but also–and equally important–to develop a first and correct onboarding system for new users and make all these systems available to other language Wikipedias whose editors desire them. These tools will be invaluable o the organisers of the growing number of edit-a-thons and Wiki Education programmes that are often the source of problematic new articles. I would like to emphasise therefore, that due to the full scope of what is needed, we are looking first and foremost for a commitment from HR and the financial deciders that funds and personnel will be allocated to these developments, and probably the creation of a dedicated section parallel to Growthn/Community Tech that will not encroach on their bandwidth.

Rather than clutter the reviewers' general talk page, this page here is probably the best venue for all discussions on these specific issues and back-and-forth collaboration on development once priorities are defined and it gets underway. Kudpung กุดผึ้ง (talk) 20:43, 3 October 2022 (UTC)

On this last note, I will just add that this project and its talk page are an excellent example how well a dedicated team from NPP and from AfC were able to collaborate successfully, with the WMF under your tutelage. As Primefac was also a major participant and as our NPP goals will ultimately lighten the workload of AfC, he might wish to follow and comment on what we would like the WMF to undertake. Kudpung กุดผึ้ง (talk) 21:10, 3 October 2022 (UTC)

Hi again Marshall. We have now received a very cursory comment from Ms Iskander that she is aware of the issues. There is no further comment that she understands or is concerned with the importance of NPP and controlling the flux of new articles and how the reputation of Wikipedia depends on it. Likewise there has been no response from Ms Denckelmann whom we assume to be slightly more directly involved but in overall responsibility for your department. No one within the en.Wiki and even some of the WMF staff who are also regular editors here are sure about the lines of authority/responsibility within the WMF. Apart from a couple of emails, there has also not been any official response from the BoT.

I have been reminded again that dialogue with the WMF on moving forward will be difficult if at first there is not a commitment to increase the staff. I have been sent a complete breakdown from Phabricator volunteers of the estimated time and costs involved, and as a non-programmer I was surprised that only a very small budget is required. Mr Horn has organised work on Page Triage through his Wishlist that cost considerably more, and the project that you and I worked on in 2018 didn't appear to have difficulty on getting resources assigned. We think that it should be possible to establish the way forward through a preliminary meeting very soon (days rather than weeks) before discussing the actual details. Perhaps you would like to have a quick chat with one of us next week on FaceTime or Telegram. (FYI @MB and Novem Linguae:). Kudpung กุดผึ้ง (talk) 07:02, 6 October 2022 (UTC)

Hi @Kudpung -- thanks for your update. I just wanted to assure you that we haven't forgotten about this! I agree that once a project is underway, it is helpful to have detailed conversations in small groups. But the first step is figuring out the when/what of the project, and our plan is still for the first step to be this larger conversation that helps us all see the "new pages" challenge broadly, so that we can figure out what action to take. That's still getting scheduled, and we'll keep you all posted on that. I know that the call with the Board is in a few days, but we're not going to be able to get the new pages call scheduled before then. I know you've put a lot of work into getting these conversations going, and thank you for being patient. MMiller (WMF) (talk) 22:29, 17 October 2022 (UTC)

Hi Marshall, we're being as patient as possible but Mr Horn's improptue comment was 6 weeks ago and it's now already three weeks since the Open Letter was officially published.

The NPP team can explain in 5 minutes for the WMF to see the "new pages" challenge broadly. It does not take a big meeting with the entire staff and representatives of all the other Wikis and Commons. We have a holistic plan that will at the same time encourage more reviewers to do the work, and, most importantly, encourage new users to submit new articles that are not time wasting inappropriate junk, thus reducing the workload for both NPP and AfC. This is separate from Growth's major project to mentor and retain new mobile device users, which again is a WMF initiative and AFAIK not one that was asked for at the Wishlist, and it seems to be what is using all your capacity.

The greatest problem however, is one of communication with the WMF and the feeling of being ignored. We have qualified, experienced software engineers in our group who know their way round MediaWiki, and it's been calculated that developer time for what's needed would not be more than the time spent on the work you and I collaborated on in 2018 together with the fixes that were done through the Wishlist campaign in 2019. However, it's not acceptable that our NPPers should be doing the Foundation's work at Phab for free. I'll mention again that it is expected that the first meeting will conclude with a firm commitment from the WMF to allocate a budget and deploy human resources to the required tasks.

I'm sure you understand all this and that the en.Wiki is the most important of all Foundation projects, but it's convincing your superiors, and they are currently far more concerned with pursuing abstract long term plans for the future of the movement rather than the reliability and quality of the articles and the reputation of the encyclopedia. Let's move forward. FYI @MB and Novem Linguae:. Kudpung กุดผึ้ง (talk) 03:55, 18 October 2022 (UTC)

New page processing flow chart

Page process flow chart

As the effort is two-fold:

1. To address the bugs and features for PageTriage
2. Assist newly registered users to create their first page, thus reducing the workload on NPP & AfC

I'm just parking this image here that I made a while back because if and when resources are allocated it will be one of the hinge factors for discussion and development. Kudpung กุดผึ้ง (talk) 08:20, 8 October 2022 (UTC)

Reply from the CAC

Dear all, The Board of Trustees' Community Affairs Committee (CAC) is following this. We hope you will continue to engage and work with the Product team to discuss priorities. On behalf of the CAC, Shani (WMF) (talk) 19:31, 13 October 2022 (UTC)

No need to 'hope'. I will certainly be more than ready and pleased to engage and work with the Product team to discuss priorities. Kudpung กุดผึ้ง (talk) 21:25, 13 October 2022 (UTC)

New page review conversation: November 3 at 18:00 UTC

Hello everyone -- in my original reply to the open letter above, I said that our next step will be a conversation to talk about priorities for New Page Patrol and for the reviewing of new pages in the wikis more broadly. We now have a set a time for that, and this message is the invitation to attend! (@Novem Linguae -- you asked about this in the call with the Board's community affairs committee last week).

I'll be facilitating this conversation, and it will have several staff members from WMF Product leadership. In addition to people interested in NPP on English Wikipedia, we're inviting some people who review new pages on other wikis, as well as people who work on efforts to submit new pages (e.g. edit-a-thons). We think this will help us get a well-rounded perspective on new page reviewing, so that we can think about what investments could help the most people around the movement.

We're planning this as a video call (you can, of course, be off-camera) on November 3 at 18:00 UTC. Note that this is during the part of the year when the United States is on a different daylight savings schedule than other parts of the world, so please double-check what time this will be for you. This zonestamp tool may help.

We're still settling on which video conference software we want to use, and I'll be following up here with the specific link later this week.

If you plan to attend, please reply here so we know how many people to expect! Thank you, and I'm looking forward to the conversation. MMiller (WMF) (talk) 18:15, 24 October 2022 (UTC)

Attendance list

1. Planning on attending (18:00 UTC = 01:00 AM Thai time) Kudpung กุดผึ้ง (talk) 23:52, 24 October 2022 (UTC)

   Thank you for making that time work, @Kudpung. I know it's late for you, but we wanted to be able to get the right product people on the line. MMiller (WMF) (talk) 03:27, 25 October 2022 (UTC)

2. Looking forward ~ 🦝 Shushugah (he/him • talk) 02:05, 25 October 2022 (UTC)
3. Can attend. MB 04:22, 25 October 2022 (UTC)
4. Will attend. Thanks for the details. If anyone wants to double check my calculation, I think for USA pacific time, this meeting is at Nov 3 @ 11AM. Please correct me if I'm wrong. –Novem Linguae (talk) 21:15, 26 October 2022 (UTC)

   @Novem Linguae -- glad you'll be there! Yes, it's at 11 AM PT. MMiller (WMF) (talk) 21:26, 26 October 2022 (UTC)

5. Atsme here! That will be 2:00 pm Bonaire (EST) time on Nov 3rd? Atsme 💬 📧 02:05, 27 October 2022 (UTC)

   Yes, @Atsme -- thank you for attending. I can't speak to Bonaire's timezone, but it will be at 2:00 PM eastern time (e.g. New York). See you there! MMiller (WMF) (talk) 15:52, 27 October 2022 (UTC)

6. Should be able to attend this — TheresNoTime (talk • they/them) 07:15, 27 October 2022 (UTC)

   n.b. I just wish to note that I am attending entirely in my capacity as a volunteer 🙂 — TheresNoTime (talk • they/them) 20:43, 27 October 2022 (UTC)

7. I'll be there on behalf of Wiki Education. :-) --Sage (Wiki Ed) (talk) 16:30, 1 November 2022 (UTC)
8. I will be there from the Hungarian Wikipedia and from Wikimedia Hungary. Samat (talk) 21:31, 1 November 2022 (UTC)
9. I would like to join the call (not representing any affiliate or project) Ainali (talk) 09:32, 3 November 2022 (UTC)
10. I will be there.. :) --FShbib (talk) 17:21, 3 November 2022 (UTC)

Details

Hello MB, Kudpung, Novem Linguae, and Shushugah -- I now have all the details for the meeting. Let me know if you have any questions, and thank you for attending!

Time: November 3 at 18:00 UTC. Please be mindful of daylight savings in your location!

Link: We'll be using open-source Jitsi for a video call (you may be off-camera if you prefer). The link is https://meet.jit.si/NewPagesRoundtable.

Hi, Marshall. As you already know all the people from the en.Wikipedia who will be attending, we think it would be helpful if you would tell who will be attending from the WMF and from other Wikimedia projects. It would help us to formulate our part of the discussion to the benefit of the others who are not familiar with NPP. Thanks. Kudpung กุดผึ้ง (talk) 09:10, 30 October 2022 (UTC)

Hi again, Marshall. As we we have not been informed who will be participating from the WMF and as 01:00 AM is not really a convenient time, I will probably not be attending the meeting after all. The meeting was called due to an initiative of the English Wikipedia's NPP team and I have every confidence that @Novem Linguae and MB: can explain any details that you or your colleagues are not aware of already. When you have increased the resources, I will be more than happy to work with you or your team on the development as we did most successfully in 2018. Kudpung กุดผึ้ง (talk) 22:28, 1 November 2022 (UTC)

Hi @Kudpung -- I've asked the people who are planning on attending to put their names on the list above, and I see a couple have now (Sage and Samat). I know this is at a tough time for you, and I hope you're able to make it. MMiller (WMF) (talk) 23:45, 1 November 2022 (UTC)

Hi, Marshall, I was asking about your colleagues from the WMF. Will you be alone to facilitate the meeting and represent your team.or will there be others? Kudpung กุดผึ้ง (talk) 01:39, 2 November 2022 (UTC)

Hi @Kudpung -- I'm sorry for the misunderstanding, and I'm happy to say who will be attending from WMF. I'll be there along with Selena Deckelmann (CPTO), Danny Horn (Director of Product for Contributor Tools), Mayur Paul (Director of Movement Communications), and Gergő Tisza (Staff Software Engineer). I think with this group, we'll bring lots of institutional knowledge and different skillsets to the conversation. MMiller (WMF) (talk) 02:02, 2 November 2022 (UTC)

Hey there. Is there an agenda available for this meeting? If so, feel free to post it here. –Novem Linguae (talk) 08:44, 2 November 2022 (UTC)

Hello @Novem Linguae -- I haven't set too much of an agenda, because I want us to be able to have an organic conversation about whatever surfaces as most important. We'll start with introducing everyone in the meeting, and then with NPP people setting the table around the most important things to discuss. As the facilitator, I may also prompt people to chime in with their perspective (e.g. to the person from Hungarian Wikipedia: "Do you see similar problems in Hungarian Wikipedia?"). It's possible that we get into the weeds on some of the needs (an engineer will be present if we do: @Tgr (WMF)), or it may wind up being more high-level. How does that sound?

Is there anything you think we should definitely include in the agenda? MMiller (WMF) (talk) 17:39, 2 November 2022 (UTC)

Sounds good. I'll probably be providing some direction to the meeting from the NPP side, and I'll probably just go over the bullet points I emailed. Hope that works. Looking forward to a good chat :) –Novem Linguae (talk) 18:13, 2 November 2022 (UTC)

Feedback

I was happy to see so many folks interested in attending today's meeting and was a bit disappointed we weren't addressing the needs of this pedia's NPP more. It was an initial conversation, so more direct contact will be better, with any luck and focus. While I'm excited to see everyone's willingness, just like a new user's focus might be on initial success, we would be wise to keep our focus on improving the user experience and pure enjoyment of our new volunteers, a word we do not use often enough on Wikipedia, often instead using the term wikipedian as embracing all of us. A new volunteer is like found money. Positive experiences and feedback can retain many. We want more volunteers and we especially want happy volunteers, so there must be some code-related process for new users which not only helpfully facilitates, but also encourages, appreciates and even trains them in the moment. They came here to participate; many are anxious to know just how to do so. Enjoyment, activity, service, and training; these are elements which improve the user experience. If we can nail a system embracing all four, we'll have so many new helpers even Kudpung might sleep more soundly. BusterD (talk) 20:07, 3 November 2022 (UTC)

Summary of meeting #1 from Novem's perspective

New page patrol requested the following from WMF:

1. Software engineer to work on all of the 25 high priority tickets ("soon" column on this workboard)
2. Software engineer to work on some of the 100 low priority tickets ("backlog" column on this workboard)
3. Software engineer to review/approve/merge PageTriage patches written by volunteers. This is a major bottleneck in the patch writing process because so few users have access to "approve" patches (called +2)
4. Help with fleshing out a better workflow for new users who try to create an article. In particular, creating a new user welcome page, and revamping Wikipedia:Article wizard and Help:Your first article.

Also discussed:

        5. Making PageTriage work on other wikis (phab:T50552)

WMF:

• Agreed to bullet point #3, and didn't sound optimistic about the rest.
• Seemed inflexible/unable to create new resources such as new developers or new teams.
• Seemed unable to escape the bureaucracy of their Annual Planning / Budgeting process undertaken in May/June every year. Gave the impression that no software work can be done ever at WMF unless it gets planned in May/June, which seems like a very rigid system with no flexibility.
• Said to work on our software, they'd have to take software engineers away from other projects and teams.
• Think the scope of this project requires more than one software engineer.
• Don't seem to want to prioritize this software because it is too enwiki-centric.
• Packed the meeting with folks from other wikis, who talked about a bunch of other things unrelated to PageTriage and our open letter, such as FlaggedRevs. This made the scope of the meeting too wide, the focus too wide, and reduced the amount of time and focus spent on our issue.

I would prefer that our next meeting be much more focused on our open letter and PageTriage, and much more enwiki-centric. If other wikis have problems with their software, they are welcome to communicate this to the WMF and have meetings directly with WMF about it.

Ending on a positive note, thank you for having some key staff attend the meeting. The meeting was well-attended by top WMF staff. Also, thank you for 1) pledging to review volunteer patches and 2) for being open to a second meeting. Let's move forward with those two items and re-evaluate.

Sincerely, –Novem Linguae (talk) 20:53, 3 November 2022 (UTC)

Feedback from Kudpung

Very much the same as Novem Linguae and BusterD and I'm very grateful to Buster for having taken part and underlining the issues explained by Mike, Novem, and me. I do feel that Selena's answers were not quite what the en.Wiki community is hoping for, but she is new on the job. IMO the WMF could quite easily be more flexible with its glut of funds without waiting for the next callendar plan from the foundation's 'Minister of Finance' (whoever that is). The WMF's work is so diverse nowadays that it still does not fully understand that the experienced volunteers are its biggest asset, the en.Wiki is the biggest puller of donations, and that NPP is what keeps the corpus clean and maintains the movement's reputation for neutrality and accuracy of its encyclopedias.

These are all in my opinion far greater priorities than using up the Growth team's entire budget and bandwith on the mentorship programme they have been developing. I will takes years for any metrics to emerge that demonstrate any impact it may have had on new user participation and retention. This kind of planning leaves no Spielraum for contingencies and other eventualities that could and always will arise at any moment. I am still convinced that on the subject of new page creation workflows and UX, the volunteers who do this day-in, day-out, know best, and some of them are expert coders and UX professionals. The allocation of a dedicated WMF engineering team to en.Wiki's workflows is however, nevertheless absolutely essential.

The meeting was nevertheless an excellent (and the first ever of its kind) opportunity to air the communities' concerns and in this it was a success. I am confident that the next meeting will be more focused and that concerns discussed by representatives of other Wikis can be topics for other, separate meetings, but without losing sight of the goal of making PageTriage wiki-agnostic. This meeting was a new and important step towards bridging the 'us vs. them' divide between SF and the Foundation's volunteers since Danny Horn's help getting ACTRIAL approved and demonstrating that the Foundation can indeed be flexible; it has certainly led to relieving a lot of built-up tension and has paved the way to a much better understanding all-round. Kudpung กุดผึ้ง (talk) 05:04, 4 November 2022 (UTC)

Bluerasberry's meeting notes

These are my own notes from the meeting. I am presenting this as my own interpretation of what was said.

Attendees

1. Marshall Miller manages web editing and growth teams at Wikimedia Foundation convened the meeting. Meeting was audio and video recorded.
2. Selena co-hosted the meeting and said that she worked on all the wikis
3. Samat from Hungarian Wikipedia
4. Betty Wills - Atsme on Wikipedia on New Page Patrol
5. Sage Ross - Ragesoss - chief technology officer at Wiki Education, try to build systems for student editors in classes
6. Novem linguae - joint patrolers for new page patrol, volunteer software developer who contributed 15 patches to
7. Chris / kudpung - 12 years experience with New Page Patrol
8. BusterD - admin with English Wikipedia and editing since 2005
9. Jan Ainali - admin on Swedish Wikipedia doing vandalism patrol there
10. Shabib - Arab Wikipedia admin, active since 2011
11. Danny Horn - work at Wikimedia Foundation as director of product management for contributor tools, have been developing software for 8 years
12. MB user:MB, Mike - one of the coordinators of NPP with Novem. Got involved in this earlier this year when the backlog was out of control at 6000 and looking for solutions
13. Gorgo - software developer for the growth team
14. Lane - I am staff Wikimedian at the School of Data Science at the University of Virginia. I have coordinated research on detecting Wikipedia misconduct using machine learning and artificial intelligence.
15. Justice - Wikimedia Ghana user group
16. Marshall Miller - this is a casual conversation

notes

1. Novem - I will talk for about 5 minutes like I did at the board of trustees meeting
   1. thanks everyone for attending! I see a lot of staff members at the Wikimedia foundation including all the key decision makers
   2. Background: English Wikipedia has a team called new page patrol
   3. we use software which is managed on phabricator and has 150 problem tickets
   4. We needed a plan to improve the software so we intiiatiated a petition which got more than 400 signatures from English Wikipedia
   5. we have identified some of the biggest issues which have persisted for years
   6. there are volunteer software developers but it is enough
   7. the community finds this software very important for Wikipedia's quality control process
   8. the people who use this software have high expertise on Wikipedia's policy, so this is critical work by experienced Wikipedia editors who have the supprot of the community
   9. we prioritized the parts of this that we want developed
   10. https://phabricator.wikimedia.org/project/view/541/
   11. the number one priority is that we want software engineers to address the issues in the "soon" column, which are the highest priority problems
   12. we would like the ability to instantly "draftify" software
   13. we want the functionality that is built into the twinkle software for logging issues to be moved into this more accessible software
   14. other issues which are lower priority are to help us with the backlog problems, including some reports that are 10 years old
   15. we have volunteer written patches submitted but no one at the Wikimedia foundation is reviewing these pull requests, or even acknowledging them. We want them addressed because only the WMF staff have permission
   16. we have designs for new user orientation which need to be built into the interface and need assistance
2. Marshall - thanks, since you sent these to us we reviewed these.
   1. Marshall: does anyone from the Wikimedia Foundation side have anything to say about this?
   2. Selena - no comments. Comments from New Page Patrol members?
   3. Chris - We would like staff to address the bug reports.
   4. MB - The Wikimedia Foundation staff have directed us to the Wikimedia Community Wishlist, but we do not think this is a workable solution. This problem is too big for that request queue and also we have so many community members supporting this that it does not make sense to enter that competition.
   5. Selena - it seems reasonable that we can address the bug reports. We had an initial conversation as Marshall mentioned. The feature requests may be able to t be met with a more systemic approach. Like Marshall said, we expect to have follow up conversations about our approach.
   6. Marshall - Novem, it sounds like there is a cluster of requests on that board which are about extending what the tool dashboard can do. We are considering a redesign of the tool.
   7. Novem - there are already json configurable tools in the board
3. Marshall - Is there anyone from a wiki other than English Wikipedia who has comments?
   1. Samat - I have a personal interest in editor retention. I believe that having this tool has a significant effect on increasing editor retention and also supporting reviewers. We have more than 10,000 pages on my wiki which are unreviewed, and this tool could be a way to get this review. When some new user makes changes in Wikipedia and the reviewers do not quickly respond, then they become discouraged from the delay. Two or three years ago we did a configuration change of flagged revisions which meant that even though all the changes were visible before, this had a good effect on retention but the active editor community found this problematic. The Wikimedia Foundation asked the community what they should change, but the old software we were using at that time has never been updated or changed. There was another idea earlier which meant that artificial intelligence or the ORES system could help, if it had high confidence that an edit was not harmful. That is an example of one of the other ways we could reduce
   2. Marshall - yes, Hungarian Wikipedia is one of the wikis that uses flagged wikis. Many do not use this technology. It is helpful to hear.
   3. Tgr: Yes, I agree, flagged revisions is useful technology and is worth developing, but it does not fill the role of the New Page Patrol technology. They are separate and developed for different reasons.
   4. Shahib - in Arabic Wikipedia we have tried to do community organization and establish a task force to address the backlog of articles which need review. We set up groups of 2000 articles using a bot, then we recruit volunteers to review those from a list without other technology. Right now we have 16,000 articles which need to be reviewed. Page Triage is a great tool for addressing this challenge. It would be very helpful to have access to tools. In Arabic Wikipedia we mark unreviewed pages with a template which indicates that they need review.
   5. Marshall - Hmm - in English Wikipedia this is logged automatically in the database without a template on the article.
   6. Jan - in Swedish Wikipedia we do not have special tools for helping review. We have use "special:recentchanges", which is a basic feature of MediaWiki. I am interested in this as part of Movement Strategy, which says that there will be development of Cross-wiki tools. Instead of developing this just on English Wikipedia I would like for others to have access to this tool. https://meta.wikimedia.org/wiki/Movement_Strategy/Initiatives/Cross-Wiki_Tools
   7. Selena - Do we know what barriers exist to installing the extension on other wikis?
   8. Novem - there is a ticket to start New Page Patrol on other wikis. WMF staff members said that they would not want to adapt the tool for other languages without completely re-writing it. I disagree - I think the essential features of the tool can be adapated. The extension is already in translate-wiki, and volunteers have already translated the front end of the tool into various languages. I would like to get the tool to be wiki-agnostic so that it can work on any wiki. I understand taht different wikis have different workflows and there would need to be configuration options.
   9. Samat - I would like to react to Jan's comment. In the long term I also think that different wikis should not use different tools for page patrolling.
   10. Selena - I can speak to this. The way that the strategy for features works today is that it is scheduled during the annual planning process which happens once yearly. To prioritize it immediately would require looking at that entire process which just occured in June. To look at the size of this effort, which is a reconfigure which looks across many wikis, requires advanced planning. I have been at the Wikimedia Foundation for 14 weeks so am still now. For now I would like to get code reviewed because I think that can fit in the annual plan which is already approved, and beyond that, I have to look at how this might change everything else we are working on. I would like to move forward together, but it would be challenging to revise the plan that is already in place.
   11. Marshall - Page Triage is not already on a path to be developed for all wikis. Our policy is yes, we do want to develop tools which work on all wikis. Page Triage is an exception where the Wikimedia Foundation has developed it only for a few wikis.
   12. Samat - what can we expect about the future of flagged revisions?
   13. Marshall - We're not as prepared to talk about FlaggedRevisions today -- and we haven't thought about it as much. Maybe we can talk about it via email afterward.
4. Chris - Selena, this is a question for you. I have been with Page Triage since the beginning of it. I helped discuss it with Eric Moeller 12 years ago and Brandon Harris after that. Back then the plan was to make it wiki agnostic from the start. Why has it taken so many years to get to this point?
   1. Selena - Eric lives in Portland where I live. I have been here 14 weeks and just orienting myself.
   2. Chris - at a recent Wikimedia Foundation meeting we heard that you are overseeing product and will take responsibility. I do not want this challenge to persist another 12 years.
5. Selena - I would like to identify the aspects of volunteer work which will have the most impact on the Movement Priorities. We would like to make the work of the volunteers as efficiently as we can to use resources we have across a global movement. There are lots of ways that triage happens across many different wikis. A couple of engineers on their own cannot fix these challenges. I cannot allocate an engineer to fix this successfully, and it will take group effort and coordination with many volunteer groups.
   1. Tgr - I was around in the beginning as a technical volunteer. In the late 2000s there was a focus on article quality. That was when flagged revisions and wikitrust were created. At that time we also had many users leave Wikipedia and other external trends, like the shift to mobile users, scared the Wikimedia Foundation. Also many younger users were using less text content and more video content.
   2. Sage - we have a lot of students create new articles. A number of them have issues which are surfaced by the New Page Patrol process. There are barriers for users to do New Page Patrol. Using the tool requires experience and approval from trusted users, and this means that New Page Patrol is a high quality service. Interactions with how new users can interact with New Page Patrol is very opaque to new users. Like for example, when the New Page Page reviews a new page, it gives a message that "your page has been patrolled". This is supposed to be a good and encouraging thing, but it alarms some new users and it is unlike other Wiki processes. Users cannot react with patroll logs in the way that they interact with people in typical on-wiki conversations. It would be ideal to integrate the review process with our other wiki activities.
   3. Chris - what Sage is saying is interesting and strikes a chord with me. I would like to bring new users into the process and help them understand how their new article is reviewed. Right now when they get reviewed, the Wikipedia notifications bell on their user account lights up, where they get a message. I would like new users to get a more clear message, which more clearly indicates who reviewed their article, and which suggests to them to options for follow up.

Lane's own commentary

The Wikimedia Foundation sent technical staff to address these community concerns as a technical problem. In this case and in many others, the Wikimedia Foundation chooses to frame community concerns as technical problems which have solutions in software development. I think this is misguided, as people who using coding and software development are unprepared to talk about social and ethical issues, management of financial and labor resources, governance and power sharing between the Wikimedia Foundation and the Wikimedia community, or strategic priorities in the Wikimedia movement which could be assessed by interpreting user activities, surveys of community sentiment, or the Wikimedia community's demographic processes.

Here is the solution that I would propose for this: allocate budget to both Wikimedia developers who are specifically tasked with community requests (this, other community requests, and the wishlist) and then sponsor the Wikimedia community's own governance processes to convene community conversations in setting priorities and communicating them to the Wikimedia Foundation. I am not persuaded by Wikimedia Foundation claims that either there are no resources to address Wikimedia community concerns, or that this concern is not one that the Wikimedia Foundation should address. The request for New Pages Patrol tools is the largest and most coordinated request for development in the history of the Wikimedia Movement, and the Wikimedia Foundation is treating it with triviality. Dismissal is poor governance and lack of respect for democracy. If the resources are lacking, randomly de-funding any projects or randomly firing any staff of the Wikimedia Foundation is a reasonable solution when there are top-tier democratically selected concerns presented by the Wikimedia Community for years. There is no person or project at the Wikimedia Foundation with higher priority than this project, and if the Wikimedia Foundation has doubt about the democratic process which delivered this request, then randomly defund project and fire staff to fund better democratic processes. The community is asking for resources on the order of 0.1% of the annual budget and the Wikimedia Foundation is acting with disdain. This is not the correct reaction; donors trust the Wikimedia community and the Wikimedia community has a legitimate claim to 100% of the money in the Wikimedia Movement. The Wikimedia Foundation should treat this request with respect; it is modest and reasonable, and to dismiss it would be unsettling and unfair. Bluerasberry (talk) 16:20, 28 November 2022 (UTC)

Meeting #1 follow up questions

Hey @MMiller (WMF). Was great chatting with you earlier today. I had a couple follow up questions I was wondering if you can answer here. 1) Who will be assigned to review PageTriage patches? 2) Will this be a one time thing or ongoing? 3) Should we start tagging them in reviews? 4) Should we add them to mw:Git/Reviewers? 5) Should we look into calendaring the next meeting? Please share the date when you have it. Thanks a lot. Looking forward to your feedback. –Novem Linguae (talk) 07:56, 4 November 2022 (UTC)

Hello @Novem Linguae, @Kudpung, and @BusterD -- thank you so much for participating in last week's meeting and for your reflections and notes. I just wanted to let you know that I've seen them, and I'll be posting my own notes, links to the meeting recording, and next steps here a little later this week. Just taking a little time to talk to some of the engineers to figure out what can come next. Thank you! MMiller (WMF) (talk) 00:43, 8 November 2022 (UTC)

I'm not altogether happy about missing MB of the thanks list. I would just like to remind everyone that it was MB's initiative alone that made it possible to obtaining the Foundation's attention. Unfortunately, having listened/watched the video of the meeting a couple of times, I am not wholly enthusiastic that the WMF will be able or want to move this forward in the way the Wikipedia volunteers would prefer. It seems clear (to me at least) from the dialogue that funds are unlikely to be made available any time soon and that the Growth team will continue their other developments with little collaboration with the volunteers who run the processes that really matter. I'll bow out now as gracefully as possible and hand back to the NPP coordinators. Kudpung กุดผึ้ง (talk) 00:04, 13 November 2022 (UTC)

Hey @MMiller (WMF). Just a quick ping to follow up on this. It's been a week. A lot of stuff was undecided in our meeting, but I think the two actionable items are 1) picking someone to do code reviews and letting us know who it is, and 2) getting a second meeting on the calendar. Would be great to move forward with those so we have something to show for our meeting. Also full disclosure, I think someone is working on a Signpost article about our meeting, so would be nice to have something positive to report for that. Thank you. –Novem Linguae (talk) 18:55, 13 November 2022 (UTC)

@Novem Linguae -- thanks for the ping. I know this is taking me several days longer than I planned to post a reply, but it's because we're figuring out the resourcing for these things we agreed upon. Selena explained that it can be tricky to shift things around as all teams already have projects in flight, and that's exactly what I'm navigating right now. But progress is being made, and I'm hoping to post more tomorrow! MMiller (WMF) (talk) 17:34, 14 November 2022 (UTC)

Sounds good. Thanks for checking in. –Novem Linguae (talk) 18:34, 14 November 2022 (UTC)

Notes and next steps from the meeting on Nov 3

Hello everyone – thank you for your patience since our roundtable meeting on November 3.  It has taken us some time to upload the video, assemble our notes, and figure out how we can help going forward on the Wikimedia Foundation (WMF) end.  Our meeting included several WMF staff, including our new Chief Product and Technology Officer, Selena Deckelmann.  Volunteers attended from New Page Patrol, other parts of the English Wikipedia community, and from the Arabic, Hungarian, and Swedish Wikipedia communities.  Like several of you said, we agree that this was a great experience in talking openly about challenges, weighing different perspectives, and just generally understanding each other better than before.  Thank you all so much for participating and working with us to think things through!  WMF’s annual plans run from July - June, and we’re currently in the midst of this year’s plan. Having had this meeting and gone over the PageTriage needs with the NPP community, we are now well-equipped to consider those needs during the next annual planning process (planning happens in the first months of 2023 for a plan to start in July).  But during this fiscal year, we have several promising paths forward, a key part of which will be looking at PageTriage architecture to consider how it might be made easier to maintain and extend.

Materials

• Video from the meeting: Jitsi limited the length of this video, and it cuts off a little before the meeting actually concluded.
• Audio from the meeting: this was our backup recording, and begins around the place where the video cuts off.  After the meeting concluded, we continued some technical discussion, and this recording extends through those additional 20 minutes.
• Detailed notes: although several volunteers already posted notes above, I included my own detailed notes at the end of this post.  If you have anything to add or correct, please reply!

Decisions and next steps

• Organizing: we asked Novem Linguae to organize the PageTriage Phabricator board into priority order.  It looks like he has done that and reorganized the columns already – thank you!
• Code review: we decided that WMF engineers will be available for code review of volunteer patches to PageTriage on an ongoing basis.  The Growth team will be responsible for this code review.  New patches will automatically notify Growth engineers, but you can always tag kharlan, gtisza, and sgimeno.
• Longer term architecture: we will spend some time over the coming weeks looking into whether there are architectural opportunities to make PageTriage easier to maintain and extend in the future, and possibly available to other Wikipedias.  For instance, we have ideas around how volunteers might adjust and enhance the Page Curation Toolbar and New Pages Feed without requiring code changes.  Depending on what comes from this investigation, we may be able to make improvements during this fiscal year (July 2022 - June 2023).  WMF staff from the Moderator Tools team will be pursuing this line of thinking, and will be in touch with Novem about it.  The contact point for this will be Sam Walton.
• Newcomer experience: an important aspect of our meeting was discussion around how the experience of being a new article creator can be improved so that newcomers create better new articles in the first place. This would give them a more satisfying initial editing experience while also making the new page patrol work easier. Folks from NPP had ideas around onboarding and the Article Wizard which sound like they could be aligned with the roadmap of the Growth team. Kirsten Stoller from the Growth team will start a conversation about those ideas.
• Next meeting and updates: we decided together that we would like to have another meeting, hopefully before the end of the calendar year.  I will be on point for planning and scheduling that. We'll also be sure to post an update during the week of November 28 on how things are going so far.

We're getting started on these things now, and please reply with your thoughts and additional notes as we keep this conversation going.

Detailed notes from the meeting

These are notes as they were right after the meeting, and do not cover the decisions made afterward, which are listed above.

Expand to read the detailed notes

We started with introductions. Novem Linguae reviewed the open letter and the priorities of New Page Patrol. Novem and MPGuy2824 have solved many of the biggest bugs in the past months. Novem showed the PageTriage Phabricator board and highlighted some of the most important tasks to NPP (e.g. Draftify). A major priority is resources for code review.  When volunteers make patches, it would help tremendously if WMF engineers could help with code review so that the patches can be merged. New page patrollers are interested in thinking about the new article creation flow and the experiences newcomers have when they are first trying to create an article. Selena explained some of her thinking around PageTriage and New Page Patrol. We definitely want to address code review – this seems like a relatively simple way to help PageTriage improve.  We will have to figure out which engineers and teams can do this in an ongoing way. Because there is a long list of improvements to make to PageTriage, and because there will always be new good ideas in the future, we want to take a little time to consider a more systemic approach.  Rather than working on each issue one at a time, might there be ways to re-architect the extension so that it is easier to make changes in the future, or easier for volunteers to configure on their own without code changes?  For instance, perhaps new functions could be added to the Page Curation Toolbar without code changes. We’ve noticed that other tools, like Twinkle, do similar things as the Page Curation Toolbar.  Perhaps there are ways to increase the efficiency in both so that code and functionality can be shared between them. These sorts of architectural considerations are what WMF engineers are best-positioned to bring to the conversation. Volunteers talked about how page patrolling works on their wikis Samat from Hungarian Wikipedia brought up questions about Flagged Revisions, which has its own maintenance issues and how it’s confusing for newcomers. FShbib from Arabic Wikipedia said that they don’t have any tools for managing article review – they find them on Special:NewPages and tag them manually with templates and categories.  They have a backlog of thousands of pages, and he is interested in using PageTriage on Arabic Wikipedia. Samat and FShbib talked about sorting articles by topic to direct to reviewers who understand the topic (e.g. History, Music, Physics). Ainali from Swedish Wikipedia says they also have no tools for reviewing pages – they just use Special:NewPages and RecentChanges.  He pointed out that Movement Strategy Initiative 16 calls on us to “make tools easier to use in all the different Wikimedia Projects”.  He recommends that we make PageTriage wiki-agnostic before making further improvements to what it can do. We talked about making PageTriage wiki-agnostic. Novem said that although staff members had recommended a rewrite, there may be simpler ways to do it incrementally, such as by creating configuration variables to turn off functionality specific to English Wikipedia.  He has started working on this. The extension is already translatable in translatewiki.net. Novem recommends that we start by making the extension work on one additional wiki, iron out any issues there, and then proceed with another, and so on. Samat asked about our general policy for making things wiki-agnostic, and Marshall said that PageTriage is an exception, because WMF almost always builds things to be wiki-agnostic. We talked about whether English-specific improvements would have to be sequenced with going wiki-agnostic, or whether these things could happen in parallel. We talked about the experience of newcomers who submit articles to NPP. Sage from WikiEdu said that their students benefit from high-quality reviews from NPP, and they want that to continue. Sage said that the way the NPP process works and how to interact with it are very opaque and distressing for newcomers. For instance, “your page was patrolled” is a good thing, but it sounds scary.  It would be really good to make the process visible from the article view so the author can see it. Justice from Wikimedia Ghana said that he wants more people from his community to become reviewers so they can help. Kudpung explained that NPP wants to bring the creator into the process of their article being reviewed, and wants to communicate with them more clearly – even if with a talk page message instead of a notification.  There are newcomers who come with the express intention of creating a new article.  We don’t want to frighten them off – we want to help them.  And this will make page reviewing easier. Samat would be in favor of a clear button to create a new article, plus a wizard that can make sure you don’t duplicate an existing article. We concluded the meeting. Novem pointed out that the meeting did not focus specifically on solutions for PageTriage. Selena said that WMF can commit to addressing code review, and to investigating what kinds of investments WMF can make in PageTriage.  The issue around new user onboarding connects well with the existing roadmap from the Growth team, and that is a clear opportunity for further discussion. BusterD reminded us that volunteers participate when they’re happy.  Volunteers need to have a positive and fun service experience.  He wants to see more frequent calls like this to work together. We had an additional twenty minutes of technical conversation after the meeting. Tgr and Novem agreed that it is worth investigating how to make PageTriage configurable and reuse code from Twinkle. We decided to put the “Soon” column of the Phabricator board in priority order. We talked more about “Draftify” and other ideas. We talked about improving the Article Wizard to include thematic page templates, filling in infobox, etc.

-- MMiller (WMF) (talk) 06:01, 16 November 2022 (UTC)

Moderator tools team / investigation of codebase

Hi all - I’m the Product Manager for the Moderator Tools team. As Marshall noted above I’m spending some time digging into this further to better understand what the highest priority options might be for making improvements to PageTriage and NPP workflows.

Novem Linguae - since you've been working on the extension recently it would be helpful if I could get on a call with you and one of our engineers to learn more about the technical side of things. We’ve been chatting to some WMF engineers with experience of the codebase and I think it would really help to understand your perspective as we explore that side of the problem. If you’re happy to meet I’ll send you an email to organise a time (for next week, probably). I'll follow up with notes on our discussion for the benefit of everyone else if you're happy with that. Samwalton9 (WMF) (talk) 16:15, 17 November 2022 (UTC)

Hey @Samwalton9 (WMF). Sounds good. I'm happy to work out a meeting time. Feel free to email me. Thank you. –Novem Linguae (talk) 07:46, 18 November 2022 (UTC)

Thanks! For the benefit of discussion watchers - we're meeting next Tuesday (22nd), and I'll post notes on our discussion here afterwards. Samwalton9 (WMF) (talk) 11:04, 18 November 2022 (UTC)

Here are the meeting notes from my chat with Novem Linguae yesterday - Novem, feel free to edit these as necessary! Samwalton9 (WMF) (talk) 16:32, 23 November 2022 (UTC)

Sam Walton - Novem Linguae meeting notes (22 Nov)

I started with some background, explaining that our team (Moderator Tools) is spending some time looking into PageTriage, in terms of the tickets logged on the Phabricator board, doing a surface-level investigation of the codebase, and speaking with WMF engineers who worked on the extension in the past. We’re collecting this information so that we can have clearer internal discussions about priorities. One thing we’ve learned so far is that PageTriage is an old extension, using outdated technologies and designs, and this may result in a steep learning curve for current WMF Product teams which could make quick wins challenging. Novem shared some information about his history of contributing to the codebase, in addition to others like Twinkle. Since the NPP letter was sent he’s been able to submit 20 patches fixing high priority issues alongside other volunteer developers. There’s still a lot of room for improvement in the extension, but many of the most frustrating smaller issues have been resolved. The WMF Growth team have been added as reviewers for the codebase and have been providing helpful reviews and +2s. Novem hopes to gain +2 rights after submitting some more patches. We discussed the differences between PageTriage and Twinkle, and the code and functionality which is duplicated between them. Novem shared that PageTriage is harder to work with than Twinkle owing to it having both backend and frontend components, and being hosted on Gerrit rather than Github. We talked a little about whether these tools should have centralised functionality, which they could both pull from. Novem noted that PageTriage should ideally be a one-stop-shop for new page patrolling, without needing other tools or scripts. We also discussed how TwinkleStarter/TwinkleCore was intended to make it easier to setup Twinkle on other Wikimedia projects, but it doesn’t seem like it has been widely adopted, potentially because it remains technically challenging to configure. I asked for Novem’s opinion on which few tickets on the Phabricator board were the highest priority and best suited for a WMF Product team to pick up. Novem pointed to the ‘Priority big features’ tickets, specifically the top three: T124396, T207237, and T321179. We spoke about making the extension wiki-agnostic, and the work that Novem is doing to add a feature flag to turn off certain features on other wikis. This is actively being tested with some interested community members. I brought up the Growth team’s “Community Configuration” system for managing settings for the Newcomer Homepage, and we discussed how this might be useful for PageTriage. Novem agreed and has filed T323632 to explore this idea - the Growth team is in contact on that ticket. Novem asked what resources we were committing to the project. At the moment we’re just in a phase of exploring and better understanding the problem space, and I explained that all this information and the other research we’re doing is being collated to discuss in more detail.

Hey @Samwalton9 (WMF) and MMiller (WMF):. Thanks for organizing the meeting mentioned in this section. I was just wondering what you had in mind for next steps. Do we need to have another meeting, did we get some resources approved, etc? Thanks a lot. Looking forward to your feedback. –Novem Linguae (talk) 21:14, 29 November 2022 (UTC)

@Novem Linguae We're just reviewing and discussing the information that our team has been gathering - I'm hoping we can get an update for you on next steps before the end of the week. Samwalton9 (WMF) (talk) 16:46, 30 November 2022 (UTC)

1 December status update

Hi all, I have a short update from us on where we’re at with investigating PageTriage and support for NPP.

First, we’ve been supporting code review (huge thanks to KHarlan (WMF) for spending time on this) and it looks like more than 40 patches have been merged over the last couple of months. It seems like some of the most frustrating bugs have been fixed thanks to the hard work of Novem Linguae and MPGuy2824, including not being able to CSD redirects and AfD noms not working. I was also happy to see the support for Novem Linguae to receive +2 access! @Novem Linguae and MPGuy2824: Do you feel like you’re receiving sufficient support on patch reviews?

My team (Moderator Tools) have been looking into the PageTriage extension and the requests for improvement in more detail over the past few weeks. We’ve been speaking to engineers at WMF who worked on the extension in the past, and met with Novem Linguae (see notes above). This has really helped us understand the state of the codebase and the potential opportunities for taking a deliberate approach to improving it. One thing that has stood out to us is that both the technology (e.g. Javascript frameworks) and design patterns are very outdated compared to modern MediaWiki development standards. This makes it challenging for WMF Product teams to engage with and has led to some suggestions that the best way to support NPP/PageTriage in the long term, including bringing it to other Wikimedia projects, would be to make more fundamental changes to the software, possibly including a rewrite or major refactor (see, for example, these comments: 1, 2, 3). That’s obviously a much bigger project than making incremental improvements to the existing system, but might make the extension easier to support in the long term. We’re currently discussing what such a project might look like, and whether it’s a better option than small bug fixes and feature additions. I’d love to hear your thoughts on this.

Another topic we surfaced that I’d like to hear your input on is how we might increase the number of patrollers. Besides buggy software or missing features, what do you all see as the biggest barriers to engaging active editors in the NPP process? Have you seen feedback from editors stating the reasons they don’t participate, or stopped participating?

Because these discussions are still ongoing, unfortunately I don’t have an update on if, how, or when more resources might be allocated to this project, but I wanted to let you know that we’re taking it seriously and spending significant time on this on top of our planned work. We’d like to schedule a follow-up call this month to discuss this all in more detail and hopefully provide a more concrete update. Look out for details of that by the end of next week, and please let me know if you have any questions. Samwalton9 (WMF) (talk) 12:45, 1 December 2022 (UTC)

Hey Sam. Thanks for the update.

Kosta Harlan coming onboard has been great. He's taken the burden off of volunteer +2ers, his code review comments are insightful and are teaching me a lot about the codebase, and he has been able to use his staff access to mark old patches as abandoned (helping to clean the Gerrit queue). Big thanks to WMF, the Growth Team, and Kosta for this.

Perhaps @MB and @Kudpung can better answer your question about patroller recruitment and retention.

The follow up call sounds good. Looking forward to the details on that.

I have major reservations about a PageTriage rewrite, and I lay out some of my reasons at User:Novem Linguae/Essays/Rewriting PageTriage is a bad idea. However if you want to put together a proposal or discussion with some specifics, I think that'd be a good next step, so we can start talking about details. Let's try to get away from nebulous "it needs a rewrite", and start getting more specific, for example, "WMF wants to replace Backbone.js and Underscore.js with mw:Codex, and we think it'd take 100 hours, and the Moderator Tools Team would be willing to work on this in Mar–Apr 2023". We can have this discussion in phab:T318522 if you'd like. I am open to being convinced, and I think that even if we don't agree on a complete rewrite, that we could agree on some incremental steps and some Phab refactoring tickets. Perhaps a hybrid approach of refactoring plus some features would be a good option, that way we can deliver some features to new page patrollers while also improving the codebase.

Would your team be willing to share some of their specifics about the technical debt they've found, and how they propose we fix it? phab:T318522. Thank you. –Novem Linguae (talk) 13:18, 1 December 2022 (UTC)

@Novem Linguae Thanks for this - Rewriting PageTriage is a bad idea is a helpful read. Just to quickly clarify - we're definitely moreso exploring refactoring than rewriting, for many of the reasons you describe there. I'll get back to you on the specifics, or perhaps that's something worth discussing at the next meeting - our engineers can attend and speak to this directly rather than through me :) Samwalton9 (WMF) (talk) 13:26, 1 December 2022 (UTC)

Yes, Kosta has gone on a tear through our patches which were waiting for a +2. Now, we are reasonably sure that patches will be reviewed within 24 hours, and this does help with motivation. Thanks. -MPGuy2824 (talk) 13:45, 1 December 2022 (UTC)

• @Samwalton9 (WMF): I can speak to the social question above - "what do you all see as the biggest barriers to engaging active editors in the NPP process? Have you seen feedback from editors stating the reasons they don’t participate, or stopped participating?"

The answer to that question is that there is no such engagement plan in place for editor recruitment for NPP or any administrative role. The plan that I want in place is for the WMF to fund this on the order of US$100s of k annually through wiki community organizations. The most obvious path to engaging people for NPP and other admin work is investment in at least 100s, and probably 1000s, of labor hours to improve administration. Volunteers are not planning to do that at this time. I can speak to options for administrative training in meta:WALRUS at meta:WALRUS/December 2022, which has the advantage of reaching wiki trainers in the United States. It is unknown to me whether the WMF has a strategy for growing wiki administration, but if you know of one, then please share. To the extent that the WMF can share insight into "biggest barriers to engaging active editors in" wiki administration, I can organize conversation about connecting the United States wiki community into broader plans. Bluerasberry (talk) 17:34, 2 December 2022 (UTC)

Thanks @Bluerasberry - in terms of WMF strategy, I can only speak to my team's goals but that is one of the things we're interested in. Our current focus on improving mobile web is driven in part by an observation that we have a substantial and growing number of mobile editors, who are largely unable to engage in project administration tasks (from functionality as simple as undoing edits up to using tools like PageTriage or performing admin actions). One of our goals is to increase the number of mobile editors who can engage in content moderation actions. That is, of course, just one aspect of this problem area, and in the context of NPP I'm particularly interested what drives folks towards or away from the process of patrolling new pages. Samwalton9 (WMF) (talk) 14:59, 5 December 2022 (UTC)

Reviewer recruitment and retention

@Samwalton9, Bluerasberry, MPGuy2824, Novem Linguae, and MB: this is a critical issue and I'm not sure the WMF can help, though any ideas would be greatly appreciated. We don't get feedback from editors stating the reasons they don’t participate, or stopped participating. In short, we don't know why there are so few active reviewers. In 2016 I proposed and obtained the creation of a user right for reviewing new pages in order to introduce some quality into the process from experienced users. To ensure a seamless transition, some editors were grandfathered into NPP based on a set of criteria. The threshold for applying for the user right was set deliberately low so as to assure passage of the new user right. In practice, due to the required knowledge level (similar to that of an admin) the criteria for obtaining the right are necessarily interpreted by admins at WP:PERM as being much higher. The quality of reviewing improved dramatically but inevitably led to backlogs. Nevertheless, as with all additional user rights, it attracted many hat collectors who have rarely, if ever, used the tools and there are about 750 listed New Page Reviewers (which should now be radically pruned). However, fewer than around 30 are truly active and less than a dozen do 80% of the work. Because it works in the background, few people really know what NPP is. NPPers have submitted presentations for Wikimania over the years since 2012, but the WMF Wikimania committees always denied them a voice. The NPP tools (when they are working properly) have been designed to be as intuitive and easy to use as possible, but the actual lack of participation is probably due to a combination of:

• Having bitten off more than they can chew; i.e. insufficient knowledge of policies. They are probably too proud to enroll at the NPP school I created in 2012
• Extremely boring and/or low quality articles of dubious notability and pure spam. i.e. soul-destroying work
• Being expected by some factions of the community to clean up articles which is not in the reviewers' remit
• The constant stream of impolite backlash on their talk pages from users whose articles have been tagged, moved to draft, or listed for deletion.
• Constantly receiving newsletters asking them to do more.
• Lack of support during a 3-year hiatus of NPP coordination after I announced my many years of de facto shepherd of it. Since May this year we now have a new coordination team.
• NPP is arguably the most important single process on Wikipedia. The reviewers receive no formal recognition for their work, and they deserve more. All they get are complaints and threats of sanctions if they make one minor error of judgement among the 1,000s of patrols they make.

Kudpung กุดผึ้ง (talk) 05:36, 7 December 2022 (UTC)

@Kudpung Thanks for this information and history, very helpful. With regards to the user right - do you know of any analysis of reviewer numbers that might show, for example, how many people have received the user right since it was granted, how many of them remain active, when active reviewers gained their user right, etc.? This is all data we should be able to investigate ourselves but if it's already been looked at I figured you'd know where any of that analysis is!

The NPP School is interesting, I hadn't seen this before. @Barkeep49, @Cassiopeia, @Atsme I wonder if you have any perspectives you could share, as active NPP school trainers. What reasons are you seeing for editors either not wanting to do new page patrol or not 'graduating' from the training? Samwalton9 (WMF) (talk) 13:58, 8 December 2022 (UTC)

@Samwalton9: the data you mentioned was once requested earlier this year but it may not have been reliable. It did however confirm that a very large number had not made any reviews in the 12 month sample. There are currently 726 holders of the user right. The list starts here. It would be interesting to find out:

• How many have never made any reviews
• How many users have not made any reviews since 1 year after obtaining the right
• How many have not many any reviews in the last 12 months
• How many have not edited Wikipedia in the last 12 months
• A list of all reviewers by total of reviews.

There are probably other stats that are useful but this would be a start. Kudpung กุดผึ้ง (talk) 14:27, 8 December 2022 (UTC)

So I would love to have more reviewers and during my stint as coordinator training and retention of reviewers was my biggest focus (along with attempts to simplify the NPP tutorial). I think Kudpung hits on a lot of valuable points above and I don't have a lot to add to it. So I'm going to note a slightly different angle not because I disagree with what's above but because I think in some ways I wonder if it's the right question. While Wikipedia allows for "many hands make light work" scenarios, I think it's more typical that a small number of editors - sometimes 1 - do the lion share of work for any major piece of wikiwork. For instance it's often a single editor that does the largest degree of work to take an article to FA. Other wikiproceses see similar dynamics to NPP where there might be a single "workhorse" editor who does a lot of the work supported by a few other dedicated volunteers doing some work, and a much larger set of editors doing very small amounts of work. I also think that enwiki, on the whole, is increasingly seeing a capacity crunch around editors capable and willing to do certain kinds of non-content work like NPP so NPP is fighting for capacity among a decreasing supply. So as you think about the work your team can do, I would love a focus on anything you can do to address what Kudpung listed out (though there are limits as some of that are community dynamics that are for volunteers to solve among ourselves) but also to make the pool of reviewers we do have more efficient so that with the same amount of time they could review more articles. Best, Barkeep49 (talk) 15:46, 8 December 2022 (UTC)

Hi, Sam! Good to hear from you. My perspective about NPP:School is quite simply that it is not highly publicized, users can obtain the right without schooling, and while I cannot speak for the other trainers, I have not had any issues attracting students to my course in numbers that I can handle, or graduating them once they've begun. I've had an admin take my course, a graduate who became an admin, and graduates who have worked on the backlogs with incredible dedication, one of whom won the award for the NPP backlog drive back in August, and expressed his appreciation in this diff, which may help give you some insight as to what we need to do to teach, encourage, inspire NPP reviewers. I am also aware of the frustration and burnout that results because NPP has no "teeth" even though trained reviewers know WP:PAG inside and out. What ends up happening is that the articles they draftify or CSD or send to AfD are met with resistance by some admins and users who are critical of those processes as they relate to new articles. They probably do not take into consideration the many decade old improvement tags in the headers of unacceptable articles that should have been deleted. No telling how many of those tags have been removed that no one noticed, and that doesn't even start the thefts and reverts of redirects. I would imagine COI and UPE editors love that gaping hole in the process. The dichotomy between NPP and the inclusionists is a vast one, and it does not serve to help our recruiting efforts. In my view, it adds to the frustration and feeling of working one's ass off in a thankless job. How long would you put up with it? Yes, yes, yes - I understand all the yada yada about consensus, but I also understand the hegemony of the asshole consensus, and the two are in a rope pull contest. Atsme 💬 📧 14:58, 8 December 2022 (UTC)

15 December update

Hi all - I have a quick update from the Moderator Tools team on our status with respect to PageTriage. I wasn’t able to find a good time to organise another call this week, though I heard good things about the discussion the Growth team hosted about article creation workflows last week! I think that's a super important piece of the puzzle.

As I noted in my last update, our team has been speaking with WMF engineers, reading documentation about past work on PageTriage, and investigating the codebase, to build a picture of what maintaining or building on PageTriage would look like for us. I also just want to highlight that some WMF engineers are currently participating in a Community Tech Wishathon where some PageTriage tickets are being worked on informally. Kosta has submitted a bunch of patches this week!

We’re close to making a decision about if and how the WMF might be able to provide more support for PageTriage, but aren’t quite ready to finalise any plans. The end of the year is fast approaching and most WMF staff will be taking a week or two holiday after this week. We’ll be able to share more about our plans in January, when we should have something much more concrete to share. Apologies that I don’t have anything more substantial right now, I know it's frustrating, but I want to reassure you we’re taking this seriously and having meaningful discussions about what our options are. Samwalton9 (WMF) (talk) 15:13, 15 December 2022 (UTC)

@Samwalton9 (WMF) apologies if this has been more clearly communicated either in a call or onwiki, but should NPP be planning on submitting to the wishlist as a backup? Best, Barkeep49 (talk) 15:56, 15 December 2022 (UTC)

@Barkeep49, we have made progress in having patches, written by volunteers (mostly NL & MPGuy), reviewed by WMF developers so they can be released and that has help greatly in resolving bug fixes. Most of the bigger enhancements are beyond the scope of the Wishlist, so we are not planning on utilizing that. MB 19:41, 15 December 2022 (UTC)

@Barkeep49 In my opinion I think it could make sense to plan to write a wish. I feel fairly confident that we'll have more concrete information to share before the proposal phase starts (January 23) so then you can make a decision about whether to go ahead with a proposal or not. Samwalton9 (WMF) (talk) 10:58, 16 December 2022 (UTC)

• @Samwalton9, Bluerasberry, MPGuy2824, Novem Linguae, MB, and Barkeep49: and also @KStoller-WMF, MMiller (WMF), and DannyH (WMF): The confusion is that we are getting mixed messages from the same departments within the WMF. At the same time as being encouraged to list our requirements at the wishlist, we've been told several times that Community Tech neither has the bandwidth nor the funds to address the scope what we would be putting forward at the wishlist. Also, these are not new requests - they already exist as Phab tickets. Furthermore, as we have a potential of 444 editors who would support our requests, we would win in numbers and that would not be fair to the requests from other volunteer communities for their required gadgets and enhancements. Our discussions to date have explained to Ms Deckelmann that NPP software needs a dedicated team, which she has understood. Hence, unless your more concrete information includes an increase in Community Tech's resources, it's unlikely that the NPP volunteers would wish to dedicate time to drawing up the complex request, and and inviting the disappointment of others when the en.Wiki will once again have commandeered the capacity.

In having taken a holistic approach that will relieve the workload at NPP by enhancing the tools and by offering more guidance to new users who want to create articles, we have begun discussions with Kirsten's Growth Team at Newcomer experience + new article creation and have had a very successful first meeting with another one scheduled for next month to discuss landing pages and article wizards. However, while there may be some inter-departmental overlap, the issues concerning PageTriage and the onboarding of new users are fundamentally different. The PageTriage issues are ones of code which is the speciality of the Foundation's software engineers, whereas newcomer UX is mainly conceptual and based on the years of anecdotal evidence of the New Page Reviewers and some recent WMF research, but for which the solutions will require engineering. Discussing the possible solutions will probably take us up to the new financial year when resources will need to be allocated to coding what we decide on from exchanging ideas and wire frames with Growth.

At the moment, I believe Novem Linguae and MPGuy2824 are looking forward to the next promised meeting before the end of this year for addressing the pressing PageTriage issues. Kudpung กุดผึ้ง (talk) 07:56, 18 December 2022 (UTC)

Hi Kudpung: (This is DannyH responding with my volunteer account, I'm having login problems.) As Sam said, we had some unexpected absence/delays around the holidays that got in the way of having an agreed-on proposal for working on PageTriage. I apologize for the delay; I know you've been eagerly waiting for an answer. We'll get back to you in early January for sure, when we'll post our proposal on-wiki, and we'll talk on-wiki and in a call then.

As for the wishlist question: Barkeep49 asked if NPP should write a proposal for the next survey, and Sam said maybe. The wishlist survey is open to all active contributors to submit proposals, and I think we've been consistent in encouraging everyone to participate. But as Sam pointed out, you'll be hearing from us well before the wishlist survey opens on Jan 23rd, and you can make a decision about whether you want to participate or not. -- DannyH (WMF) via Toughpigs (talk) 18:52, 20 December 2022 (UTC)

Growth team / newcomer experience

Hello, I'm the Product Manager for the Growth team. To continue the Newcomer experience discussion as it relates to article creation and NPP, I've started a discussion here: Newcomer experience + new article creation.

Please join that discussion if you are interested. Thanks! KStoller-WMF (talk) 18:57, 17 November 2022 (UTC)

Thank you TheresNoTime

I'd like to give a big thank you to @TheresNoTime, who donated their time as a volunteer to review, test, and +2 dozens of our volunteer patches. Without them coming onboard, volunteer patches during the period of Jul–Oct 2022 would have been completely stuck, and some key volunteers might have lost interest early on.

Working on an unmaintained extension as a volunteer is always a risky business due to the difficulty of securing +2s. TheresNoTime didn't just agree to review a couple random patches, but became our main patch reviewer, adding themself to mw:Git/Reviewers, which was absolutely invaluable.

I just want to make sure TheresNoTime gets the proper credit. They are so humble and did a lot of this work behind the scenes, and this is our chance to let them know we appreciate their efforts. So thank you very much TheresNoTime. We really appreciate all you do :) –Novem Linguae (talk) 20:45, 5 December 2022 (UTC)

That's very kind of you, thank you I hope it helped get things moving — TheresNoTime (talk • they/them) 22:50, 5 December 2022 (UTC)

All the more laudable, given your lack of time. ;-) -MPGuy2824 (talk) 03:00, 7 December 2022 (UTC)

I would also like to add my thanks to TheresNoTime and of course to Novem Linguae for all the unpaid work developing all the patches. Kudpung กุดผึ้ง (talk) 01:24, 7 December 2022 (UTC)

+1 more here!! Thank you! Atsme 💬 📧 14:59, 8 December 2022 (UTC)

2023 Moderator Tools project

Hi all, and happy new year! I have (hopefully!) good news - our team is planning to work on PageTriage in April - June 2023. I’ve posted further information at Wikipedia:Page Curation/2023 Moderator Tools project, but the short version is that we’re going to work on a focused project to bring the extension to a place where we’re more comfortable maintaining it in the long-term, while doing research to evaluate where we should go afterwards. Please feel free to respond here if you have any thoughts or questions I can answer, though it might be best to consolidate the conversation on the project page. Samwalton9 (WMF) (talk) 09:34, 6 January 2023 (UTC)

I'd be happy to schedule a call to discuss this project in the next couple of weeks if folks would be interested, or I'm happy to just discuss on talk pages for the time being - let me know. Samwalton9 (WMF) (talk) 10:19, 6 January 2023 (UTC)

Thanks for taking this on. Talk pages sound fine for now until we get closer, unless your team has questions or wants to have a call earlier. Looking forward to collaborating. –Novem Linguae (talk) 10:36, 6 January 2023 (UTC)

WMF Moderator Tools team update

Hi - I just wanted to drop a note here to let you know that we've just published some research findings from interviews with new page patrollers in advance of our work on PageTriage prompted by this letter. We'd love to know what you think! Samwalton9 (WMF) (talk) 14:27, 27 February 2023 (UTC)

April WMF update

Our team's work on PageTriage is now underway, and we've posted a new update summarising what we're working on. Samwalton9 (WMF) (talk) 19:20, 17 April 2023 (UTC)