Page MenuHomePhabricator
Create Task
Maniphest T349836

Simplify Wikifunctions e2e test cases
Closed, ResolvedPublic
Actions

Description

Currently the e2e tests are (a) tightly coupled to Codex implementation and (b) written similar to unit tests (i.e. testing edge cases), we'd like to simplify and make the tests more robust by defining a "happy path" to test and leveraging simpler selectors for codex components.

Happy path:

  1. Create a function
  2. Add an code implementation
  3. Add a tester
  4. Add a composition implementation
  5. Run the function
  6. Enable the function
  • basic.js
  • function.js
  • tester.js
  • implementation.js
  • connect.js
  • type.js — Probably just delete these, not reasonably tested.

Details

SubjectRepoBranchLines +/-
selenium: Fixing broken and flaky tests in connect.jsmediawiki/extensions/WikiLambdamaster+73 -349
WIP selenium: Adding dynamic waits for checkbox and button statesmediawiki/extensions/WikiLambdamaster+26 -44
selenium: Fix for boolean dropdown selectormediawiki/extensions/WikiLambdamaster+5 -2
selenium: Adding data-testid selectors for e2e testingmediawiki/extensions/WikiLambdamaster+19 -22
selenium: Adding additional fixes for implementation.jsmediawiki/extensions/WikiLambdamaster+12 -10
selenium: Removing type.js from selenium test suitemediawiki/extensions/WikiLambdamaster+0 -762
selenium: Adding implementation.js into selenium-dailymediawiki/extensions/WikiLambdamaster+11 -7
selenium: Fixing implementation.js and related filesmediawiki/extensions/WikiLambdamaster+176 -308
selenium: Changing xpath selector in function.jsmediawiki/extensions/WikiLambdamaster+2 -2
WIP selenium: Editing tester.js related files to pass in DUCTmediawiki/extensions/WikiLambdamaster+38 -7
selenium: Changing xpath selector and clearing cookiesmediawiki/extensions/WikiLambdamaster+14 -16
selenium: Adding data-testid selectors into tester.js related filesmediawiki/extensions/WikiLambdamaster+24 -52
selenium: Adding tester.js back into selenium daily testsmediawiki/extensions/WikiLambdamaster+1 -1
selenium: Adding data-testid selectors into tester.jsmediawiki/extensions/WikiLambdamaster+26 -18
selenium: Fixing and refactoring tester.jsmediawiki/extensions/WikiLambdamaster+82 -182
selenium: Updating README and updating to dynamic selectormediawiki/extensions/WikiLambdamaster+8 -6
selenium: Adding function.js into selenium-dailymediawiki/extensions/WikiLambdamaster+1 -1
selenium: Adding localization strings for more dynamic testingmediawiki/extensions/WikiLambdamaster+13 -4
selenium: Adding some test IDs and fixing local broken tests for function.jsmediawiki/extensions/WikiLambdamaster+29 -54
e2e: Run only installation checks in the daily Jenkins jobmediawiki/extensions/WikiLambdamaster+1 -1
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
vaughnwalters subscribed.Oct 26 2023, 4:00 PM

https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/WikiLambda/+/refs/heads/master
https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/WikiLambda/+/refs/heads/master#quick-start
https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/WikiLambda/+/refs/heads/master#quickstart

zeljkofilipin moved this task from Backlog 🪒 to Deep work 🌊 on the User-zeljkofilipin board.Oct 26 2023, 4:10 PM
Jdforrester-WMF subscribed.Oct 30 2023, 1:47 PM

Why did this task get created in a legacy todo column? Should it go through the team's regular triage process? Sorry if I missed something!

SDunlap moved this task from To triage: Un-phased work to To triage on the Abstract Wikipedia team board.Oct 30 2023, 3:18 PM

Sorry, that was my bad. I wasn't aware that was a legacy column. I'll move it to triage. FWIW this will primarily be worked on by Vaughn with assistance from Zeljko and I.

Jdforrester-WMF added projects: Abstract Wikipedia Fix-It tasks, WikiLambda.Oct 31 2023, 6:18 PM
Jdforrester-WMF triaged this task as Medium priority.Nov 16 2023, 5:38 PM
Jdforrester-WMF moved this task from To triage to Backlog on the Abstract Wikipedia team board.
zeljkofilipin moved this task from Deep work 🌊 to Waiting 🏹 on the User-zeljkofilipin board.Nov 17 2023, 4:14 PM
zeljkofilipin moved this task from Waiting 🏹 to Deep work 🌊 on the User-zeljkofilipin board.Jan 5 2024, 4:35 PM
vaughnwalters moved this task from Backlog to In Progress on the User-vaughnwalters board.Jan 9 2024, 9:54 PM
vaughnwalters added a subtask: T354710: Basic.js test causing rendering of chrome page to crash on betacluster.Jan 9 2024, 10:42 PM
gerritbot added a comment.Jan 10 2024, 11:19 AM

Change 989464 had a related patch set uploaded (by Zfilipin; author: Zfilipin):

[mediawiki/extensions/WikiLambda@master] e2e: Run only installation checks in the daily Jenkins job

https://gerrit.wikimedia.org/r/989464

gerritbot added a project: Patch-For-Review.Jan 10 2024, 11:19 AM
gerritbot added a comment.Jan 10 2024, 11:32 AM

Change 989464 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] e2e: Run only installation checks in the daily Jenkins job

https://gerrit.wikimedia.org/r/989464

Maintenance_bot removed a project: Patch-For-Review.Jan 10 2024, 12:30 PM
ReleaseTaggerBot added a project: MW-1.42-notes (1.42.0-wmf.14; 2024-01-16).Jan 10 2024, 1:00 PM
Peter subscribed.Jan 10 2024, 2:09 PM
• Mcastro changed the status of subtask T354710: Basic.js test causing rendering of chrome page to crash on betacluster from Open to In Progress.Jan 11 2024, 5:37 PM
Jdforrester-WMF changed the task status from Open to In Progress.Jan 11 2024, 9:05 PM
Jdforrester-WMF assigned this task to vaughnwalters.
Jdforrester-WMF moved this task from Backlog to In Progress on the Abstract Wikipedia team board.
zeljkofilipin subscribed.Jan 12 2024, 10:09 AM
In T349836#9449813, @gerritbot wrote:

https://gerrit.wikimedia.org/r/989464

I thought this didn't help, but looks like there were two passing test runs (in a row!). That didn't happen for at least the last 30 days, if not longer.

Screenshot 2024-01-12 at 11.08.44.png (214×504 px, 27 KB)

I still don't know why a lot of the time the job times out after 60 minutes (for example today), but at least there's some progress.

zeljkofilipin moved this task from Deep work 🌊 to Q2 👔 on the User-zeljkofilipin board.Jan 29 2024, 2:42 PM
zeljkofilipin closed subtask T354710: Basic.js test causing rendering of chrome page to crash on betacluster as Resolved.Feb 5 2024, 2:48 PM
zeljkofilipin moved this task from Q2 👔 to Backlog 🪒 on the User-zeljkofilipin board.Feb 19 2024, 5:30 PM
vaughnwalters updated the task description. (Show Details)Mar 7 2024, 2:37 PM
gerritbot added a comment.Apr 1 2024, 5:29 PM

Change #1015458 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] Bug: T349836

https://gerrit.wikimedia.org/r/1015458

gerritbot added a project: Patch-For-Review.Apr 1 2024, 5:29 PM
gerritbot added a comment.Apr 1 2024, 5:32 PM

Change #1015458 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Adding some test IDs and fixing local broken tests for function.js

https://gerrit.wikimedia.org/r/1015458

gerritbot added a comment.Apr 2 2024, 2:19 PM

Change #1015464 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] Bug: T349836

https://gerrit.wikimedia.org/r/1015464

gerritbot added a comment.Apr 2 2024, 2:28 PM

Change #1015464 abandoned by Vaughn Walters:

[mediawiki/extensions/WikiLambda@master] Bug: T349836

Reason:

duplicate

https://gerrit.wikimedia.org/r/1015464

gerritbot added a comment.Apr 4 2024, 9:18 PM

Change #1015458 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Adding some test IDs and fixing local broken tests for function.js

https://gerrit.wikimedia.org/r/1015458

ReleaseTaggerBot edited projects, added MW-1.42-notes (1.42.0-wmf.26; 2024-04-09); removed MW-1.42-notes (1.42.0-wmf.14; 2024-01-16).Apr 4 2024, 10:00 PM
gerritbot added a comment.Apr 5 2024, 4:32 PM

Change #1017083 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Adding localization strings for more dynamic testing

https://gerrit.wikimedia.org/r/1017083

gerritbot added a comment.Apr 5 2024, 7:31 PM

Change #1017083 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Adding localization strings for more dynamic testing

https://gerrit.wikimedia.org/r/1017083

gerritbot added a comment.Apr 5 2024, 9:43 PM

Change #1017085 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Adding function.js back into selenium-test

https://gerrit.wikimedia.org/r/1017085

gerritbot added a comment.Apr 8 2024, 3:37 PM

Change #1017085 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Adding function.js into selenium-daily

https://gerrit.wikimedia.org/r/1017085

Jdforrester-WMF moved this task from In Progress to 24Q4 (Apr–Jun) on the Abstract Wikipedia team board.Apr 29 2024, 5:56 PM
Jdforrester-WMF edited projects, added Abstract Wikipedia team (24Q4 (Apr–Jun)); removed Abstract Wikipedia team.
Jdforrester-WMF moved this task from Incoming to In Progress on the Abstract Wikipedia team (24Q4 (Apr–Jun)) board.
Jdforrester-WMF added a parent task: T363389: [24Q4] Explore more integration with MW core tests .Apr 29 2024, 6:33 PM
gerritbot added a comment.May 1 2024, 6:16 PM

Change #1025909 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Updating README and updating to dynamic selector

https://gerrit.wikimedia.org/r/1025909

gerritbot added a comment.May 7 2024, 3:03 PM

Change #1025909 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Updating README and updating to dynamic selector

https://gerrit.wikimedia.org/r/1025909

ReleaseTaggerBot added a project: MW-1.43-notes (1.43.0-wmf.5; 2024-05-14).May 7 2024, 4:00 PM
gerritbot added a comment.May 8 2024, 9:39 PM

Change #1028933 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Fixing and refactoring tester.js

https://gerrit.wikimedia.org/r/1028933

gerritbot added a comment.May 13 2024, 1:09 PM

Change #1028933 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Fixing and refactoring tester.js

https://gerrit.wikimedia.org/r/1028933

Jdforrester-WMF updated the task description. (Show Details)May 13 2024, 1:15 PM
vaughnwalters updated the task description. (Show Details)May 13 2024, 4:04 PM
vaughnwalters updated the task description. (Show Details)
gerritbot added a comment.May 14 2024, 6:39 PM

Change #1030562 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Adding tester.js back into selenium daily tests

https://gerrit.wikimedia.org/r/1030562

gerritbot added a comment.May 14 2024, 9:37 PM

Change #1031586 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Adding data-testid selectors into tester.js

https://gerrit.wikimedia.org/r/1031586

Jdforrester-WMF updated the task description. (Show Details)May 15 2024, 5:15 PM
Jdforrester-WMF mentioned this in T363389: [24Q4] Explore more integration with MW core tests .
gerritbot added a comment.May 15 2024, 5:44 PM

Change #1030562 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Adding tester.js back into selenium daily tests

https://gerrit.wikimedia.org/r/1030562

gerritbot added a comment.May 15 2024, 5:46 PM

Change #1031586 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Adding data-testid selectors into tester.js

https://gerrit.wikimedia.org/r/1031586

ReleaseTaggerBot edited projects, added MW-1.43-notes (1.43.0-wmf.6; 2024-05-21); removed MW-1.43-notes (1.43.0-wmf.5; 2024-05-14).May 15 2024, 6:00 PM
gerritbot added a comment.May 16 2024, 12:20 AM

Change #1031597 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Adding data-testid selectors into tester.js related files

https://gerrit.wikimedia.org/r/1031597

gerritbot added a comment.May 17 2024, 9:29 PM

Change #1032739 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Editing tester.js related files to pass in DUCT

https://gerrit.wikimedia.org/r/1032739

gerritbot added a comment.May 20 2024, 2:29 PM

Change #1031597 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Adding data-testid selectors into tester.js related files

https://gerrit.wikimedia.org/r/1031597

gerritbot added a comment.Jun 6 2024, 7:40 AM

Change #1039586 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Changing xpath selector and clearing cookies

https://gerrit.wikimedia.org/r/1039586

gerritbot added a comment.Jun 6 2024, 4:46 PM

Change #1039586 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Changing xpath selector and clearing cookies

https://gerrit.wikimedia.org/r/1039586

ReleaseTaggerBot edited projects, added MW-1.43-notes (1.43.0-wmf.9; 2024-06-11); removed MW-1.43-notes (1.43.0-wmf.6; 2024-05-21).Jun 6 2024, 5:00 PM
gerritbot added a comment.Jun 7 2024, 7:37 PM

Change #1032739 abandoned by Vaughn Walters:

[mediawiki/extensions/WikiLambda@master] WIP selenium: Editing tester.js related files to pass in DUCT

Reason:

Done with testing this for now

https://gerrit.wikimedia.org/r/1032739

gerritbot added a comment.Jun 10 2024, 7:09 PM

Change #1041199 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Fixing implementation.js and related files

https://gerrit.wikimedia.org/r/1041199

gerritbot added a comment.Jun 12 2024, 7:19 AM

Change #1042059 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Changing xpath selector in function.js

https://gerrit.wikimedia.org/r/1042059

gerritbot added a comment.Jun 12 2024, 4:25 PM

Change #1042059 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Changing xpath selector in function.js

https://gerrit.wikimedia.org/r/1042059

ReleaseTaggerBot edited projects, added MW-1.43-notes (1.43.0-wmf.10; 2024-06-18); removed MW-1.43-notes (1.43.0-wmf.9; 2024-06-11).Jun 12 2024, 5:07 PM
Jdforrester-WMF updated the task description. (Show Details)Jun 12 2024, 5:09 PM
gerritbot added a comment.Jun 12 2024, 6:03 PM

Change #1041199 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Fixing implementation.js and related files

https://gerrit.wikimedia.org/r/1041199

gerritbot added a comment.Jun 12 2024, 9:30 PM

Change #1042380 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Adding implementation.js into selenium-daily

https://gerrit.wikimedia.org/r/1042380

gerritbot added a comment.Jun 13 2024, 11:59 AM

Change #1042380 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Adding implementation.js into selenium-daily

https://gerrit.wikimedia.org/r/1042380

gerritbot added a comment.Jun 13 2024, 6:54 PM

Change #1043191 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Removing type.js from selenium test suite

https://gerrit.wikimedia.org/r/1043191

gerritbot added a comment.Jun 13 2024, 9:07 PM

Change #1043235 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Adding additional fixes for implementation.js

https://gerrit.wikimedia.org/r/1043235

vaughnwalters updated the task description. (Show Details)Jun 14 2024, 3:46 AM
gerritbot added a comment.Jun 14 2024, 2:32 PM

Change #1043191 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Removing type.js from selenium test suite

https://gerrit.wikimedia.org/r/1043191

gerritbot added a comment.Jun 14 2024, 2:32 PM

Change #1043235 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Adding additional fixes for implementation.js

https://gerrit.wikimedia.org/r/1043235

gerritbot added a comment.Jun 14 2024, 10:20 PM

Change #1043902 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Adding data-test id selectors

https://gerrit.wikimedia.org/r/1043902

gerritbot added a comment.Jun 17 2024, 1:54 PM

Change #1043902 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Adding data-testid selectors for e2e testing

https://gerrit.wikimedia.org/r/1043902

gerritbot added a comment.Jun 17 2024, 7:22 PM

Change #1046759 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Fix for boolean dropdown selector

https://gerrit.wikimedia.org/r/1046759

gerritbot added a comment.Jun 17 2024, 7:55 PM

Change #1046759 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Fix for boolean dropdown selector

https://gerrit.wikimedia.org/r/1046759

gerritbot added a comment.Jun 25 2024, 3:37 PM

Change #1049589 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] selenium: Fixing broken and flaky tests in connect.js

https://gerrit.wikimedia.org/r/1049589

Jdforrester-WMF moved this task from 24Q4 (Apr–Jun) to 25Q1 (Jul–Sep) on the Abstract Wikipedia team board.Jul 3 2024, 4:25 PM
Jdforrester-WMF edited projects, added Abstract Wikipedia team (25Q1 (Jul–Sep)); removed Abstract Wikipedia team (24Q4 (Apr–Jun)).
Jdforrester-WMF moved this task from Incoming to Essential work on the Abstract Wikipedia team (25Q1 (Jul–Sep)) board.
gerritbot added a comment.Jul 3 2024, 7:05 PM

Change #1051825 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/WikiLambda@master] WIP selenium: Adding dynamic waits for checkbox and button states

https://gerrit.wikimedia.org/r/1051825

gerritbot added a comment.Jul 3 2024, 8:19 PM

Change #1051825 abandoned by Vaughn Walters:

[mediawiki/extensions/WikiLambda@master] WIP selenium: Adding dynamic waits for checkbox and button states

Reason:

including this code in another patch

https://gerrit.wikimedia.org/r/1051825

gerritbot added a comment.Jul 8 2024, 3:16 PM

Change #1049589 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Fixing broken and flaky tests in connect.js

https://gerrit.wikimedia.org/r/1049589

Jdforrester-WMF updated the task description. (Show Details)Jul 8 2024, 3:19 PM
ReleaseTaggerBot edited projects, added MW-1.43-notes (1.43.0-wmf.13; 2024-07-09); removed MW-1.43-notes (1.43.0-wmf.10; 2024-06-18).Jul 8 2024, 4:00 PM
Jdforrester-WMF moved this task from Essential work to Ready to deploy on the Abstract Wikipedia team (25Q1 (Jul–Sep)) board.Jul 11 2024, 12:03 PM
Jdforrester-WMF moved this task from Ready to deploy to Needs Sign-off on the Abstract Wikipedia team (25Q1 (Jul–Sep)) board.Jul 17 2024, 4:55 PM
Jdforrester-WMF closed this task as Resolved.Jul 18 2024, 3:05 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits