Task description update

• ADDED target finish date for this task to the task description – Friday, 14-May-2021 – per the conversation @MNeisler and I had on 28-April-2021.

Per T263505#7054699, work on this task can technically begin as early as this upcoming Thursday, 6 May. Although, @MNeisler and I are planning for work on this task to begin a week from today, on Monday, 10 May.

@DLynch - I checked VisualEditorFeatureUse today and have not seen any of the new events logged since deployment last Thursday. I'm wondering if it's possible that there have been no completed edits by users without JS logged since deployment (seems unlikely but may be possible due to sampling?) or if there might be an issue with the instrumentation.

Here are all the event actions logged in May 2021 for event.feature = 'mwSave'. There are no instances of event.action = 'source-has-js'.

Data via

SELECT
    event.action,
    COUNT(*)
FROM event.visualeditorfeatureuse
WHERE
     YEAR = 2021
    AND MONTH = 05
    AND event.feature = 'mwSave'
GROUP BY
    event.action

Hm. It should be about as common as the dialog-save action. I'll look into it.

Okay, there's a problem with the session ids: https://logstash.wikimedia.org/app/dashboards#/view/AXN5OoJu3_NNwgAUlbUT?_g=h@ddc44fc&_a=h@ea9a9a5

Change 689984 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/WikiEditor@master] PHP VisualEditorFeatureUse logging: properly record session id

https://gerrit.wikimedia.org/r/689984

Change 689984 merged by DLynch:

[mediawiki/extensions/WikiEditor@master] PHP VisualEditorFeatureUse logging: properly record session id

https://gerrit.wikimedia.org/r/689984

Change 689901 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/WikiEditor@wmf/1.37.0-wmf.4] PHP VisualEditorFeatureUse logging: properly record session id

https://gerrit.wikimedia.org/r/689901

Change 689902 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/WikiEditor@wmf/1.37.0-wmf.5] PHP VisualEditorFeatureUse logging: properly record session id

https://gerrit.wikimedia.org/r/689902

Change 689902 merged by jenkins-bot:

[mediawiki/extensions/WikiEditor@wmf/1.37.0-wmf.5] PHP VisualEditorFeatureUse logging: properly record session id

https://gerrit.wikimedia.org/r/689902

Mentioned in SAL (#wikimedia-operations) [2021-05-12T23:40:11Z] <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.5/extensions/WikiEditor/includes/WikiEditorHooks.php: rEWEDef4139628a36: PHP VisualEditorFeatureUse logging: properly record session id (T281409) (duration: 01m 08s)

Change 689901 merged by Urbanecm:

[mediawiki/extensions/WikiEditor@wmf/1.37.0-wmf.4] PHP VisualEditorFeatureUse logging: properly record session id

https://gerrit.wikimedia.org/r/689901

Mentioned in SAL (#wikimedia-operations) [2021-05-12T23:48:30Z] <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.4/extensions/WikiEditor/includes/WikiEditorHooks.php: rEWED2f6af514c49d? PHP VisualEditorFeatureUse logging: properly record session id (T281409) (duration: 01m 07s)

We have confirmed that the related errors have stopped, and data is being logged. Checks on the quality of said data have not yet occurred.

@DLynch

Working on post-deployment QA and noticed just one potential issue so far:

We have not logged any event.action = 'source-has-js' events for the mobile platform since the fix was deployed yesterday. There have been 12,409 no js wikitext events logged to date and they have all been recorded as coming from desktop.

Any ideas on why that might be? For comparison, there were 9,165 event.action = 'dialog-save' events on the mobile platform logged during the same timeframe.

I'll follow-up with my notes from the QA but everything else looks good so far.

Here's the notebook I'm performing the QA checks for reference.

That'd be expected -- WikiEditor only logs events as being desktop. See T249944 for a discussion of the issue.

(Conveniently, anything logging as phone like those dialog-save events inherently is using JS, since it can only possibly be logged via MobileFrontend.)

That'd be expected -- WikiEditor only logs events as being desktop. See T249944 for a discussion of the issue.

Per discussion with @ppelberg and @DLynch , we decided to use the user-agent data to approximate the mobile vs desktop wikitext edits to complete the analysis described in T240697. Note: User agent data will only be able to review for the last 90 days.

I've completed the remaining QA checks and everything else appears to be logging as expected.

Summary of checks and observations re event.action = 'source-has-js events:

• We have recorded a total of 25,492 events by 9,214 distinct users (12 May to 14 May 2021). These numbers are similar in scale to the number of events marked as event.action = 'dialog-save' recorded around the same time.
• Confirmed that events are only recorded for the event.feature = 'mwSave'
• We've recorded events for 250 projects so far. The larger wikis (English, German, Spanish, and French) have the most total events logged as expected. A look at the percentage of all wikitext edits by js users for each project (which will be done in the analysis) will provide more info on how frequently these types of edits occur.
• The majority of sessions (66%) are by logged-in users.
• Confirmed that all events are also recorded as saved (event.action = saveSuccess) in EditAttemptStep. I will use a join to EditAttemptStep to determine the percent of edits coming from users without JS for the analysis.

Please see QA notebook for further details

@ppelberg - Reassigning to you for sign-off. Let me know if you have any questions.

In T281409#7089320, @MNeisler wrote:

I've completed the remaining QA checks and everything else appears to be logging as expected.

Excellent.

@ppelberg - Reassigning to you for sign-off. Let me know if you have any questions.

@MNeisler: does any instrumentation documentation need to be updated? [i]

i.

Done

• @MNeisler: once we've verified the events implemented in T263505 have are working as expected, edit the related documentation (e.g. https://meta.wikimedia.org/wiki/Schema:VisualEditorFeatureUse) to include the new events that we've verified to be working as expected

@MNeisler: does any instrumentation documentation need to be updated?

@ppelberg Good call - thanks for checking on this. I just updated the VisualEditorFeatureUse Dictionary with the new values used to track these events.

See: https://www.mediawiki.org/w/index.php?title=VisualEditor/FeatureUse_data_dictionary&diff=4603143&oldid=4412666