Thanks @BTullis for your feedback. It's really interesting and valuable. it's helping a lot to decide the right option here. So, according to what we know so far (https://docs.google.com/document/d/11exlY2vFBKpqUTKU_dwQdm88-oJn-94icM041kOe3-8), we could say that:

• Option 1 (Yarn script + Airflow DAG + Bacula) could be discarded because its complexity
• Option 2 (K8s CronJob + Bacula) seems doable and also simple. And there is no limitations regarding the script we can write. We could use nodejs, for example
• Option 3 (Background job embedded with MPIC) seems to be the simplest solution that meets all our needs; simple, automatic and we can configure periodicity.
• Option 4 (Manual script) It's even simpler than the previous one but we must run manually in case of disaster. With a tiny extra effort we could schedule the task using Option 3, for example
• Option 5 (scheduled task + S3 bucket): Seems great but more complex than others. We could discard it because of that and we always can improve the option we choose to include a S3 backup later

I have updated the ticket title and description to include all we have decided as per slack discussion

Hi @EChukwukere-WMF
We are still working on creating a specific read view for experiments. There is a task (T373695: MPIC: Create read view for experiments) to work on that. For now, we are reusing same view we use for instruments and then some fields shouldn't be there

Hi @EChukwukere-WMF
We are still working on creating a specific read view for experiments. There is a task (T373695: MPIC: Create read view for experiments) to work on that. For now, we are reusing same view we use for instruments and then some fields shouldn't be there

Hi @EChukwukere-WMF
Could you provide backend log for this issue? Is there anything in the browser console? The update button should work and any log could be really useful to see this specific issue

Based on our discussion I have rewritten the engineering AC (from creating a separate ticket to implement using this) and I have moved this task to the backlog so someone can pick it to start working on

@phuedx
I was wondering whether it makes sense to include `agent_app_install_id as an optional contextual attribute for the JS client library. The attribute is defined as valid but, in reality, we are not filling its value right now (at least through the integration we have for EventLogging). Should it be there as an optional contextual attribute?

I think that I explained me wrongly (again) in my last comment (https://phabricator.wikimedia.org/T374984#10183323), probably because I have been learning some details about the contextual attributes involved in this task while working on this. I think right now we are already in the right way. We found that agent_client_platform and agent_client_platform_family are already automatically added by the JS client library. That means that nobody needs to add them manually to be considered for the related instrument configuration. And that includes the user and also MPIC as a tool. I mean, we don't even need that MPIC adds automatically the fields because the client library will do it anyway.

I have updated the ticket description to add all the work done here:

• Toolforge mediawiki for MPIC Alpha demo: https://gitlab.wikimedia.org/sfaci/mpic-alpha-demo
• Step by step instructions: https://docs.google.com/document/d/1q2zK3i6t7e3aQ_i9nHNip14wNXFJLhIFWEWkREU6BvM/edit#heading=h.iuxwy6izclz4
• Demo recording: https://drive.google.com/file/d/1b3VzMeEa05WjjjowoLbK2mA28dSlj0jg/view?usp=sharing
• Sam's Snippet to add some "instrument code" to show when the user is enrolled or not in an experiment: https://gitlab.wikimedia.org/-/snippets/180

@Sarai-WMF it looks good! I'll create the engineering ticket based on your work here. Thanks!

@Sarai-WMF Maybe we already mentioned that in other related ticket. We have some others related to design issues. Anyway, I agree on what you said. We can fix it with the other ticket. Thanks!

Base streams have been already published so we can start using them with MPIC and we can also remove them from being added manually to the Action API response when we need to show the active streams in the MPIC UI.

I have rescheduled the change to be merged

• October 2nd: 07:00-08:00 UTC -> https://wikitech.wikimedia.org/wiki/Deployments#Wednesday,_October_2

@SGupta-WMF @Sarai-WMF T374742: Update Metrics Platform docs on required contextual attributes is pretty related to this task. There we were discussing about which contextual attributes are automatically added by the client libraries. In the case of the JS one, we have found that agent_client_platform and agent_client_platform_family are added automatically so I think we could expand the current task and consider agent_client_platform_family also as a contextual attribute that users don't need to add while using MPIC.

Thanks for details @phuedx!

@apaskulin I also remember we were confused because some stream configurations didn't declare agent_client_platform_family as a contextual attribute. My last comment should explain that. I guess sometimes was added and others not just because but the reality is that we don't need to add it

@phuedx @apaskulin I'm sorry! It seems I was wrong again. I decided double check this and I have found https://gitlab.wikimedia.org/repos/data-engineering/metrics-platform/-/blob/main/js/src/ContextController.js?ref_type=heads#L33 where I can see that JS client library does add some agent contextual attributes automatically. In fact, it adds same contextual one as the PHP one: agent_client_platform and agent_client_platform_family. So, the following is what actually happens:

At this time the edit button MPIC has for this view has a label that says "Edit baseline instrument" or "Edit A/B test" depending on the type. But there is an effort to determine the right language for MPIC (https://phabricator.wikimedia.org/T373474) and, according to that and as per Slack discussion with @Sarai-WMF, we agreed on we should change the edit button to show only "Edit" regardless the user is viewing a baseline instrument or an A/B test.

I agree with @Sarai-WMF. None of them will look fine because of the current component we have (all selected contextual attributes are placed in the same place where we want to place the message) but I guess that's a temporary fix until we add the new component for this field. I guess all selected items will be place inside the field with the new component.

A change is already prepared for MPIC to use new names for web/base streams: https://gitlab.wikimedia.org/repos/data-engineering/mpic/-/merge_requests/104

@Sarai-WMF This work is already deployed on staging/production, so it's already available for you to review. Thanks!

@Sarai-WMF This work is already deployed on staging/production, so it's already available for you to review. Thanks!

Ok! Thanks!
I'll add a new AC here to cover that requirement and I'll prepare the proper change for MPIC to be aligned with the work done here

@cjming @phuedx I have just realized that MPIC uses product_metrics.test_app_base and product_metrics.test_web_base as stream names when user chooses App/Web base schema type while filling the Baseline instrument form. I think we should change that. Those stream names should be the same we are going to merge here, right? (and we decided that product_metris.app_base and product_metrics.web_base would be their names (without the test_ part)

The related MR is ready for review: https://gitlab.wikimedia.org/repos/data-engineering/mpic/-/merge_requests/103

I have found a new, maybe related/same issue. When typing a long word in the Baseline instrument name field, the page makes also wider same way the issue described in this ticket. Just posting this here to be considered when working on this

@cjming @phuedx This is ready for review. Thanks!

In fact, I think I have to correct my own words. I mentioned above we should add, at least, the required attributes for the Java client library and some others for the web one, but the reality is that Java and PHP clients are going to add some required ones automatically. It wouldn't make sense to do it.

I would say the only concern here should be that agent_client_platform_family is mandatory (event validation will fail if it's not included) and JS client library is not adding it automatically for now. But if we want to change that behaviour soon I guess we could already prepare the documentation considering it as done.

It seems slug validator is much stricter than the function we use to generate the slug for instruments. At this time "spaces" are the only restriction we have to build the slug and all special characters are included while the user types the human readable one, but validator doesn't allow punctuation marks (? ! , . . . .). Should they be allowed? Should we be more restrictive and remove punctuation marks when building the slug?

I found some new useful information regarding what I said above, wrongly, about the Java client library. That client is not adding only agent_client_platform_family but all agent contextual attributes automatically according to the REQUIRED_PROPERTIES list and how its content is added automatically. So, not only the one I mentioned above is added automatically for this client.

Based on the second AC, it seems we should reach a consensus for which contextual attributes need to be added to the base streams.My first guess was that, at least, we should add those that are mandatory for the specific base schema the stream is based on. I mean:

• agent_client_platform and agent_client_platform_family and for the web_base` stream
• It seems that all agents contextual attributes are required by the Java client library so we could add them for the app_base stream (https://gitlab.wikimedia.org/repos/data-engineering/metrics-platform/-/blob/main/java/src/main/java/org/wikimedia/metrics_platform/ContextController.java?ref_type=heads#L77)

Apart from the EventLogging change above, there is also a metrics-platform one ready for review: https://ggitlab.wikimedia.org/repos/data-engineering/metrics-platform/-/merge_requests/71