Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[framework]Add semantics role to table rows. #163337

Merged
merged 35 commits into from
Mar 13, 2025

Conversation

hannah-hyj
Copy link
Member

@hannah-hyj hannah-hyj commented Feb 14, 2025

1. framework side:
This PR Create semantics node for rows in table's assembleSemanticsNode function.

2. web side:
I tested on my mac, and i need to remove the
<flt-semantics-container> between table and row, row and cell to traverse inside table, removing those transfom intermediate containers on web will be a bit of hassle and will be in another separate PR.

For example this code can only announce table but can’t get into cells.

<flt-semantics id="flt-semantic-node-4" role="table" style="position: absolute; overflow: visible; width: 751px; height: 56px; transform-origin: 0px 0px 0px; transform: matrix(1, 0, 0, 1, 0, 56); pointer-events: none; z-index: 1;">
    <flt-semantics-container style="position: absolute; pointer-events: none; top: 0px; left: 0px;">
        <flt-semantics id="flt-semantic-node-6" role="row" style="position: absolute; overflow: visible; width: 751px; height: 56px; top: 0px; left: 0px; pointer-events: none;">
            <flt-semantics-container style="position: absolute; pointer-events: none; top: 0px; left: 0px;">
                <flt-semantics id="flt-semantic-node-5" role="columnheader" aria-label="Name" style="position: absolute; overflow: visible; width: 751px; height: 56px; top: 0px; left: 0px; pointer-events: all;"></flt-semantics>
            </flt-semantics-container>
        </flt-semantics>
    </flt-semantics-container>
</flt-semantics>

If I removed the in between </flt-semantics-container>, the code come

<flt-semantics id="flt-semantic-node-4" role="table" style="position: absolute; overflow: visible; width: 751px; height: 56px; transform-origin: 0px 0px 0px; transform: matrix(1, 0, 0, 1, 0, 56); pointer-events: none; z-index: 1;">
    <flt-semantics id="flt-semantic-node-6" role="row" style="position: absolute; overflow: visible; width: 751px; height: 56px; top: 0px; left: 0px; pointer-events: none;">
        <flt-semantics id="flt-semantic-node-5" role="columnheader" aria-label="Name" style="position: absolute; overflow: visible; width: 751px; height: 56px; top: 0px; left: 0px; pointer-events: all;"></flt-semantics>
    </flt-semantics>
</flt-semantics>

And I can get into table cells.

3. Other aria-attributes:
aria-colcount ,aria-rowcount  aria-colindex  aria-rowindex 

  • theres attributes are only needed if some rows and columns are hidden in the Dom tree. havn't added them yet

aria-rowspan , aria-colspan :
*we currently don't support row span and col span in our widgets. related issue: #21594

related: #162339
issue: #45205

Pre-launch Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

@github-actions github-actions bot added framework flutter/packages/flutter repository. See also f: labels. engine flutter/engine repository. See also e: labels. f: material design flutter/packages/flutter/material repository. a: accessibility Accessibility, e.g. VoiceOver or TalkBack. (aka a11y) platform-web Web applications specifically labels Feb 14, 2025
@hannah-hyj hannah-hyj requested review from yjbanov, goderbauer and chunhtai and removed request for goderbauer February 14, 2025 20:08
@hannah-hyj
Copy link
Member Author

hannah-hyj commented Feb 14, 2025

hmmm looks like i should not simply remove <flt-semantics-container>, it will mess up with some transform. i will mark this PR as draft and do more testing.

@yjbanov
Copy link
Contributor

yjbanov commented Feb 14, 2025

hmmm looks like i should not simply remove <flt-semantics-container>, it will mess up with some transform. i will mark this PR as draft and do more testing.

Oof, that's a tricky problem. What's the typical value of rect passed for tables in semantic updates. I wonder if they are simple enough that we could use an alternative strategy that doesn't need auxiliary child container elements. For example, I think if rect.top and rect.left are zero, we can do everything with just one element.

@github-actions github-actions bot removed engine flutter/engine repository. See also e: labels. platform-web Web applications specifically labels Feb 19, 2025
@hannah-hyj hannah-hyj requested a review from chunhtai February 20, 2025 20:59
@hannah-hyj hannah-hyj changed the title Add semantics role to table rows. [framework]Add semantics role to table rows. Feb 20, 2025
@hannah-hyj
Copy link
Member Author

@chunhtai I want to merge this framework changes PR first and the "removing intermediate container layers in web engine" will be a separate effort in another PR.


final SemanticsConfiguration configuration =
SemanticsConfiguration()
..indexInParent = i
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this going to be used by the row index? if so, does the cell also uses this to set the col index? we should unify the way to set these indices in web.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated indexInParent for cells

children: const <TableRow>[
TableRow(children: <Widget>[Text('AAAAAA'), Text('B'), Text('C')]),
],
TableCell(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this needed for semantics check? do we need to worry about this being a breaking change?

Copy link
Member Author

@hannah-hyj hannah-hyj Feb 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes because the test case here before is a very corner case putting a table inside a tablerow child, so it became table-> row-> table and it failed the new a11y check, i think cases like this are very rare and they are just made up for tests

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, this is a case i replicate from internal code.

Should the tableRow assign the TableCell to its children internally?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the tableRow assign the TableCell to its children internally?
I'm already doing that in assembleSemanticsNode, i assign the role to its children if the children have no roles.
but if the children already have roles(in this case, it's table), do we want to override that role? or do we want to insert a semantics node with a cell role between them?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm already doing that in assembleSemanticsNode

Assigning child property in assembleSemanticsNode seems a bit hacky. We should do it through the TableCell widget if possible.

do we want to insert a semantics node with a cell role between them

This would be ideal, but doing it in widget layer means the Table here needs to be wrap with a semantics container. I think for now, let's just override it without creating a container. As you said this may be a rare case, we can fix it if it really is an issue

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also looks like this pr will make the table to be a container so there may not be overriding role issue in this particular case

Copy link
Member Author

@hannah-hyj hannah-hyj Mar 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assigning child property in assembleSemanticsNode seems a bit hacky. We should do it through the TableCell widget if possible.

We can't because we don't enforce user to use TableCell widget in table, TableRow can take any widgets as its children. User can just assign a Text widget to TableRow's children. That's why I assign the role here.

I think for now, let's just override it without creating a container.

SGTM,

there may not be overriding role issue in this particular case

in This particular case, the test case was table-> row-> table- > row-> cell, If I don't override the second table role , it will fail the check because row's children shouldn't be a table, if i override the second table role to be a cell, it will become table-> row-> cell- > row-> cell, it will fail the check because row's parent shouldn't be a cell. So I see the only way to fix this test case is to insert a cell and it will become table-> row-> cell -> table- > row-> cell,

Copy link
Member Author

@hannah-hyj hannah-hyj Mar 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be ideal

Updated this PR to insert a semantics node with a cell role between them.

@hannah-hyj
Copy link
Member Author

hannah-hyj commented Feb 21, 2025

I just realized I had some misunderstanding about semanticsNode rect before. I thought the tree dump rect is actual semanticsnode.rect, but it's actually semanticsnode.rect with semanticsnode.transform.

Instead of only setting row's and cell's rect, actually i should set their transform. Because rect in the a semantics node won't change their children's position, only transform will.

Updated PR.

@hannah-hyj hannah-hyj requested a review from chunhtai February 21, 2025 21:57
children: const <TableRow>[
TableRow(children: <Widget>[Text('AAAAAA'), Text('B'), Text('C')]),
],
TableCell(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm already doing that in assembleSemanticsNode

Assigning child property in assembleSemanticsNode seems a bit hacky. We should do it through the TableCell widget if possible.

do we want to insert a semantics node with a cell role between them

This would be ideal, but doing it in widget layer means the Table here needs to be wrap with a semantics container. I think for now, let's just override it without creating a container. As you said this may be a rare case, we can fix it if it really is an issue

@@ -2869,6 +2895,13 @@ class SemanticsNode with DiagnosticableTreeMixin {
/// {@endtemplate}
SemanticsRole get role => _role;
SemanticsRole _role = _kEmptyConfig.role;
set role(SemanticsRole value) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should avoid this

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to assign the role cell to the child in assembleSemanticsNode.
And i want to only change its role. So i think this is needed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is removed after offline discussion.

we won't directly change semantics node's role, just wrap them with a new semantics node with a cell role.

children: const <TableRow>[
TableRow(children: <Widget>[Text('AAAAAA'), Text('B'), Text('C')]),
],
TableCell(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also looks like this pr will make the table to be a container so there may not be overriding role issue in this particular case

@chunhtai
Copy link
Contributor

chunhtai commented Mar 3, 2025

We can't because we don't enforce user to use TableCell widget in table

I mean we wrap it internally in either Table widget or TableRow widget

@chunhtai
Copy link
Contributor

chunhtai commented Mar 3, 2025

I mean we wrap it internally in either Table widget or TableRow widget

looking at the code, we shouldn't add semanticsRole.cell in TableCell. It should added internally in Table or TableRow, this also avoid forcing people to use TableCell to have the correct semantics

@hannah-hyj
Copy link
Member Author

hannah-hyj commented Mar 3, 2025

I mean we wrap it internally in either Table widget or TableRow widget

If you mean like to wrap TableRow's every children with Semantics( role: cell: child: child), I tried that and it didn't work out because TableCell is a ParentDataWidget so it assumes its parent is a Table, insert a Semantics widget between TableCell and Table will crash it. So I chose to assign the semantics role to cells in assembleSemanticsNode although it looks a bit hack 😅 maybe there is a better way to do it ?

we shouldn't add semanticsRole.cell in TableCell

make sense, I will remove that.

@hannah-hyj hannah-hyj requested review from chunhtai March 3, 2025 22:03
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 13, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 13, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 13, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 13, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 13, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 13, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 13, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 14, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 14, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 14, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 15, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 15, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 16, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 16, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 16, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 17, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a: accessibility Accessibility, e.g. VoiceOver or TalkBack. (aka a11y) f: material design flutter/packages/flutter/material repository. framework flutter/packages/flutter repository. See also f: labels.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants