Apple: Accessibility and Localization API schemas

[dgovil]

This commit adds two applied API schemas to UsdUI as an attempt to make it a Universally Accessible Scene Description.

Disclaimer: This addition does not represent any future feature support in our products. These schemas are borne out of discussing the needs for the ecosystem to accommodate a wider range of users.

Overview

A brief overview of the two schemas:

Accessibility API

Proposal

def "Foo"(
    apiSchemas = ["AccessibilityAPI:default"]
) {
    string accessibility:label = "Short Label"
    string accessibility:description = "A longer description of this prim"
    uniform token accessibility:priority = "standard"
}

UsdUIAccessibilityAPI is a multiple apply schema.
Each instance has three attributes:

label: A short description of the prim
description: An extended description of the prim
priority: A hint to a runtime on how this instance should be valued

The default instance does not namespace itself with the instance name as we believe it will be the only one specified in many cases, and the excess namespace doesn't add clarity.

As a convenience, the API specifies a default name so users of Apply() do not need to specify a name most of the time.

Localization API

Proposal

def "Foo" (
    apiSchemas = ["LocalizationAPI:default", "LocalizationAPI:fr_CA"]
)
{
     uniform token localization:lang = "en_US"
     string foo = "Hello"
     string foo:lang:fr_CA = "Bonjour"
     string foo:lang:hi_IN = "नमस्ते"
}

UsdUILocalizationAPI is also a multiple apply schema. However here it uses the multiple apply to denote what language are available or that a default localization is available on a prim.

It includes one attribute that doesn't use an instance name called localization:langauge which specifies the default language to assume.

All other applications are suffixed to properties with :lang:<language>

Feedback and Remaining Work

It would be greatly appreciated if feedback could be provided with a focus on the schema first so we can nail down how to represent the data in the USD.

I also haven't finished wrapping the LocalizationAPI methods in Python because I would like to first get feedback on the C++ API so that I may iterate faster. Once we feel the API looks good, I'll handle the Python bindings and write appropriate tests for them as well.

• I have verified that all unit tests pass with the proposed changes

• I have submitted a signed Contributor License Agreement

[nvmkuruc]

Do you have any thoughts on how localization should work across schema domains? For example-- consider the following scene description.

def Xform "prim_with_primvar" (apiSchemas = ["LocalizationAPI:default", "LocalizationAPI:fr_CA"]) {
    string primvars:greeting = "hello"
    string primvars:greeting:lang:fr_CA = "bonjour"
}

I could see some clients may interact with primvars:greeting as a localized attribute with the UsdUILocalizationAPI and others may use the UsdGeomPrimvarsAPI and treat greeting and greeting:lang:fr_CA as two distinct primvars.

Localization and UsdShade is an interesting case to consider as well.

[dgovil]

@nvmkuruc That's a good question and something I forgot to mention in my PR docs (I had a line in the proposal discussion somewhere)

But yes, I think I wanted to have language on restricting the set of attributes one should localize. I wasn't sure how best to phrase that or whether the API should do validation itself around that.

I think longer term, my thought would be that localization should only be used for things like strings (localized text), asset paths (localized textures?) and rels (localized animation bindings for lip-syncs?).

However, even there, I think at the start my recommendation phrasing is that it should be limited to only strings at the start, and that it's only for strings that will be presented to a user as part of content consumption.

I think that recommendation would perhaps side-step the issue you mention since I don't know of a scenario where a primvar would fall under that. If that kind of wording would help, I think we could brainstorm how to phrase it here.

[nvmkuruc]

I'm not sure that it sidesteps it because string and asset paths are commonly used as primvars and shader inputs for specifying or generating texture paths.

I'd expect the current implementation of UsdGeomPrimarsAPI and UsdShadeShader would return a unique input or primvar that had localized string or asset properties.
https://openusd.org/dev/api/class_usd_geom_primvars_a_p_i.html#a2336d8d750d52f1a8950abe5cfafb75a
https://openusd.org/dev/api/class_usd_shade_shader.html#aef8e1bf11a790edd7ffa7a90de4c5cb7

[dgovil]

Hmm, I think that would be fine if they get returned in the list of properties.
Unless a UI is showing every single property on a prim, I don't believe it would cause any issues since the use would be targeted, so the existence of extra properties would be benign? Would you have a specific scenario where it might cause issues?

But also I think the initial recommendation of only limiting it to strings that are presented to users for consumption might avoid that for the time being

[jesschimein]

Filed as internal issue #USD-10083

[jesschimein]

/AzurePipelines run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[spiffmon]

Apologies for joining the party so late, but my hope would be that the schemas' scoping to UsdUI would (self and explicit) document that we should only be using it for localization/accessibility of UI-facing data, and such localization should not have any cross-schema interaction (I can't decide whether UsdUI is actually "lower level" than, e.g. UsdGeom, but it doesn't feel right for UI concerns to affect scene evaluation for rendering).

@nvmkuruc , is it reasonable to say that asset rendering localization is a different problem than Accessibility, or are we really trying to get both of those birds in the same sack?

[dgovil]

@spiffmon I believe I'm advocating for the same thing as you. I think (at least to start) it should only be used for strings that are presented to a user in a consumable way. Ie not generic attributes that are shown in an editor UI, but strings that themselves are shown as UI (such as the Autodesk text proposal) or expressed to the UI like accessibility over.

I do think that people might use it to express other localizations in the future, which is why I mention the other possibilities. However I have language in the doc to say that any uses of localization are dependent on knowing what the endpoints support.

I can strengthen the wording to emphasize the recommendation to only use it for user facing strings etc...

[spiffmon]

From my POV, it would be good to keep the scoping tighter for now... there's just some meaty problems to talk through and solve if we start allowing it to be used for localizing texture names, especially in a procedural way, and/or through the primvar-propagation mechanism.

[dgovil]

Yep, agreed. I'll try and come up with suitable language for the doc to strengthen that recommendation

[nvmkuruc]

Yep, agreed. I'll try and come up with suitable language for the doc to strengthen that recommendation

I think one of the things that has been throwing me is that the example in the PR description shows string foo as an attribute that can be localized. That implied to me that "any attribute" (or "any string attribute") can be localized and why I started to wonder about the implications for primvars, shader inputs, etc.

[spiffmon]

I agree that's a concern, @nvmkuruc , and the only way in USD-of-today to make the limited scope obvious in the data itself instead of documentation would be to require a namespace for all localizable strings.

In the future, if we are copacetic with core USD behaviors relying on OpenExec, I believe it should be possible for the "localization behavior" (which means tracking dependencies between attributes to get proper dirtying, as well as evaluation behavior for which attribute should be consulted) to be injected where it is needed, given an applied schema to impart it. I am not 100% sure of that and haven't had time to talk to exec folks about it yet. But if we prescribe a namespace now, it cuts off that possibility. If we leave it in the not-completely-satisfying-or-validatable state of "limitations by fiat/documentation", then we leave the door open.

Would it be worth taking some time to see how far we can go with validation without generating false negatives?

[dgovil]

I'm thinking a reasonable check right now might be:

• Is a string attribute
• Is not a primvar or an input/output

I think that would cover most of the concerns?

[dgovil]

Just pinging this thread again, do the changes in my previous message sound reasonable? I can update the PR accordingly?