VersionedProperty support in the JSPs

[eggrobin]

The invariant test have a versioned property syntax, e.g., \p{U4.1:XID_Continue}, implemented in VersionedProperty.

unicodetools/unicodetools/src/main/java/org/unicode/text/UCD/TestUnicodeInvariants.java

Lines 1033 to 1072 in 70dce2c

	static class VersionedProperty {
	private String propertyName;
	private String version;
	private UnicodeProperty.Factory propSource;
	private UnicodeProperty property;
	private final transient PatternMatcher matcher = new UnicodeProperty.RegexMatcher();
	public VersionedProperty set(String xPropertyName) {
	xPropertyName = xPropertyName.trim();
	if (xPropertyName.contains(":")) {
	final String[] names = xPropertyName.split(":");
	if (names.length != 2 || !names[0].startsWith("U")) {
	throw new IllegalArgumentException("Too many ':' fields in " + xPropertyName);
	}
	if (names[0].equalsIgnoreCase("U-1")) {
	version = LAST_VERSION;
	} else {
	version = names[0].substring(1);
	}
	xPropertyName = names[1];
	} else {
	version = LATEST_VERSION;
	}
	;
	propertyName = xPropertyName;
	propSource = getProperties(version);
	property = propSource.getProperty(xPropertyName);
	if (property == null) {
	propSource = getIndexedProperties(version);
	property = propSource.getProperty(xPropertyName);
	if (property == null) {
	throw new IllegalArgumentException(
	"Can't create property from name: "
	+ propertyName
	+ " and version: "
	+ version);
	}
	}
	return this;
	}

Something like that would be useful for list-unicodeset.jsp.
It sounds like there is limited support for multiple versions with the β syntax, but I cannot figure out how that one is implemented.

[macchiati]

I'll talk a look. For the beta, the idea is that you have multiple release versions plus 1 dev version. Do you think we need multiple?

…

On Wed, Apr 5, 2023, 19:16 Robin Leroy ***@***.***> wrote: The invariant test have a versioned property syntax, *e.g.*, \p{U4.1:XID_Continue}, implemented in VersionedProperty. https://github.com/unicode-org/unicodetools/blob/70dce2c89f185c65b436c28404ae5b7bdb32c2d1/unicodetools/src/main/java/org/unicode/text/UCD/TestUnicodeInvariants.java#L1033-L1072 Something like that would be useful for list-unicodeset.jsp. It sounds like there is limited support for multiple versions with the β syntax, but I cannot figure out how that one is implemented. — Reply to this email directly, view it on GitHub <#432>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACJLEMD5PBSPDU3QAEXYX5TW7YRQLANCNFSM6AAAAAAWUZZAPA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[eggrobin]

For the beta, the idea is that you have multiple release versions plus 1 dev version.

So, matching the subfolders of ucd? That was what I had in mind.

Do you think we need multiple?

Multiple different βs? That sounds like it would confuse things rather than help.

[macchiati]

When I read "limited support for multiple versions with the β syntax", I thought you were asking for multiple betas. Glad you're not!

…

On Thu, Apr 6, 2023 at 7:48 AM Robin Leroy ***@***.***> wrote: For the beta, the idea is that you have multiple release versions plus 1 dev version. So, matching the subfolders of ucd? That was what I had in mind. Do you think we need multiple? Multiple different βs? That sounds like it would confuse things rather than help. — Reply to this email directly, view it on GitHub <#432 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACJLEMAQFLWLTKFPEM75DGTW73JSTANCNFSM6AAAAAAWUZZAPA> . You are receiving this because you commented.Message ID: ***@***.***>

[eggrobin]

@macchiati It seems like unicodetools has (at least) two ways to look at versioned properties:

1. ToolUnicodePropertySource (which is used to compute the derived properties to produce the new data files, as well as used by VersionedProperty in the invariants above, but has correctness issues for old versions, see The versioning of ToolUnicodePropertySource is neither tested nor correct #484);
2. IndexUnicodeProperties (which is used by, e.g., TestProperties; I don’t know if it has similar issues to ToolUnicodePropertySource).

IndexUnicodeProperties seems more modern (compare the pre-2013 histories of IndexUnicodeProperties and ToolUnicodePropertySource), and since it does not have the derivations, it might have a better chance of being correct for older versions than ToolUnicodePropertySource. Should the invariants use it instead of ToolUnicodePropertySource?

[eggrobin]

Should the invariants use it instead of ToolUnicodePropertySource?

Wait, they currently use both, with a preference for ToolUnicodePropertySource. So I guess the question is whether this should be inverted.

         propSource = getProperties(version);  // ToolUnicodePropertySource.
         property = propSource.getProperty(xPropertyName); 
         if (property == null) { 
             propSource = getIndexedProperties(version);   // IndexUnicodeProperties.
             property = propSource.getProperty(xPropertyName); 

[macchiati]

I agree, it should be inverted. There are a few constructed properties (non-UCD) that are only available via Tool... That is, the extra normalization properties. So those should fallback to the Tool

…

On Thu, May 25, 2023 at 5:18 AM Robin Leroy ***@***.***> wrote: Should the invariants use it instead of ToolUnicodePropertySource? Wait, they currently use *both*, with a preference for ToolUnicodePropertySource. So I guess the question is whether this should be inverted. propSource = getProperties(version); // ToolUnicodePropertySource. property = propSource.getProperty(xPropertyName); if (property == null) { propSource = getIndexedProperties(version); // IndexUnicodeProperties. property = propSource.getProperty(xPropertyName); — Reply to this email directly, view it on GitHub <#432 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACJLEMEJW2FASG5YN2NRKRLXH5E3DANCNFSM6AAAAAAWUZZAPA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[eggrobin]

it should be inverted

Ah, that doesn’t work, because IndexUnicodeProperties, while closer to the truth for non-default values, it has issues with default values (I think it relies on @‌missing lines that are not always there).

At a glance, it is missing defaults at least for:

1. CCC,
2. the non-special values of the properties in SpecialCasing (those used to have @‌missing lines, which IndexUnicodeProperties.txt knows how to read, but they got removed in 7.0),
3. the case foldings.

(The failings of IndexUnicodeProperties should probably turn into their own issue, but I am on a train with spotty internet, so I will be happy if this comment goes through.)

[macchiati]

The failings of IndexUnicodeProperties should probably turn into their

own issue Agreed, and thanks for tracking this down

…

On Thu, May 25, 2023 at 9:55 AM Robin Leroy ***@***.***> wrote: it should be inverted Ah, that doesn’t work, because IndexUnicodeProperties, while closer to the truth for non-default values, it has issues with default values (I think it relies on @‌missing lines that are not always there). At a glance, it is missing defaults at least for: 1. CCC, 2. the non-special values of the properties in SpecialCasing (those used to have @‌missing lines, which IndexUnicodeProperties.txt knows how to read, but they got removed in 7.0), 3. the case foldings. (The failings of IndexUnicodeProperties should probably turn into their own issue, but I am on a train with spotty internet, so I will be happy if this comment goes through.) — Reply to this email directly, view it on GitHub <#432 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACJLEMEY236EVLDNDCS6QYDXH6FJRANCNFSM6AAAAAAWUZZAPA> . You are receiving this because you were mentioned.Message ID: ***@***.***>