[feature] Relax eXist functions that require parameter of type xs:anyURI to accept xs:string

[joewiz]

Is your feature request related to a problem? Please describe.

I frequently encounter errors when I use functions that require URLs and URIs to be typed as xs:anyURI—instead of, as is the case in the standard function library, xs:string (see fn:collection, which "interpret[s] $arg as an xs:anyURI", fn:doc, fn:doc-available, fn:json-doc; the standard functions may return xs:anyURI when appropriate but never require this type in parameters). The xs:anyURI requirement, while a correct description of the expected content, doesn't actually provide much benefit. On the other hand, it imposes a moderate burden on XQuery coders.

Describe the solution you'd like

I would propose a review of the modules that do this and raise these for consideration by the community. Questions could include:

• Does the parameter really need to be xs:anyURI?
• Is this function so different from fn:doc, etc. that it merits the additional developer burden on XQuery coders?
• Would the function not work as well if it were relaxed to xs:string?

If we can identify functions that can adopt the same behavior as the standard function library, perhaps the function signatures of these could be adjusted to allow xs:string and to "interpret" this argument "as an xs:anyURI" (to borrow a phrase from the spec for fn:collection).

Describe alternatives you've considered

The alternative would be to take no action.

Additional context

The following query generates a list of all built-in, java-based functions that take a parameter typed as xs:anyURI:

xquery version "3.1";

declare namespace xqdoc="http://www.xqdoc.org/1.0";

<div>{
    for $sig in collection("/db/apps/fundocs/data")//xqdoc:signature[matches(., "as xs:anyURI[\?\*\+]?[,\)]")]
    let $function := $sig/..
    let $xqdoc := $sig/ancestor::xqdoc:xqdoc
    let $module := $xqdoc/xqdoc:module
    let $module-name := ($module/xqdoc:name/string(), "fn")[normalize-space(.) ne ""][1]
    let $module-uri := $module/xqdoc:uri
    where starts-with($xqdoc/xqdoc:control/xqdoc:location, "java:")
    group by $module-name
    order by $module-name
    return
        <div>
            <p><strong>{$module-name} ({head($module-uri)/string()})</strong></p>
            <ol>{
                for $s in $sig
                order by $s
                return
                    <li>{analyze-string($s, "xs:anyURI")/* ! (if (self::fn:match) then <strong>{string(.)}</strong> else text{.})}</li>
            }</ol>
        </div>
}</div>

Here's the list:

exrest (http://exquery.org/ns/restxq/exist)

1. exrest:deregister-module($module as xs:anyURI) as document-node()
2. exrest:deregister-resource-function($module as xs:anyURI, $function-signature as xs:string) as xs:boolean
3. exrest:find-resource-functions($module as xs:anyURI) as document-node()
4. exrest:register-module($module as xs:anyURI) as document-node()
5. exrest:register-resource-function($module as xs:anyURI, $function-signature as xs:string) as xs:boolean

image (http://exist-db.org/xquery/image)

1. image:thumbnail($collection as xs:anyURI, $thumbnail-location as xs:anyURI?, $dimension as xs:integer*, $prefix as xs:string?) as xs:string*

inspect (http://exist-db.org/xquery/inspection)

1. inspect:inspect-module($location as xs:anyURI) as element()?
2. inspect:inspect-module-uri($uri as xs:anyURI) as element()?
3. inspect:module-functions($location as xs:anyURI) as function(*)*
4. inspect:module-functions-by-uri($uri as xs:anyURI) as function(*)*

response (http://exist-db.org/xquery/response)

1. response:redirect-to($uri as xs:anyURI) as empty-sequence()

session (http://exist-db.org/xquery/session)

1. session:encode-url($url as xs:anyURI) as xs:anyURI

sm (http://exist-db.org/xquery/securitymanager)

1. sm:add-group-ace($path as xs:anyURI, $group-name as xs:string, $allowed as xs:boolean, $mode as xs:string) as empty-sequence()
2. sm:add-user-ace($path as xs:anyURI, $user-name as xs:string, $allowed as xs:boolean, $mode as xs:string) as empty-sequence()
3. sm:chgrp($path as xs:anyURI, $group-name as xs:string) as empty-sequence()
4. sm:chmod($path as xs:anyURI, $mode as xs:string) as empty-sequence()
5. sm:chown($path as xs:anyURI, $owner as xs:string) as empty-sequence()
6. sm:clear-acl($path as xs:anyURI) as empty-sequence()
7. sm:get-account-metadata($username as xs:string, $attribute as xs:anyURI) as xs:string?
8. sm:get-group-metadata($group-name as xs:string, $attribute as xs:anyURI) as xs:string?
9. sm:get-permissions($path as xs:anyURI) as document-node()
10. sm:has-access($path as xs:anyURI, $mode as xs:string) as xs:boolean
11. sm:insert-group-ace($path as xs:anyURI, $index as xs:int, $group-name as xs:string, $allowed as xs:boolean, $mode as xs:string) as empty-sequence()
12. sm:insert-user-ace($path as xs:anyURI, $index as xs:int, $user-name as xs:string, $allowed as xs:boolean, $mode as xs:string) as empty-sequence()
13. sm:modify-ace($path as xs:anyURI, $index as xs:int, $allowed as xs:boolean, $mode as xs:string) as empty-sequence()
14. sm:remove-ace($path as xs:anyURI, $index as xs:int) as empty-sequence()
15. sm:set-account-metadata($username as xs:string, $attribute as xs:anyURI, $value as xs:string) as empty-sequence()
16. sm:set-group-metadata($group-name as xs:string, $attribute as xs:anyURI, $value as xs:string) as empty-sequence()

util (http://exist-db.org/xquery/util)

1. util:declare-namespace($prefix as xs:string, $namespace-uri as xs:anyURI) as item()
2. util:import-module($module-uri as xs:anyURI, $prefix as xs:string, $location as xs:anyURI*) as item()

validation (http://exist-db.org/xquery/validation)

1. validation:pre-parse-grammar($grammar as xs:anyURI*) as xs:string*

xmldb (http://exist-db.org/xquery/xmldb)

1. xmldb:decode-uri($uri as xs:anyURI) as xs:string
2. xmldb:get-mime-type($resource-uri as xs:anyURI) as xs:string?
3. xmldb:set-mime-type($resource-uri as xs:anyURI, $mime-type as xs:string?) as empty-sequence()

xqdm (http://exist-db.org/xquery/xqdoc)

1. xqdm:scan($uri as xs:anyURI) as node()*

zip (http://expath.org/ns/zip)

1. zip:binary-entry($href as xs:anyURI, $entry as xs:string) as xs:base64Binary
2. zip:entries($href as xs:anyURI) as node()
3. zip:html-entry($href as xs:anyURI, $entry as xs:string) as document-node()
4. zip:text-entry($href as xs:anyURI, $entry as xs:string) as xs:string
5. zip:update($href as xs:anyURI, $paths as xs:string+, $binaries as xs:base64Binary+) as xs:base64Binary?
6. zip:xml-entry($href as xs:anyURI, $entry as xs:string) as document-node()

[line-o]

I 100 percent agree.

[adamretter]

I think it might be better to discuss adding an addditional Type Promotion rule to XQuery 4 instead to address this

[joewiz]

@adamretter I've proposed the idea in the xpath-ng channel on the XML.com Slack. (Direct link for those already on the XML.com Slack.)

[line-o]

What I do not understand: What is the benefit of requiring xs:anyURI at some points but allow xs:string at others? It is this inconsistency that hurts the developer experience.

[line-o]

Who will benefit from the type promotion. Is it users declaring their script to be XQuery 4? When will this be available for eXist-db? Is there a plan to add XQuery 4 to the platform - a timeline?
Until then we will be stuck with this and I am therefore advocating to loosen all of the function signatures above to allow xs:string.
Especially, since xmlb:store returns an instance of xs:string and that can then not directly be used with sm:chmod et al.

[line-o]

Loosening it now will still allow us to declare the type to xs:anyURI once and if this will be available at some point.

[joewiz]

While the discussion in Slack continues about the viability of altering coercion rules or developing a new notion of "primitive type families" that would "offer implicit coercion within the family" in XPath 4 and its derived languages, I thought I'd try a naïve attempt at the original proposal for the functions from the list above that I was able to find in the eXist core source base - joewiz@8fc5bda. In a couple of cases I adjusted the parameter name and description to clarify that a URI is expected, now that the type doesn't say xs:anyURI. However, the tests produce tons of errors, all saying:

org.exist.xquery.value.StringValue cannot be cast to org.exist.xquery.value.AnyURIValue

So clearly it's not simply a matter of changing the function signatures...

[dizzzz]

a conversion is needed to replace these kind of constructs

it needs to be replaced by org/exist/xquery/value/AnyURIValue.java:125 (invoking constructor)

[joewiz]

The EXPath ZIP spec uses xs:anyURI, so I think its functions are best left out of this issue. (I notice that BaseX removed its own support for this spec in July of this year.)

[joewiz]

While the XML.com Slack post I linked to above is no longer accessible (due to Slack's limitation on retrieving posts >90 days old), I wrote a new post to ask about the idea of adding an additional Type Promotion rule to XPath (and thus XQuery) 4 allowing xs:string to be type-promoted to xs:anyURI.

Liam Quinn responded:

i think that early on, XQuery/XPath developers and the authors of XML Schema kinda thought that the xs:anyURI type would have a constraint that its content must be a "valid URL".

Unfortunately there's no useful formal definition of "valid URL", let alone "valid IRI". Worse, incoming data is as likely to have xmlns="what is this" as anything else.

So really it just, as you say, expresses intent, which is good, but functionally it’s a (possibly empty) string.

I think it's a useful idea; the way forward is to propose it as an issue on the qt4 github thingy, probably, if that didn't happen already.

So I wrote up a proposal and posted it to qt4cg/qtspecs#480.