Solid basic form builder (Google Forms but the Solid way)

[phochste]

Pitch

A user would like to have a basic form handling backed by a Solid server. The Solid Forms application provides a tools to:

Create a form XYZ

• Create basic forms with choice of these ingredients:
  - Form name + title + description
  - Input field (with name and value and flag required or not)
  - Text field (with name and value and flag required or not)
  - Select field (with name , possible values and flag required or not)
  - File upload field (with name, drag/drop values and flag required or not)
• Forms are not nested , no paging, just a flat list of fields

Form XYZ resource overview

• The user selects the form XYZ
• The user sees a listing of all RDF resources already created by form XYZ
  • These RDF resources can be deleted or edited
• The user can add a new RDF resource using the form XYZ

Form entry

• When editing or adding an RDF resource the form fields for XYZ are presented
• The user adds/edits the form field
• The user submits the results

Desired solution

Form creation

This has a bit the look and feel of Google Forms. Or created by uploading a basic (OWL, SHACL,...) schema with allows only the subset of fields that are described above.

Form overview + entry

The Form overview provides a flat list of previous created RDF Resources and options to edit/delete them, plus a button to add a new resource. Optionally there is a search field to filter the displayed list of RDF resources

The Form entry provides a form based on the (OWL,SHACL,...) schema that is stored when creating a Form.

Acceptance criteria

• The application should deliberately seek to limit the types of forms that can be created to a easy to manage subset of name/value pairs.
• The result of the application should be an LDP Container with RDF Resources that are build with the Form
• The Forms can change over time..there is no guarantee that all RDF Resources in the resulting LDP Container will have the same fields

Challenges

• Drag & drop form builder app to build a basic RDF form definition #64
• Form render application that can render an RDF-form + RDF data + store it in a pod of choice #65

Pointers

• https://github.com/danielbeeke/rdf-form
• https://ruben.verborgh.org/blog/2019/06/17/shaping-linked-data-apps/

Scenarios

• https://docs.google.com/document/d/1U1fHwFZUKjjIRG7gxsJrBNMHXkEs7pR2e3IkeK7qh5U/edit?usp=sharing

[RubenVerborgh]

Sounds great; can we perhaps apply this to one or more use cases, such that we have an acceptance criterion for how complex the form should be?

---

Regarding existing work, note that https://github.com/solid/solid-ui contains a form system (there is a weak reference to it in my aforementioned shapes blog post). The idea is that:

• https://www.w3.org/ns/ui is used to describe a form
• https://github.com/solid/solid-ui contains one implementation to render such forms in the browser

And then it gets very meta with the FormForm (https://github.com/solid/solid-ui/blob/8108886c4b4b7c2dd8a43853d4e800e556c0dffa/Documentation/form-examples/edit-form-form.html), which is a form for editing forms.

[jeswr]

In work I've done in the past (but unfortunately had to put on the backburner while I finished off my honours), I was working on a SHACL form editor which was close to handling the full SHACL specification before other priorities got in the way

https://github.com/schimatos/shacl-form-react
I definitely want to spend a week or two finishing it off at some stage, probably in a few months.

For a very old version of this tool see
https://github.com/schimatos/schimatos.org - the code quality on this one is pretty horrific as it was my first time using JS and React

I was also starting to create a package called LDfields which was designed to customise individual fields based on the constraints on a value/iri and emitted a RDF.Term on the onChange event.

---

I feel that it is worth spelling out the design vision I had for the newer version of the tool - if anything to save a reinventing if the wheel on parts of the project that I have done a lot of work for but not yet had the time to complete/release.

First, the form generator is given the following inputs

1. An LDflex entity of type sh:NodeShape - this would be used to generate the form.
2. An LDflex entity to act as a Subject of the form. All edits to the form are then made by adding modifying data about this entity, or traversing a path via LDflex (to achieve this cleanly I was planning on creating LDflex handlers that traversed sh:paths) and then modifying those entities.

Internally - the form is generated by traversing through the sh:shape (this has been done). To add type safety here I was working on a package called on2ts to generate Typescript typings from SHACL Schema - I was then going to use the sh:ShapeShape definition to create Typescript bindings for the sh:NodeShape entity.

However - there are a few blockers in this

1. on2ts is still a WIP
2. LDflex does not have types (though I made a branch about a year or 2 ago where I started work on this - this was in turn blocked by issues around the typings for Proxies)
3. In order for the assumptions to be met in these generated typings, you quite often require implicit facts (hence why I started working in reasoning for my honours). I was also considering writing a custom SHACL validator using LDflex that could enforce these typings.

When it comes to generating input fields I was working on a package called LDfields which aimed to modularise and add more customisability to the way fields were generated. The input would be either undefined or an RDF.Term - and possibly a few basic annotations such as the rdf:label. The metadata given to the component would be any constraints in that term (termType, value range, regex, datatype, language, rdf:type, is a label required etc.). The field would then emit an RDF.Term and annotations on an onChange event.
The fields look then customises based in what you are entering (e.g. dropdown for URIs which displayed the rdf:label where possible, customised field for regex constraints, text-box for language tagged literals etc.)

In hindsight these packages should have been built on top of Components.js

I'm happy to discuss this some time as I have spent many months working on this problem and developing tangential tooling as noted in SolidOS/form-playground#9

[danielbeeke]

Hi all,

I am also very interested in this.
I think there are very exciting ideas in all the mentioned form tools.

• Schimatos validation of data with SHACL
• Typescript support for developers would be awesome
• The form generator from the form UI ontology is also a great idea
• rdf-form has custom widgets types so that a developer can add their own widget
• automatic generation of forms by SHACL, Schimatos
• form definitions solid form UI and rdf-form
• referencing an URI via an API or SPARQL source
• object ontology may have a reference to the form definition (basically load object and edit)
• a view mode, rdf-form also has a way of displaying all the properties without form widgets.

In rdf-form we had a concept that would enrich data as a reaction of a specific value in a field:
Imagine a schema:Video, where the schema:url is filled with a Youtube link, the schema:image or schema:poster etc is filled with the image and schema:abstract is filled with the video description.
The same concept we had for books with the OpenLibrary API, fill the ISBN and all the other data was added.

_About definition vs generation _
Automatic generation might work very good for very free form RDF data and knowledge graphs, I think for some other things a form definition would be good.

My background is in Drupal. I used to work mostly with Drupal.
Great form ideas that you have there:

• reference (create a reference (a new thing) in a popup or new tab)
• sub forms
• conditional logic (only show field if a = checked)
• custom field widgets (date range for example, or select staff member in hierarchy tree)
• multilingual support baked in at the field level
• form alters (every little detail of a form can be changed in the PHP runtime). A module would define something and you as a developer can hide or add things.

To note a couple things that I struggled with in rdf-form:

• Translations. At the moment rdf-form support "02-01-2022"@en as a field value. It loses the data type but it gives translations that are enabled at the field level.
• json-ld @list / RDF lists support. I think that items in a list should be ordered and it would be great to be able to create items with multiple fields. Because of this rdf-form has a strange abstraction that is not really clean. I wrestled with creating a widget that can be used for the property schema:image but schema:image may also be an object, in that case you would have schema:url inside of it. And that schema:image would possible have a color or width and height etc. I think it would be very valuable to have a clean way of accessing that kind of data.

I think it might be interesting to have some kind of call with lightning talks and a conversation.
Just to understand more about the different use cases and the solutions that were built before and all the ideas that were not yet built. I would love to give a demo of what we are doing with rdf-form now. I can of course also give some good screenshots.

I would be available for this if wanted.

[danielbeeke]

Below is a screenshot of rdf-form in our testing website hence all the lorem ipsum.

Things on the screenshot:

• references to other things in the CMS (clicking on it navigates to the edit page of those)
• layouts of forms
• sub forms (the access control and the references form are re-usable parts.)

[phochste]

@danielbeeke that looks great. I'll work on some use-cases . But this looks exactly like the form generation I had in mind

[phochste]

See also https://libstore.ugent.be/fulltxt/RUG01/003/008/246/RUG01-003008246_2021_0001_AC.pdf and https://gitlab.ilabt.imec.be/data-validation/schimatos-authorization

[phochste]

@jeswr @danielbeeke would you be interested to find in the comming weeks some timeslot to discuss what the state of the art is with these tools and to see what would be possible?

[jeswr]

would you be interested

@phochste - for sure; if you pick any time between 5pm-12pm AEDT I should be free to attend :) (my email is [email protected])

https://gitlab.ilabt.imec.be/data-validation/schimatos-authorization

What is this? Is there a way for me (as a non-IDlab member) to view it?

https://libstore.ugent.be/fulltxt/RUG01/003/008/246/RUG01-003008246_2021_0001_AC.pdf

Haven't read the full thing yet - but I would highlight that this is based on a very old version of the tool (which I wrote at the end of the first year of my degree; whilst learning JS and React at the same time) - that is to say; the new version that I plan on releasing is a complete rewrite, and written much better internally.

[danielbeeke]

@phochste yes me also. Great! I am quite flexible regarding time. All the timeslots @jeswr named are fine for me. My email is [email protected]

[phochste]

As discussed by @danielbeeke it would be great to have form generation and storage of RDF / nonRDF not tightly coupled to Solid but available as a storage plugin. E.g. using Comunica to figure out how and where to store the data.

There are implementation and research challenges still in these types of applications. Certainly regarding how the handling of RDF and nonRDF can be both be handled in formal way and how all the different kinds of existing libraries can be integrated.

Creating pure RDF has also its challenges when updating data. E.g. in context of generating bibliographical data it is quite important to have the ordering of fields correct (e.g. ordering of author names of a publication).

There are also implementation concerns how these forms can align with current front-end frameworks.

It would be good to investigate what research challenges are and implementation challenges to find out the best way to allocate resources to a project like this.

[phochste]

CLEF https://polifonia-project.github.io/clef/ is a server side application (with a SPARQL backeend) that creates these types of forms and interface for crowd sourcing applications. The project uses a JSON document as a template for the generated fields:

[
 {
  "id": "1644228165691",
  "value": "Literal",
  "placeholder": "My beautiful title",
  "type": "Textbox",
  "label": "title",
  "property": "http://schema.org/title",
  "disambiguate": "True",
  "prepend": "A textbox with a literal value, flagged as main title of the record (mandatory)",
  "browse": "False",
  "searchWikidata": "False",
  "searchGeonames": "False",
  "disabled": "False",
  "class": "col-md-11",
  "cache_autocomplete": "off"
 },
 ...
 ]

[CopChristophe]

Last year, we developed a small demo survey app.
If interesting enough, I can share the code.

Here is the background info and the link to the demo applications.

https://www.konsolidate.eu/stories/building-a-survey-application-with-personal-data-vaults-solid-project
https://survey-creator.demo.konsolidate.eu/
https://survey-app.demo.konsolidate.eu/

[phochste]

@CopChristophe Yes that survey app is the type of workflow we had in mind:

• Form Builder app (1) that is a creator of a form description
• Form Viewer app (2) that is a viewer of the form and can store results in a Solid Pod
• The response itself has provenance information about the form and generation

What we miss is that the form description is not in a standard vocabulary. When e.g. SHACL is used, then Form Builder app(1) and Form Viewer app(2) can be built by anyone in the world. What we would also would like to see is in the form description is optional information where the results should be posted. In your survey app this is hard coded in the same container as where the form can be found. In our case that can be different.

[phochste]

Here is a proof of concept of a browser based Form Viewer (2) app that can used @danielbeeke rdf-form and a published form to store data in a Solid pod. https://github.com/phochste/FormViewer

[github-actions]

Please provide a status update about this challenge. Every ongoing challenge needs at least one status update every 2 weeks. Thanks!

[phochste]

Had meetings with Daniël Beeke and the Solidos team (Timea Turdean & Jeff Zucker) about possible overlap between the rdf-form, frm and solid-ui projects. In the month of August we'll have a student joining us on this project.

It would be a great example also for a schema alignment experiment /cc @andreipopi .

[github-actions]

Please provide a status update about this challenge. Every ongoing challenge needs at least one status update every 2 weeks. Thanks!

[pheyvaer]

@phochste What is the status of this challenge?

[phochste]

• @smessie will be working for two weeks on this challenge
  • His main focus will be to create a drag&drop environment to create RDF forms definitions (to be decided which vocabulary will be used)
  • He will create this RDF form generator as a separate app to prove this challenge support a decentralized setup where the RDF form generation, RDF form input, RDF data, RDF Form, RDF storage and RDF viewer can be made and developed in an independent way
• @danielbeeke has been working on a SHACL based version of this RDF redering tools
• I've been working on adding some features in @danielbeeke 's rdf-from to make it easier to support schema alignment
  • An example how schema alignment can be used is available in solidui2rdfform
  • This is a Notation3 rule set to transform solid-ui into rdf-form
  • The reason for this is to investigate how RDF form rendering implementations that use seperate vocabularies can be integrated

[pheyvaer]

@phochste Considering the different things you mention, does it make sense to split this up in different challenges?

[pheyvaer]

@phochste Looking at the newly created challenges, can we turn this challenge then into a scenario?

[phochste]

@pheyvaer done

[pheyvaer]

@phochste What needs to be done for this scenario to be complete? The two linked challenge are completed.

[phochste]

I don't know what the criterions are to close a scenario. From a users-perspective I won't call this a solved issue. From a developers perspective, yes, the challenges have status completed.

Maybe we can compare the scenario as a duck hunting season. The season of 2022 is closed.

[pheyvaer]

@phochste Are there challenges that need to be solved? If not then I would consider this scenario done.

[pheyvaer]

@phochste Ruben V still has to review it though 😉

[josephguillaume]

From my perspective, I think there are still at least two solid/RDF specific challenges that still need to be tackled.

A form generator needs to validate the form it loads. As it stands, the nominated solution https://github.com/smessie/FormGenerator silently ignores features it doesn't support, and makes implicit assumptions about the structure of data within the loaded document. While this is fine in a closed system and satisfies the stated acceptance criteria for that challenge, it's problematic within an open ecosystem where forms may use custom elements.

In addition, this scenario currently assumes the user will choose a vocab for the form generator, and that form renderers need to support multiple form vocabs.
It would be useful to add a challenge that uses inference to automatically translate between the form vocabs/between different shapes. As discussed elsewhere, in the long term this inference could be either a server side or client side feature, but in the context of this scenario, it would be useful to provide a proof of concept for interoperability (building on existing alignment work between the vocabs).

[phochste]

@josephguillaume Yes indeed that is a good observation. Currently @smessie is working on his master thesis project that tackles certainly your second point. The plan is to put an N3 reasoning component in every Form Generator and Form Viewer app that will allow for schema+data validation/alignment and provide hints what should happen next when the submit button is pressed in these applications (declaratively state how data should be stored in a backend system).

There is already server based N3 available, there are also already some client based N3 implementations. We hope that during this project we also will see a full EYE N3 reasoner on the web (at Inrupt @jeswr has a working prototype).

These are very broad subjects I would be happy to see a proof of concept.

[pheyvaer]

@josephguillaume We can still finish this scenario and open new challenges (as the ones you mention). We have to be aware though that a scenario/challenge doesn't keep expanding until it never will be finished. Therefore, if the acceptance criteria are satisfied, we close this specific scenario. People can start new scenarios/challenges based on extra features that are needed when looking at the output from this scenario.

[RubenVerborgh]

All good for me for this scenario, thanks all!

[pheyvaer]

You find the report for this scenario here.