As part of my bachelor thesis I implemented a software prototype containing two components:

1. An desktop component based on Express (Node.js) which acts as a server (this repository)
2. An Android app based on Flutter which acts as a client (corresponding repository)

The concept

• People use more and more digital media but may be forced (or prefer) to work with paper in some areas
• It would be nice to have a resolution mechanism from paper to digital content - something like a QR code but suitable for handwriting
• Instead of a QR code let the user define a short word which can be written on paper and assign it an URL (1)
• The mobile app can be used to scan the written word (2), whatever content is associated with the assigned URL will automatically be displayed in a suitable fashion

(1):

(2):

Additionally the concept describes a mechanism to refer to a specific part of digital content (like a page number for PDF files, or a timestamp for video content - also implemented). This mechanism is also suitable for handwriting.
Further details can be found in my thesis (only available in German).

Implementation

Mobile app:

• Obtain OCR text of the written word via Google Cloud Vision
• Check which URL was assigned to the word and send it to the desktop component
  • Server availability in the local network is discovered via Android NDS and communication is handled with a simple REST call

Desktop component:

• Applies some processing based on the MIME type of the resource and resolves the URL
• Displays the content in a locally installed web browser