Document system, modularization, and interaction life-cycle

[jywarren]

Hi @rexagod - Linking to discussion here: https://publiclab.org/notes/rexagod/03-11-2019/gsoc-proposal-mapknitter-orb-descriptor-w-auto-stitching-pattern-training-and-live-video-support-and-ldi-revamp-major-ui-enhancements#c22147 and https://publiclab.org/notes/rexagod/03-11-2019/gsoc-proposal-mapknitter-orb-descriptor-w-auto-stitching-pattern-training-and-live-video-support-and-ldi-revamp-major-ui-enhancements#c23688

So perhaps the docs might outline something like:

1. initialize matcher
   • inputs:
   • outputs: object with methods findPoints() and findPointMatches()
   • link to code
   • notes: takes X seconds to initialize
2. findPoints(img)
   • inputs:
   • outputs: points in format: [{...},{...}]
   • link to code
3. findPointMatches(img1points, img2points)
   • inputs:
   • outputs: matches in format: [{...},{...}]
   • link to code

Creating good documentation (see https://www.npmjs.com/package/inline-markdown-editor for some really simple but specific documentation) will help us make good decisions too!

[rexagod]

@jywarren I've added proper documentation adhering to the standards above. Let me know what do you think.

[jywarren]

Hi, the docs are really nice! Fantastic work! I wonder if an even simpler non-technical intro sentence or two is worthwhile, something like your one-liner at the top of the page (Pattern-mining module for detecting key-points in images 🎈):

matcher-core is a JavaScript-based pattern-mining module for detecting key-points in images. It accepts 2 image URLs, and returns a collection of points that match between the two images.

Performance - Also thinking - could there be a brief section on performance - like, roughly, what parts take the longest, and some sense of how long one might expect small or large images to take at each major stage. Like, "a few seconds" or "almost instantly" or "up to 60 seconds".

Also, do you resize the images before running this, for optimization? What if the images are really huge? We should note if you're resizing, or if this is an option.

Oops, it looks like the demo is offline? https://rexagod.github.io/matcher-core/demo/

Great work on this!!!!!

[jywarren]

Ah, and one more thing, sorry! Let's add a section almost at the top discussing uses - like, we can link to LDI and say "here's one use case, and an example of how integration can work" Make sense?

[rexagod]

@jywarren I think,

roughly, what parts take the longest, and some sense of how long one might expect small or large images to take at each major stage. Like, "a few seconds" or "almost instantly" or "up to 60 seconds".

I think e7fbf6f satisfies this? What do you think? Should I go in deeper and add detailed and more specific info for this, or is it okay?

Also, do you resize the images before running this, for optimization? What if the images are really huge? We should note if you're resizing, or if this is an option.

@jywarren As I stated in my proposal, all images are pyrdowned to 640 by 480 and then evaluated. I thought about including this info in the docs but was afraid that this would be too much "in-depth" for the common user who just wants the points. What do you think? Should I add this to the readme?

like, we can link to LDI and say "here's one use case, and an example of how integration can work" Make sense?

Definitely! And the image-sequencer one too!

[jywarren]

I think the docs are looking good. Is there an easy way to set the image downscale size as a parameter?

…

On Sat, Jul 20, 2019, 3:28 PM Pranshu Srivastava ***@***.***> wrote: @jywarren <https://github.com/jywarren> I think, roughly, what parts take the longest, and some sense of how long one might expect small or large images to take at each major stage. Like, "a few seconds" or "almost instantly" or "up to 60 seconds". I think e7fbf6f <e7fbf6f> satisfies this? What do you think? Should I go in deeper and add detailed and more specific info for this, or is it okay? Also, do you resize the images before running this, for optimization? What if the images are really huge? We should note if you're resizing, or if this is an option. @jywarren <https://github.com/jywarren> As I stated in my proposal, all images are pyrdown <https://docs.opencv.org/2.4/doc/tutorials/imgproc/pyramids/pyramids.html>ed to 640 by 480 and then evaluated. I thought about including this info in the docs but was afraid that this would be too much "in-depth" for the common user who just wants the points. What do you think? Should I add this to the readme? like, we can link to LDI and say "here's one use case, and an example of how integration can work" Make sense? Definitely! And the image-sequencer one too! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2?email_source=notifications&email_token=AAAF6J435PFNYNUPQIN2R53QANRKDA5CNFSM4HYZBVYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2NUQUY#issuecomment-513493075>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAF6JZ344FNEQRP4YLACQ3QANRKDANCNFSM4HYZBVYA> .

[rexagod]

matcher-core/src/orb.core.js

Line 142 in 92d54fd

let i = 640 * 480;

@jywarren The algorithm is optimized best for 640 by 480 images and increasing this will only increase the computational overhead, while decreasing these dimensions could cause loss of accuracy. I'd suggest not exposing these params to the user since it could very well cause produce errors (incorrect ratios, etc.) or simply wrong results (gaussian pyramids of smaller sizes are susceptible to overlaps), unless there is some advantage to this that I'm not seeing, or maybe some measure by which this could be prevented.

Also, all this "downsizing" is done in the virtual canvas that's not appended to the DOM, just created, evaluated, and destroyed, while the original image is used for overlay manipulations.

What do you think?

[rexagod]

Was thinking about this and maybe we can pass in ratios smaller than 640:480? Although I'm not sure if that would be a solid implementation as it is right now so let me run some test cases for this and get back to you!

[rexagod]

Okay, so I just tried this, and the overhead was neglible (even for 1080p images, or higher!!), which can be due to the performance upgrade that was incorporated. I never knew we had optimized the algorithm to such extent! Also, I've tested out this on small canvas sizes too (till 352x240) and was not able to find any outliers, but this may become a problem when the two images are very similar, almost identical in nature and the user expects a larger set of (accurate) points where these smaller pyramids may generate lesser and inaccurate results.

Perhaps we can add a warning to alert users that smaller dimensions imply lesser points (in rare cases) just to be on the safe side, or maybe restrict them to resolutions equal to, or above 480p?

[jywarren]

OK, so why don't we default to some strong default, and add a few other suggested dimensions and some of the text from this issue as guidance (or just link to it?) so that people can make good decisions with your extra context here, yeah! Docs can specify the ideal constraints. And we can even say "if not sure, just stick to the defaults!" :-)

[rexagod]

Done! Please have a look!