User workflow documentation / understanding how components fit together

[clintgibler]

First off, thanks so much for creating promnesia, HPI, grasp, and more! 🙏

The past few months, I've been searching for tool(s) that fit my goals for storing and easily accessing information, and reading your promnesia blog post was like reading a better, fleshed out version of what I had in my head 🙌

I'm starting to play around with the awesome ecosystem of tools you've built, but a few things currently aren't clear.

Apologies for the length of this issue 🤣 I wanted to give you some background of what I'm looking to do and my current understanding of how your various libraries work, so that hopefully the documentation can be fleshed out and made clearer for myself and others.

Happy to help document based on my experiences using it, if that'd be helpful.

Context: My Goals

To give you a bit of context, here's what I'm currently doing and what I'm looking to accomplish.

Info sources I want to store:

• Meeting notes, personal notes, snippets from emails
• Conference talk notes
• Highlights and notes from PDFs, web pages, Twitter
• Articles and their highlights from Pocket

Storage details:

• I'd like the canonical source of all data to be in org mode files in one git repo.
• The notes will likely be structured using some combination of org-roam and PARA.

Functionality/workflow goals:

• I'd like to be able to easily and automatically ingest info from all of the above sources and import them into the central org mode repo.
• When I browse the web, what I've already seen and highlighted should be easily and automatically surfaced, and it should be easy to save/highlight new things.

How it All Fits Together

I think pretty much all of the above can be done (and you're already doing) with the assorted tools and libraries you've built, per your personal infrastructure diagram.

But please permit me to summarize here to make sure I'm understanding things correctly:

• Human Programming Interface (HPI) - Helps export data from a variety of third party services and stores them locally, as JSON/XML/etc.
  • Supports interacting with the downloaded data from any of the supported services as Python objects.
• orger - Given input from HPI, enables exporting the data to org mode files.
  • Not totally sure about ^, but guessing based on this.
  • Supports parsing org mode files and accessing them as Python objects
• grasp - A browser extension that "adds a button/keybinding to capture the current page title and url, possibly selected text, additional comments or tags and adds it into your Org Mode file."
• promnesia - A browser extension that shows you relevant info about the page you're currently viewing.
  • Like which page led to the current page, if you were linked via a Telegram message, which links on a page you've already viewed, overlaying highlights from other services like Instapaper and Pocket, and more.
  • All of the relevant info and context promnesia uses is sourced from HPI (and maybe orger) based on files on your local file system.
• open-in-editor - Enables you to click on links in the promnesia sidebar and open the relevant file in your editor.

Questions

I think one thing that's currently a bit fuzzy for me is the relationship between HPI and orger - what exactly each of them does and how they work together. Sorry if this is obvious, but can you provide any additional details?

Will promnesia show highlights and links that have been captured with grasp?

Can promnesia capture URLs/highlights, or does it just display web info that has been previously captured by grasp/Pocket/other services that have been imported by HPI/orger?

In several of the videos, you use CTRL+Alt+V to show links you've already visited, but I don't see that referenced in the README.org anywhere. Am I missing it? Does promnesia support any other hotkeys?

• It might be useful to have a section listing hotkeys in the README.org, and potentially in the browser extension help page.

Thoughts

I think it'd be really nice to have a concise, opinionated, "here's how to get set up and use all of these parts together" guide, which I'd be happy to help with.

Most (or all) of the info is currently there, but at least to me, it doesn't seem to be all in one place (you have to reference a number of repos and blog posts), and it's not obvious how the core parts fit together, what should be set up in what order, etc.

But if this already exists, please let me know!

Thanks for your time, and for reading til the end 😅

[karlicoss]

Thanks for a long a detailed comment! I'm glad someone dug through all of this and bothered enough to try and understand :)
Happy to help/discuss and please don't hesitate to ask more questions.

re: Context: My Goals

Thanks for this section, it makes it much easier to communicate! :) All makes sense to me and is doable!

One comment:

I'd like the canonical source of all data to be in org mode files in one git repo.

I guess depends on what you mean as 'canonical, but for example, with Pocket in particular -- the central source of Pocket data is always the Pocket backend, so your best bet is to have a 'copy' of Pocket data as org-mode (that's why I call it a Mirror ).

There is an alternative workflow -- you can accept that you're only using Pocket for reading and highlighting (because the interface is clean and nice, mobile app, etc), but then you import these highlights into org-mode and handle them there. This is doable with Orger, for example I'm processing saved Reddit posts and Hackernews this way. The only downside is that this is a one-way process -- for example if you delete the highlight in org-mode, it will still remain in the Pocket app. Two way sync isn't technically impossible, but way more tedious.

re: How it All Fits Together

Human Programming Interface (HPI) Helps export data from a variety of third party services and stores them locally, as JSON/XML/etc.

Sort of! For exporting itself, as far as possible, I'm trying to rely on third party tools. Then HPI only consumes the raw data, normalises, merges it, and exposes as Python objects. That way HPI doesn't have to deal with tokens/network etc, making everything much simpler. I explain more about it here: https://beepb00p.xyz/exports.html#design

orger: given input from HPI, enables exporting the data to org mode files.

Yep, mostly correct! Orger core itself is more of a framework that simplifies converting data to org-mode, but most modules do use HPI at the moment.

orger: Supports parsing org mode files and accessing them as Python objects

Nope! It only deals with generating org-mode, for parsing I'm using orgparse

promnesia - A browser extension that shows you relevant info about the page you're currently viewing.

yep!

promnesia: All of the relevant info and context promnesia uses is sourced from HPI (and maybe orger) based on files on your local file system.

Yep, most modules at the moment are using HPI, but it's not a strict requirement, it's easy to write new modules that use custom data sources. For example, telegram module is using the sqlite database directly (at the moment, this code ideally probably belongs to HPI as well, but it's quite old).

re: Questions

I think one thing that's currently a bit fuzzy for me is the relationship between HPI and orger - what exactly each of them does and how they work together. Sorry if this is obvious, but can you provide any additional details?

Like I mentioned above, most orger modules simply get HPI data as input. For example, hypothes.is module is just 30 lines long thanks to HPI, and all the code that is present is very straightforward and only has to do with rendering org-mode.

But HPI is not a requirement -- for example, here I'm using Orger directly with Telegram API.

Will promnesia show highlights and links that have been captured with grasp?

Yep! E.g. on this screenshot. The highlighting algorithm is very crude and doesn't work 100% of the time, but this is something I hope to improve, potentially collaborating with other open source projects on it, because there is a lot of opportunity to work together.

Can promnesia capture URLs/highlights, or does it just display web info that has been previously captured by grasp/Pocket/other services that have been imported by HPI/orger?

Nope, and I'll likely keep it that way to make sure everything is simple! There are lots of great tools for annotation already. Also when real-time indexing is supported, it would be possible to update Promnesia sidebar instantly after capture without Promensia ever being aware of Orger.

It might be useful to have a section listing hotkeys in the README.org, and potentially in the browser extension help page.

Thanks, agree! I guess I was assuming everyone looked at the browser hotkey settings, it shows which keys are exposed by each extension.

re: Thoughts

Most (or all) of the info is currently there, but at least to me, it doesn't seem to be all in one place (you have to reference a number of repos and blog posts), and it's not obvious how the core parts fit together, what should be set up in what order, etc. But if this already exists, please let me know!

Yep I can see that!
I've been doing kind of a wiki approach so far with my PKM setup and infra map, although it's kinda cryptic.
My hope (and goal!) is that each of the tools makes sense in separation -- e.g. it's not necessary to use HPI to use Orger, or you can use Grasp without using any of the other tools, etc. So hopefully it's not necessary to understand everything at once and people can do that gradually if they want.

But I totally agree it's useful to have a simplified version of the complete setup, with less technical details but at least easy to understand. There are also few more specific diagrams here and here, hard to tell how helpful though!

I think it'd be really nice to have a concise, opinionated, "here's how to get set up and use all of these parts together" guide, which I'd be happy to help with.

Thanks, would be great, even flagging specific bits of documentation that are confusing would be super helpful. And if you set everything up and document it in a post or something, I'd be very happy to link to it, so let me know!

[clintgibler]

Haha thanks for taking the time to read my comment and for your detailed response, I really appreciate it! 🙏

I feel like I understand things a lot better now :)

re: Context: My Goals

I agree with your reasoning and approach to export info from third-party services and not worry about two-way syncing. I think that adds a lot of complexity and isn't worth it.

I think what I want is basically the same as your Reddit/Hackernews workflow:

1. Save article to Pocket
2. Read and highlight in Pocket.
3. Periodically, automatically export data from Pocket locally (e.g ~/raw_data/pocket/...)
4. When new data has been exported from Pocket (or other services), import them into ~/org_notes/pocket/{cool_article|other_article|...}.org

re: How it All Fits Together

For exporting itself, as far as possible, I'm trying to rely on third party tools. Then HPI only consumes the raw data, normalises, merges it, and exposes as Python objects. That way HPI doesn't have to deal with tokens/network etc, making everything much simpler.

Ahh thanks, I feel like I get HPI now. I had read that page before, but it "clicks" more after having poked at things for awhile, rather than just reading it.

re: Questions

Highlighting in promnesia

I was going to ask if there was a way for promnesia to display highlights on web pages of text that I've manually copied into my org notes, not using grasp, but then I tried it and it just worked 😍 🙌

Are there any gotchas I should be aware of regarding the structure of the notes - where the link is included, formatting in org mode that can break the highlighting, etc.?

I added some quick about the hotkeys to the README: #127.

re: Thoughts

My hope (and goal!) is that each of the tools makes sense in separation -- e.g. it's not necessary to use HPI to use Orger, or you can use Grasp without using any of the other tools, etc. So hopefully it's not necessary to understand everything at once and people can do that gradually if they want.

Totally agree that having standalone, useful tools is a great way to go about it 👍

The dataflow diagrams are definitely useful for getting a high level understanding of how a component works and how different pieces fit together. I think what's not currently as clear, at least to me, is:

• Given a 50,000 foot view of the whole system, which components do I set up, and in what order?
• How do I configure them to work well with each other?
• What does day to day usage look like?
• What are some example workflows that use multiple (or all) of the components? (this might already exist in part or in total across existing blog posts/READMEs)

flagging specific bits of documentation that are confusing would be super helpful

Sure, will do!

"here's how to get set up and use all of these parts together"

I thought about this a little more, and I think what I'm getting at is basically to address people who come across your awesome blog posts and tools you've written and think, "This is perfect, I've bought into it, I want to set this up for me and customize it for my workflow."

I really appreciate all of the reasoning, motivation, and details in your blog posts, what I'm thinking is a minimal, stripped down guide (that references other docs/posts as appropriate), that's focuses on just actionable steps.

It might look like an extended, blog / doc page version of this:

1. OK first, we need to start getting all of your info in one place. To do that, we're going to use HPI. Here's how to set it up and here are links to each data source you might want.
2. Now that we have the raw info from external sources, we want to import that data structured in many different ways into our central org mode knowledge base. Here's how to set up Orger to do that.
3. We also want to make it easy to import web content we're viewing, not just from third party sources, so we install grasp.
4. Alright, we've made it easy to aggregate our info from many sources as well as manually, now let's augment how we browse the web given this knowledge. For that, we set up Promnesia.
5. (and probably other steps I've missed or out of order)

Importing web content

I haven't played with grasp much yet, but I'm not sure if it quite meets what I'm looking for. I'll write this out in more detail later, but I wanted to be able to specify an arbitrary number of hotkeys that extract different things on different web pages and store it differently in org mode.

I came across this shortkeys extension which seems like a really flexible way to add a number of hotkeys that each run arbitrary JavaScript (so you can extract whatever you want from the page).

Combining shortkeys with org-roam protocol I think will be pretty neat and powerful. I've literally just started this, but wanted to share as I thought you might find it interesting :)

[karlicoss]

Right, I'm back, sorry it's taking long to reply these days! :)

I think what I want is basically the same as your Reddit/Hackernews workflow

Yep, what you're suggesting is doable with Orger! You can start with existing pocket module, the only change you'd need is making it a Queue rather than Mirror, similarly to a hackernews module, for example. Let me know if you have any issues with that!

Are there any gotchas I should be aware of regarding the structure of the notes - where the link is included, formatting in org mode that can break the highlighting, etc.?

I guess you've discovered some of it: #30 :) As I mentioned, it's very basic, so not 100% relieable -- something I'm going to work on later.

re: re: Thoughts

Thanks, all reasonable suggestions! Another thing that might help here (at least for somewhat technical people) is a docker-compose file capable of bootstrapping Promnesia along with the exporters/etc. #55 (comment)

wanted to be able to specify an arbitrary number of hotkeys that extract different things on different web pages and store it differently in org mode.

Yeah, if you don't mind using org-protocol, it's probably better to use it! My main motivation for grasp was using it from a computer where I didn't have emacs

For browser navigation, I'm using surfingkeys; it also lets you use arbitrary javascript, although I personally haven't played with it that much yet!