Using HedgeDoc to help generate RSS (Podcasting 2.0)

[elreydetoda]

Feature Description

Based on my comment in Matrix:

You mentioned posting it to a GitHub repo from your notes doc...what if (tl;dr version) we could reach into the notes doc and just grab all the episode information...🤔
That way you don't have to publish the information anywhere else because you (and JB team) already did all the hard work to create that notes doc. Plus, it's already in Markdown, so the only thing would be to make sure it's in the expected format (we can talk more about that) and let the automation take care of the rest 😁

Summary

I wanted to see if it was possible to use HedgeDoc's API (the current JB note platform) to grab the JB show notes and then we can parse the markdown file to generate the RSS feed from that document.

Details

So, it would act just like the scrapper to where it's an automated process in the GH repo, but could be manually invoked (via GUI or API). This process would just be yet another way to push data to the repo so we can deploy episodes (we'll have to figure out about the scrapper collisions)

IIRC, JB uses Hedgedoc and it looks like they have an API. So there could be a single note and maybe that note has the ID or link to the show which needs to be publish + a date/time it need to be published by. This would help control which episode is getting released and making sure it's not getting published too early. This also helps with the not disclosing information on a public repo too early, because you don't have anything you need to do 🙃

Small Security Side-note

There is technically an attack vector here for someone who has write access to the repo and trying to pull other notes from the Hedge doc instance, but we already have that with the SSH key to a JB server. (BTW as we expand the JB website's reach into the JB infra this is going to be more important, and I would be happy to help set it up with you and the team - this will help prevent the attack because people with write vs people who can approve can be different)

[elreydetoda]

Some more convo around it in matrix:

message:

I do like this line of thinking, I do kind of recall us debating and we had sort of settled on publishing a subset of information to GitHub but I think it might be worth revisiting that discussion.

Because there is a real nice simplicity to your idea for the hosts.

message:

ya, and that's the format thing I was thinking about. We could just take a portion of the document if we want to (like after a specific header), so you could leave the rest as notes in a format that works for you and the team.

message:

One aspect to keep in mind would be what JB wants to keep private in a show note (doesn't happen often, but occasionally does), and how curating links (and link descriptions, etc) is an important aspect of a notes-to-show process.

But, overall this sounds fun/amazing/great/efficient!

chrislas
We been thinking about creating like a meta data section at the top of the doc
Like a specific example for the links for that episode that we want to publish publicly

brentgphoto
right!
frontmatter..

elrey741
ya, I love all these details 😁

Yep, in the end (while extremely important) I think all this can be worked around because it'll just be another python script which reads markdown and we should be able to slice up the data as we see fit 🥳

brentgphoto
Just Another Python Script™

elrey
BTW, I (at least personally) would love to know what other things you all have chatted about as potential solutions. Maybe those topics/convos could help drive other things for the website in the future as well 🥳😁

plus maybe have those discussions with other people in the community, someone else would've come up with this faster than me just now thinking about it (honestly can't believe I didn't think of it sooner 😞) 😅

chancem
So like front matter in hugo

brentgphoto
yep! and we already kinda do this.. so not a far stretch at all

elrey741
yes and no, mainly because we'll essentially having them create a "whole" doc inside of a doc. So, it'll be frontmatter (maybe for meta data section it will be) but not for the content + it'll be after some identifier we determine so we're not grabbing more than we need to and exposing internal (or potentially sensitive) information.
I'd say we can use the 🦒 emoji as the "identifier", but we've already used it for a show name and tag 🤣😁

brentgphoto
DARN! What a miss.. ; )

chrislas
Yeah I agree we need to sync up on that ✅
What I've been thinking would be the most productive is if Wes, Brent and my self could jump on a call with you
elrey741
But that's tricky from a scheduling standpoint. Sounds like we have some time set aside this afternoon and so I'll see if West can make that too.

[elreydetoda]

Also, BTW I created a quick postman collection (folder) + environment (environment variables) of the Hedgedoc API.

https://www.postman.com/elreydetoda/workspace/jb-apis/

From a minimum PoC (Proof of Concept) perspective this can work, we'd pull in the markdown via download request. From there we'd parse it based on the frontmatter/deliminator, which is still up for conversation around.

(also, just FYI) Postman is a tool developers can use to create an share API. An API the mechanism we can interact with Hedgedoc from an automated perspective via scripts. (typically you can do so via curl as well, as an example)

If you want to test this out, you can pretty quickly spin up a Hedgedoc PikaPod which will only bill you for how much time you use and IIRC gives you a $15 initial credit. (I talk about them on my blog if you want to know more) It's how I did my research for this issue.