You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The stats generation process relies heavily on having access to the git executable (using the simple-git library).
However, that is not the most user-friendly way, and cuts off this feature from many who are not aware of it, or who are not comfortable with a CLI.
Investigate if there is a way to compute the repository stats from the extension.
GitHub's API should allow getting the list of tags
Getting a list of files and date edited for each file is possible, but would be really expensive (my personal text-hoarder-store has 10k files already - that's A LOT of API calls)
Other options:
Make the extension download the repository as an archive and see if all the necessary information can be extracted from there
Alternatively, the UI could prompt the user to drag-and-drop their checked out repository - though that is less user-friendly than getting it automatically
Do stats computation on the back end - I would be really against this both because of the privacy implications and because of the need to maintain a server
The text was updated successfully, but these errors were encountered:
Update:
Was able to use a compression stream to download gzip file and uncompress it
The resulting file is a .tar archive - was able to unpack that using a few libraries
However, the archive does not preserve the modified date for files, nor does it preserve the .git folder, making stats extraction harder.
The .zip github endpoint similarly returns an archive without modified dates preserved
The git tree and git blob github API endpoints can return many entires at once, but they don't include create/modified date
The git commit endpoint contains created/modified date but would require making an API call per each commit (10k requests in case of my personal text-hoarder-store)
I do not want to abuse github's api
There is isomorphic-git, a web-based git client, but due to github's cors issues, it requires a proxy server to download a repository - I don't want that burden.
Solutions:
Use github's .tar.gz endpoint and provide limited stats (without tags or created dates). I can still get the year created as it's part of the file path, and it happens to be the most important date. And then the web page would tell users that more stats are available in the CLI and give them docs for the CLI
Keep it simple for now - I definitely won't be using the limited web-based stats when a cli is available. Plus, a CLI would provide other valuable features like file pre-processing for TTS so CLI would be useful. And the maintenance burden of having web-based stats and cli stats is extra overhead.
Thus, I will push my code for working with .tar.gz to a separate branch for now, and instead create good docs for working with the CLI. The extension does require some technical knowledge after all, since it's dealing with github repositories, and that was my intention from the beginning - that's the target audience I am going for, as there are already plenty of "reader mode" or "pocket" type extensions that very closed ecosystem and limited or even paid, but don't require technical knowledge.
The stats generation process relies heavily on having access to the git executable (using the simple-git library).
However, that is not the most user-friendly way, and cuts off this feature from many who are not aware of it, or who are not comfortable with a CLI.
Investigate if there is a way to compute the repository stats from the extension.
GitHub's API should allow getting the list of tags
Getting a list of files and date edited for each file is possible, but would be really expensive (my personal text-hoarder-store has 10k files already - that's A LOT of API calls)
Other options:
The text was updated successfully, but these errors were encountered: