ToDo List #174
Comments
|
I saw there was some discussion last year of integrating with Pinboard. Is this still a desired feature? (I personally would like it). If so, I can take it. Pinboard has a nice API with an endpoint for fetching all bookmarks and recent bookmarks (up to 100), so an import to Buku feature at least seems doable. |
|
I have to think about it. Whether we really want to integrate or want to go solo. ;) |
|
Gotcha, sounds good! |
|
I was going through the Pinboard docs. They do support export/import to/from html. I think we can refrain from adding some service specific code to authenticate to pinboard, probably along with some auth token hanging around. Buku is a solution in itself, not an ad-hoc to any solution. Now, coming to the integration part with popular web services, why don't we just document the fact that Buku supports bookmarks.html? That should be enough to have all your bookmarks in from anywhere (that too, incrementally). Yes, it's a manual procedure. But it's safer/smarter than accessing any random web service's data directly. |
|
Yeah that's reasonable, especially given that we can't integrate with every service. Sounds like import via bookmarks.html format should satisfy most services. Didn't know about the html format being the standard for bookmarks, interesting. (I might write some separate small pinboard-to-buku script using https://user:[email protected]/v1/posts/all?format=json endpoint to allow for a straight dump into buku) Where are you thinking of documenting the support for bookmarks.html? I could add a line in the Import section of the Operational Notes wiki, just to be explicit about it. Should it go elsewhere too? |
Of course!
You are in the right place! |
|
Additionally, in the Introduction, please make the following modification: "For GUI integration (or to sync bookmarks with your favourite bookmark management service), refer to the wiki page on System integration." |
# imported collections module for named tuple # add "colormap" from googler project # add named tuple for id and title, url, desc, tag # adjusted print_single_rec so colors can be changed for each item # add valid color string checker from googler # adjusted man options # add autocompletions
|
@alex-bender the colors support is alredy being worked on by @shv-q3 here. He hijacked this but he is almost done! ;) I changed the owner. EDIT: Now I remember he attempted to do this once before with colord. The current approach has been our preferred approach. |
|
@mosegontar I have added the note on web service integration. |
|
Ah thanks; (funny timing, I was just about to do this). I've updated the man page (and I'll update the wiki), PR coming in a second. |
|
Now I see what has happened. I had the README change locally, forgot to push it and went to sleep. :) Pushed it now. |
# sorted import in alphabetical order # adjusted var name's # add color table to man page
This reverts commit f4d56ea.
# sorted import in alphabetical order # adjusted var name's # add color table to man page # add --colors option to readme "Usage" section
# imported collections module for named tuple # add "colormap" from googler project # add named tuple for id and title, url, desc, tag # adjusted print_single_rec so colors can be changed for each item # add valid color string checker from googler # adjusted man options # add autocompletions
# sorted import in alphabetical order # adjusted var name's # add colors table to man file # add color os ENV option # add --color documentation in README under "Usage"
* implement "Support custom colours" from (#174) # imported collections module for named tuple # add "colormap" from googler project # add named tuple for id and title, url, desc, tag # adjusted print_single_rec so colors can be changed for each item # add valid color string checker from googler # adjusted man options # add autocompletions * Hotfix - implement "Support custom colours" from (#174) # sorted import in alphabetical order # adjusted var name's # add colors table to man file # add color os ENV option # add --color documentation in README under "Usage" * fix typo error in man page color section * Update buku-completion.bash --color reordered in alphabetical order * Update buku.fish --color reordered in alphabetical order * Update _buku --color reordered in alphabetical order * Hotfix2 - implement "Support custom colours" - restructure feature # separated id from ID_str and ID_DB_STR so it can have separate color # add 5 string color for (id, title, url, desc, tag) # adjusted print_single_rec for 5 colors result # fixed syntax .BI \--colors " COLORS" in man page # replaced four with five letter string for color # fixed default colors to "GKlxe" and "\x1b[2m" for DB index # add example to README and man page # removed check for colorstr_env set by user for the moment * Fixed reviews * Review fixes
|
@alex-bender @mosegontar how is it going? Planning to make a release early next month, probably in the first week. Can we expect to have the features assigned to you in? |
|
@jarun sorry I've been a bit delayed. I'm moving this week and things have been hectic. I've updated about half of With regard to generating the documentation with Sphinx, I experimented a bit with it last week. I should be able to do it, but I do expect there will be some finicky aspects. Also wrt autogenerating the docs, that's something I still need to look into. So this part might not be finished in time for the release. |
|
Hi @jarun! Im going back from vacation right now so cant say for sure for now. I'll ping you in a few days. |
|
Thanks guys! |
|
@mosegontar I can see it's going very well! 👍 |
|
Just an update, docstrings are updated up to the Editor Mode Functions, so about 80% complete. Hoping to be done by tomorrow or Tuesday! |
|
Simply awesome! |
|
Hi @jarun First, amazing work. I love using Buku. I know there is a |
|
|
|
I'm thinking something with aiohttp to fetch page. But looking at the code, not sure how feasible this is and if this is maintainable It also only used when importing a lot of bookmark which is not essential. So I'm still not sure about this |
The thread is more of a propaganda than having any real meat. Here's my take:
|
|
I had several URLs in my notepad and I couldn't find a way to import them directly into buku. The add option accepts only one URL. So first I add them with an add-on in firefox, then export all the bookmarks and finally import them into buku. The problem was that it took some time because not only the new bookmarks were imported but also some thousands bookmarks which were already in firefox. So, what about importing from a text file which contains URLs separated by new lines or spaces or semicolons? Or even batch import from the console? |
It would be a very small script in ANY scripting language (shell/python/perl...). Please write it yourself. |
|
@mosegontar @rachmadaniHaryono can you guys audit the add bookmark path and see if there's any way to optimize the performance anywhere? It can be reduction of condition checks, loops, less variables, anything... It's not that we are slow, but it's always great to be audited (other than IRS). |
|
Sure 👍 i'll take a look |
|
two suggestions:
[edit] found python plugin concept, will make an implementation as proposal |
This is definitely a new plugin/project. Please create one and I'll add a mention to it. |
Are you creating a plugin framework or a plugin in this case? The framework would be a part of Buku, the plugin will be a separate project. |
|
See also http://yapsy.sourceforge.net I'm thinking the plugin workflow should be like this
Buku just have to provide a way to enable/disable plugin. See project beet with their plugin system |
|
I think it would be greatly useful. Please collaborate and go ahead. Please add an issue and a branch ( @questor I am adding you as a collab. Please accept and work on the branch for plugin framework. |
|
I have created the branch Guys, I'll be away for a few days till next Sun and won't be active other than responding to mails (family time ;)). For the plugin fw branch please do peer code review, merge and work together. |
|
okay, to give you some more insights what I want to achieve and what workflow I have in mind some explanations:
I have already started with a very simple plugin system, but I need some help with the decisions where to store additional information from or in the plugin (external table or in the current table). But I agree, the plugin-framework has to be part of buku, but the plugins are separate projects. |
|
btw: the template changes are really simple and does not interfere with the other options. the changes can be seen here: https://github.com/questor/Buku/commit/eb4b8c745f793eb631842e290a650d3a8259e6ff |
|
I will need some time to think about it. At the same time, the plugins should be as detached as possible. |
|
I think a plugin should have its own database for its own data. The foreign key should be the URL which is unique in Buku DB. Also, for any changes to Buku database the plugin should use Buku APIs or request. |
|
Let's say the user updates description through plugin. It should be in the desc field of Buku. For fields which are available both Buku and plugins work on the same field. That is, no data duplication. |
|
@rachmadaniHaryono are there any test cases for shorten and expand? I believe for well-known services like google the shortened url is always the same. Can you please add the test case if not in place already? This we ensure we know if the tny.im service is active. |
|
@jarun sorry it may take some time as i focus on other things on these recent days |
|
Sure! Please take your time. |
|
@questor are you working on the plugin framework? |
|
Also @questor, please raise a PR for the template stuff so we can review it. |
|
I'm thinking through the stuff and the things I want to have (which of course might not necessarily is the same the rest want to have). my current idea is to have a template-output (already done, will make a PR later) with the possibility to have a static webpage with the output and some javascript code to make filtering in the page (something like filtrify). I opted for the "template-export" orthogonal to "plugin-concept" approach simply because it's easier to code as I'm not a python-guru myself. but I can push what I have so far (but I have to admit it's not that much up to now). the more extensive option would be to make the template-export somehow aware of plugins and make the export query the plugins for more information (more work, but cleaner design). |
|
If the requirement is to have a template from which a static webpage can be generated I don't think we would really need a plugin. Have the template and build it up by fetching data from the webpages separately. What one really needs is the links. Why would we need to store additional data for this requirement in the Buku database. If you want a cool page, you need to ave the thumbnails which would need page fetches anyway. |
|
okay, you are thinking about a datascraper-plugin and another export-plugin or something combined thinking about an export-plugin using the buku-db-url, downloading some more data and doing the export? my approach would have been to make separate scrape-plugins (with the tables in the main-database) and an generic export-plugin where you can specify from where to take the data to insert into the page (and the possibility to specify multiple tables in one page). To get things going I try to do an complete example to be able to transport my idea what I want to have and then you can decide if you want to go that way or another one. |
|
My idea is pretty straight for this plugin (set of plugins): get the urls from the DB and do whatever you want with your own data. |
|
Rolled at #233. |
Continued from #135.
Notes
The list below is a growing one. While suggesting new features please consider contributing to
Buku. The code is intentionally kept simple and easy to understand with comments. We'll be happy to assist any new contributor. We need your help!Some of the fresh-baked features may not have been released yet. Grab the master branch for those.
Identified tasks
profiles.ini(see Firefox auto-import breaks if profile name does not end with .default #212, thanks @alex-bender)--formatin search results (ref, thanks @mosegontar)importdb()]The text was updated successfully, but these errors were encountered: