Dataview compatable metadata? #158

Galphanore · 2022-05-25T19:11:56Z

Galphanore
May 25, 2022

Is there any way to get this plugin to format the metadata it gets from kindle in a "Dataview" readable format? Basically, two colons instead of one and no apostrophe around the date. So, for instance, instead of:

---
kindle-sync:
  bookId: '20575'
  title: 'Dynamic Story Creation in Plain English: Drake''s Brutal Writing Advice'
  author: Maxwell Alexander Drake
  asin: B01IGSWNIO
  lastAnnotatedDate: '2019-05-01'
  bookImageUrl: 'https://m.media-amazon.com/images/I/719pcrXjAjL._SY160.jpg'
  highlightsCount: 125
---

It would be:

---
kindle-sync:
  bookId:: '20575'
  title:: 'Dynamic Story Creation in Plain English: Drake''s Brutal Writing Advice'
  author:: Maxwell Alexander Drake
  asin:: B01IGSWNIO
  lastAnnotatedDate:: 2019-05-01
  bookImageUrl:: 'https://m.media-amazon.com/images/I/719pcrXjAjL._SY160.jpg'
  highlightsCount:: 125
---

Maybe as a toggle? It would make importing notes and integrating them into my vault structure much easier.

As it stands now, if I manually change it to be dataview compatible, then the next time I re-sync it will make duplicates of the book because it can't identify that the file is the same once I change the metadata.

hadynz · 2022-05-28T02:46:41Z

hadynz
May 28, 2022
Maintainer

kindle-sync is a special and reserved front matter section that the plugin requires to function correctly.

Currently there is no way to do what you are asking for. Your only option is to manually modify every file and add a custom section for use with Dataview. The plugin will respect it.

That said, as part of implementing #136, I will make sure that I support your use case. So consider your feature request being worked on.

1 reply

Galphanore May 28, 2022
Author

Awesome, thanks.

hadynz · 2022-05-29T02:44:31Z

hadynz
May 29, 2022
Maintainer

@Galphanore can I confirm one thing with you.

Two colons in frontmatter is an invalid syntax. All frontmatter must have one colon and it will work well with Dataview.

However, outside of frontmatter, using double colons will allow information to be parsed by Dataview. Is that accurate?

The above example has created some doubt, and I am not a heavy Dataview user.

3 replies

Galphanore May 29, 2022
Author

Weirdly, in front matter, one colon works with Dataview. However, if there are spaces in front of the tag in the frontmatter it treats it as if it is outside it. So...

---
kindle-sync:
  bookId:: '20575'
  title:: 'Dynamic Story Creation in Plain English: Drake''s Brutal Writing Advice'
  author:: Maxwell Alexander Drake
  asin:: B01IGSWNIO
  lastAnnotatedDate:: 2019-05-01
  bookImageUrl:: 'https://m.media-amazon.com/images/I/719pcrXjAjL._SY160.jpg'
  highlightsCount:: 125
---

Would work with Dataview, but so would

---
kindle-sync:
bookId:'20575'
title: 'Dynamic Story Creation in Plain English: Drake''s Brutal Writing Advice'
author: Maxwell Alexander Drake
asin: B01IGSWNIO
lastAnnotatedDate: 2019-05-01
bookImageUrl:'https://m.media-amazon.com/images/I/719pcrXjAjL._SY160.jpg'
highlightsCount: 125
---

Basically, if the only thing you changed was to remove the leading spaces in the kindle-sync section, then it would make it compatible with dataview. As long as there are leading spaces, you have to use two colons to get dataview to read it.

Beyond that weirdness, yes, outside frontmatter two colons can be used to read data with dataview. That's what I'm doing right now. I was just hoping for a way to not have to manually replicate it each time I import notes.

hadynz May 29, 2022
Maintainer

Sweet as. Then my understanding is correct.

The feature that I am working on now is allowing you to set your own custom front matter template by exposing all data about the book that is there. I will also be working on allowing you to customise the entire template of the file, allowing you to use double colons outside of front matter if you need.

I will leave the kindle-sync front matter with its indented content as it is. This is propriety state information that the plugin requires. It is much simpler to have it stand on its own.

Galphanore May 29, 2022
Author

Awesome! That sounds like it would be perfect for my use case as well as a wide variety of other ones. Thanks.

hadynz · 2022-06-06T11:18:16Z

hadynz
Jun 6, 2022
Maintainer

I've implemented your request in version 1.6.0. When you go to the plugin settings, you will be able to customise the default template used for note highlights. You should be able to add a YAML front matter section, or add double colons as you wanted to play well with Dataview.

If you can confirm this release works well for you that would be great.

5 replies

Galphanore Jun 6, 2022
Author

I'm testing it out but running into a weird issue where when I try to sync it just shows this:

instead of giving you the options of selecting syncing to kindle or uploading a file like it did before. It's showing the same thing regardless of whether I have my own custom values in the "File Template" section. When I first updated, it worked about two times. I tried adding a custom template, it started doing the above, and now any time I do it gives the same results. Even if I uninstall the module and reinstall it.

Here's the last file template I attempted to use before things went awry:

# {{title}}
## Metadata
Author:: {{author}}
Pages:: {{pages}}
Publisher:: {{publisher}}
Processed:: No

## Highlights
{{highlights}}

hadynz Jun 7, 2022
Maintainer

Can you toggle developers tools and share any errors that you maybe seeing there?

Galphanore Jun 8, 2022
Author

Can you toggle developers tools and share any errors that you maybe seeing there?

Dataview: version 0.5.26 (requires obsidian 0.13.11)
VM161:16 Kindle Highlights plugin: loading plugin 6/8/2022, 10:05:08 AM
VM161:16 Dataview: all 70 files have been indexed in 0.34s (70 cached, 0 skipped).
VM161:327 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'toString')
    at Je (eval at <anonymous> (app.js:1:1494667), <anonymous>:327:50197)
    at Object.p (eval at <anonymous> (app.js:1:1494667), <anonymous>:327:50052)
    at V (eval at <anonymous> (app.js:1:1494667), <anonymous>:327:35122)
DevTools failed to load source map: Could not load content for app://obsidian.md/main.js.map: System error: net::ERR_FILE_NOT_FOUND

hadynz Jun 9, 2022
Maintainer

This could be a tricky one to debug.

Can you try the latest version that I just published - 1.6.1.

If the problem still exists, can you send me any console errors again please. To make sure that you only send me errors related to the Kindle Plugin, can you:

(a) disable Kindle Highlights
(b) clear your console logs
(c) re-enable Kindle Highlights

Galphanore Jun 9, 2022
Author

Ok, so I updated to 1.6.1. Still got the issue so I disabled Kindle Highlights, cleared the console log but left it open and then re-enabled it. As soon as I re-enabled it without even closing the "community plugins" window it put this in the console log:

DevTools failed to load source map: Could not load content for app://obsidian.md/main.js.map: System error: net::ERR_FILE_NOT_FOUND

Then I closed the community plugins and tried to run Kindle Highlights Sync. As soon as I clicked on the sync button, it gave this while opening the kindle sync window:

plugin:obsidian-kindle-plugin:327 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'toString')
    at Je (plugin:obsidian-kindle-plugin:327:50224)
    at Object.p (plugin:obsidian-kindle-plugin:327:50079)
    at V (plugin:obsidian-kindle-plugin:327:35122)

So I closed the window and tried to click sync again. The second time, it didn't even open the kindle sync window and the log added:

Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'toString')
    at Je (plugin:obsidian-kindle-plugin:327:50224)
    at qe (plugin:obsidian-kindle-plugin:327:49224)
    at ne (plugin:obsidian-kindle-plugin:327:36900)
    at new Xe (plugin:obsidian-kindle-plugin:327:50417)
    at Ve (plugin:obsidian-kindle-plugin:327:50486)
    at ne (plugin:obsidian-kindle-plugin:327:36900)
    at new Ze (plugin:obsidian-kindle-plugin:327:51029)
    at Array.It (plugin:obsidian-kindle-plugin:330:4624)
    at Pt (plugin:obsidian-kindle-plugin:330:5093)
    at ne (plugin:obsidian-kindle-plugin:327:36900)
    at new Rt (plugin:obsidian-kindle-plugin:330:5728)
    at Bt.show (plugin:obsidian-kindle-plugin:330:6234)

Any further attempts to click the sync button just duplicate that last error. So, I turned on safe mode, then back off again and repeated the steps you requested with the same results. So, to narrow things down further, I individually disabled all other community plugins (I use "Calendar", "Dataview", and "Templater" so it's not a lot, but I wanted to be thorough.

So, I disabled those three, uninstalled kindle highlights, exited out of obsidian, relaunched it and then reinstalled kindle highlights with a blank console log.

Doing this gave the exact same results as above, with an added message about kindle highlights styles.css not being found when I installed the plugin.

Kindle Highlights: styles.css not found
plugin:obsidian-kindle-plugin:16 Kindle Highlights plugin: loading plugin 6/9/2022, 11:25:49 AM
DevTools failed to load source map: Could not load content for app://obsidian.md/main.js.map: System error: net::ERR_FILE_NOT_FOUND
plugin:obsidian-kindle-plugin:327 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'toString')
    at Je (plugin:obsidian-kindle-plugin:327:50224)
    at Object.p (plugin:obsidian-kindle-plugin:327:50079)
    at V (plugin:obsidian-kindle-plugin:327:35122)
Je @ plugin:obsidian-kindle-plugin:327
p @ plugin:obsidian-kindle-plugin:327
V @ plugin:obsidian-kindle-plugin:327
Promise.then (async)
q @ plugin:obsidian-kindle-plugin:327
eval @ plugin:obsidian-kindle-plugin:327
eval @ plugin:obsidian-kindle-plugin:327
o @ plugin:obsidian-kindle-plugin:327
eval @ plugin:obsidian-kindle-plugin:327
Promise.then (async)
e @ plugin:obsidian-kindle-plugin:327
eval @ plugin:obsidian-kindle-plugin:327
subscribe @ plugin:obsidian-kindle-plugin:327
S @ plugin:obsidian-kindle-plugin:327
w @ plugin:obsidian-kindle-plugin:327
$e @ plugin:obsidian-kindle-plugin:327
ne @ plugin:obsidian-kindle-plugin:327
Xe @ plugin:obsidian-kindle-plugin:327
Ve @ plugin:obsidian-kindle-plugin:327
ne @ plugin:obsidian-kindle-plugin:327
Ze @ plugin:obsidian-kindle-plugin:327
It @ plugin:obsidian-kindle-plugin:330
Pt @ plugin:obsidian-kindle-plugin:330
ne @ plugin:obsidian-kindle-plugin:327
Rt @ plugin:obsidian-kindle-plugin:330
show @ plugin:obsidian-kindle-plugin:330
await in show (async)
showSyncModal @ plugin:obsidian-kindle-plugin:335
eval @ plugin:obsidian-kindle-plugin:335
r @ plugin:obsidian-kindle-plugin:335
plugin:obsidian-kindle-plugin:327 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'toString')
    at Je (plugin:obsidian-kindle-plugin:327:50224)
    at qe (plugin:obsidian-kindle-plugin:327:49224)
    at ne (plugin:obsidian-kindle-plugin:327:36900)
    at new Xe (plugin:obsidian-kindle-plugin:327:50417)
    at Ve (plugin:obsidian-kindle-plugin:327:50486)
    at ne (plugin:obsidian-kindle-plugin:327:36900)
    at new Ze (plugin:obsidian-kindle-plugin:327:51029)
    at Array.It (plugin:obsidian-kindle-plugin:330:4624)
    at Pt (plugin:obsidian-kindle-plugin:330:5093)
    at ne (plugin:obsidian-kindle-plugin:327:36900)
    at new Rt (plugin:obsidian-kindle-plugin:330:5728)
    at Bt.show (plugin:obsidian-kindle-plugin:330:6234)

So, I created an entirely new vault. Completely empty, nothing in it. Only thing I did in this new vault was install kindle highlights. When I did this, it showed the expected "Choose a sync method..." window with the amazon cloud and upload my clippings file options but also gave this error:

POST https://o46028.ingest.sentry.io/api/6073196/envelope/?sentry_key=590b8641240b49bfba55e4662b0b0149&sentry_version=7 429
eval @ plugin:obsidian-kindle-plugin:335
a @ plugin:obsidian-kindle-plugin:16
eval @ plugin:obsidian-kindle-plugin:335
add @ plugin:obsidian-kindle-plugin:335
Ri._sendRequest @ plugin:obsidian-kindle-plugin:335
Ii.sendEvent @ plugin:obsidian-kindle-plugin:335
Xr.sendEvent @ plugin:obsidian-kindle-plugin:335
i._sendEvent @ plugin:obsidian-kindle-plugin:335
ls._sendEvent @ plugin:obsidian-kindle-plugin:335
eval @ plugin:obsidian-kindle-plugin:335
eval @ plugin:obsidian-kindle-plugin:16
eval @ plugin:obsidian-kindle-plugin:16
_executeHandlers @ plugin:obsidian-kindle-plugin:16
eval @ plugin:obsidian-kindle-plugin:16
a @ plugin:obsidian-kindle-plugin:16
a.then @ plugin:obsidian-kindle-plugin:16
i._processEvent @ plugin:obsidian-kindle-plugin:335
i._captureEvent @ plugin:obsidian-kindle-plugin:335
i.captureEvent @ plugin:obsidian-kindle-plugin:335
_._invokeClient @ plugin:obsidian-kindle-plugin:1
_.captureEvent @ plugin:obsidian-kindle-plugin:1
h.finish @ plugin:obsidian-kindle-plugin:1
f.finish @ plugin:obsidian-kindle-plugin:1
eval @ plugin:obsidian-kindle-plugin:1
r @ plugin:obsidian-kindle-plugin:335
setTimeout (async)
eval @ plugin:obsidian-kindle-plugin:335
f @ plugin:obsidian-kindle-plugin:1
eval @ plugin:obsidian-kindle-plugin:1
aa._createRouteTransaction @ plugin:obsidian-kindle-plugin:335
eval @ plugin:obsidian-kindle-plugin:335
eval @ plugin:obsidian-kindle-plugin:335
f @ plugin:obsidian-kindle-plugin:16
a.onpopstate @ plugin:obsidian-kindle-plugin:16

However, it let my sync my data. Since it was a new vault, it was doing all my highlights from scratch and not updating any existing highlights but it worked fine. So I let it download everything and then added the same file template I listed in my previous comment and synced it again.

# {{title}}
## Metadata
Author:: {{author}}
Pages:: {{pages}}
Publisher:: {{publisher}}
Processed:: No

## Highlights
{{highlights}}

It didn't update any of the existing notes since they didn't change, so I deleted one and re-synced. It added the lines from my template, but only filled out "Author", not pages or publisher. So I deleted all my notes and re-downloaded them again. None of them have pages or publisher listed, so maybe those template variables are not working correctly? authorURL doesn't work either.

Either way, with an entirely new vault it works fine. It's just error-ing with my existing vault, and I'm not really sure why. Even recreated the folder-tree structure in the new vault to make sure it wasn't a folder path problem.

OogieM · 2022-06-18T13:20:44Z

OogieM
Jun 18, 2022

I also want toadd some dataview metadata but I'm stuck on not even being able to edit the file template. I'm sure it's a stupid user error but I can't copy the existing one out nor can I figure out how to actually add something to it.

2 replies

intrnal Aug 2, 2022

Same here. Can't edit and add to the default template. Seems like I could erase the entire default template but I can't find a copy to add it back if I do erase it. If someone can make a copy of their default template can they post it here. I'm on windows.

hadynz Aug 2, 2022
Maintainer

Sounds like you are describing the issue that has been outlined here #174

I'll try and create time this week to fix it since it is blocking so many people.

Seems like I could erase the entire default template but I can't find a copy to add it back if I do erase it

Also sounds like giving you a way to revert to default template would also be handy!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dataview compatable metadata? #158

{{title}}

Replies: 4 comments 11 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Dataview compatable metadata? #158

Galphanore May 25, 2022

Replies: 4 comments · 11 replies

hadynz May 28, 2022 Maintainer

Galphanore May 28, 2022 Author

hadynz May 29, 2022 Maintainer

Galphanore May 29, 2022 Author

hadynz May 29, 2022 Maintainer

Galphanore May 29, 2022 Author

hadynz Jun 6, 2022 Maintainer

Galphanore Jun 6, 2022 Author

hadynz Jun 7, 2022 Maintainer

Galphanore Jun 8, 2022 Author

hadynz Jun 9, 2022 Maintainer

Galphanore Jun 9, 2022 Author

OogieM Jun 18, 2022

intrnal Aug 2, 2022

hadynz Aug 2, 2022 Maintainer

Galphanore
May 25, 2022

Replies: 4 comments 11 replies

hadynz
May 28, 2022
Maintainer

Galphanore May 28, 2022
Author

hadynz
May 29, 2022
Maintainer

Galphanore May 29, 2022
Author

hadynz May 29, 2022
Maintainer

Galphanore May 29, 2022
Author

hadynz
Jun 6, 2022
Maintainer

Galphanore Jun 6, 2022
Author

hadynz Jun 7, 2022
Maintainer

Galphanore Jun 8, 2022
Author

hadynz Jun 9, 2022
Maintainer

Galphanore Jun 9, 2022
Author

OogieM
Jun 18, 2022

hadynz Aug 2, 2022
Maintainer