support Local-Zo-Url-x field from BibDesk2Zotero_attachments

[maxheld83]

The popular Mac OS BibTex editor BibDesk (unfortunately) stores local attachments via symlinks like so:

@techreport{Abbott-2007-aa,
	Address = {Brussels, Belgium},
	Author = {Abbott, Roderick},
	Class = {States & Markets 2009, Diss Tax & Democracy 2010ff},
	Date-Modified = {2014-06-14 15:13:01 +0000},
	Institution = {European Center for International Political Economy},
	Keywords = {Political Economy,Finance, Markets},
	Title = {Are Developing Countries Deterred from Using the WTO Dispute Settlement System? - Participation of Developing Countries in the DSM in the years 1995-2005},
	Year = {2007},
	Bdsk-File-1 = {YnBsaXN0MDDUAQIDBAUGJCVYJHZlcnNpb25YJG9iamVjdHNZJGFyY2hpdmVyVCR0b3ASAAGGoKgHCBMUFRYaIVUkbnVsbNMJCgsMDxJXTlMua2V5c1pOUy5vYmplY3RzViRjbGFzc6INDoACgAOiEBGABIAFgAdccmVsYXRpdmVQYXRoWWFsaWFzRGF0YV8QE0FiYm90dC0yMDA3LWFhYS5wZGbSFwsYGVdOUy5kYXRhTxEBygAAAAABygACAAAMTWFjaW50b3NoIEhEAAAAAAAAAAAAAAAAAAAA1HG1uEgrAAAADaCXE0FiYm90dC0yMDA3LWFhYS5wZGYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAhSMDPN5C0AAAAAAAAAAAAAQACAAAJIAAAAAAAAAAAAAAAAAAAAAxoZWxkX2xpYnJhcnkAEAAIAADUcaeoAAAAEQAIAADPN4KkAAAAAQAUAA2glwANnl4ADYxKAAmmKwAGYi8AAgBRTWFjaW50b3NoIEhEOlVzZXJzOgBtYXg6AEdvb2dsZSBEcml2ZToAbGlicmFyeToAaGVsZF9saWJyYXJ5OgBBYmJvdHQtMjAwNy1hYWEucGRmAAAOACgAEwBBAGIAYgBvAHQAdAAtADIAMAAwADcALQBhAGEAYQAuAHAAZABmAA8AGgAMAE0AYQBjAGkAbgB0AG8AcwBoACAASABEABIAP1VzZXJzL21heC9Hb29nbGUgRHJpdmUvbGlicmFyeS9oZWxkX2xpYnJhcnkvQWJib3R0LTIwMDctYWFhLnBkZgAAEwABLwAAFQACAAr//wAAgAbSGxwdHlokY2xhc3NuYW1lWCRjbGFzc2VzXU5TTXV0YWJsZURhdGGjHR8gVk5TRGF0YVhOU09iamVjdNIbHCIjXE5TRGljdGlvbmFyeaIiIF8QD05TS2V5ZWRBcmNoaXZlctEmJ1Ryb290gAEACAARABoAIwAtADIANwBAAEYATQBVAGAAZwBqAGwAbgBxAHMAdQB3AIQAjgCkAKkAsQJ/AoEChgKRApoCqAKsArMCvALBAs4C0QLjAuYC6wAAAAAAAAIBAAAAAAAAACgAAAAAAAAAAAAAAAAAAALt}}

Yikes.

Naturally, ZBB and/or Zotero can't deal with that, and you loose all attachments on import.

Happily, @pmcarlton has provided this a wonderful perl script that creates sane relative paths out of the above gibberish, and appends them to a custom Local-Zo-URL-x field, like so:

@techreport{Abbott-2007-aa,
	Address = {Brussels, Belgium},
	Author = {Abbott, Roderick},
	Class = {States & Markets 2009, Diss Tax & Democracy 2010ff},
	Date-Modified = {2014-06-14 15:13:01 +0000},
	Institution = {European Center for International Political Economy},
	Keywords = {Political Economy,Finance, Markets},
	Title = {Are Developing Countries Deterred from Using the WTO Dispute Settlement System? - Participation of Developing Countries in the DSM in the years 1995-2005},
	Year = {2007},
		Local-Zo-Url-1 = {./Abbott-2007-aaa.pdf}
}

Unfortunately, this is still not quite what Zotero and ZBB need, because

• they don't know Local-Zo-Url-x
• they don't appear to like the ./ part of the path.

I have so far search-and-replaced my files to fix this, but I feel dirty doing it.

Any chance ZBB could support such a field "out of the box"?

That would be huge for BibDesk refugees like myself.

---

Ps.: I am aware that @pmcarlton ships a patched bibtex.js with his perl script to deal with these problems, but

• I worry it might be out of date (as the repo warns)
• it's bibtex, not better bibtex.

[retorquere]

Let me get this straight -- the BibDesk authors have their paths stored in bibtex as base-64 encoded binary plist files... wow.

I think I can import Local-Zo-Url-1 (is that 1 fixed or is it an index? If there are more attachments, will those be Local-Zo-Url-2 etc?), but Bdsk-File-1 is out, as I can't (sensibly) decode the plist file.

[maxheld83]

yes, I think 1 is an index.

and yes, the symlinks are baffling to me, too. There are some upsides, if I recall (you can move the attachments without breaking the link or something), but just this gibberish freaks me out.

and, as this goes to show, it really harms portability 😦

[maxheld83]

just fyi: barring a successful close on this issue, I've opened up a PR on the perl script repo that un-symlinks the attachment, just so people know that zbb now accepts Local-Zo-Url-x out of the box, and they can skip the additional step.

[retorquere]

This is still to be tested of course, but yes, once the change is confirmed to be working, that seems sensible.

[retorquere]

Can you give this a go (includes the fix from #664 BTW)?

[maxheld83]

uh no, doesn't seem to work. Should I provide a test case zip?

[retorquere]

That'd be helpful. Please include the attachment and bib file, in the folder structure that you'd normally have them.

[maxheld83]

here you go:

testcase.zip

Error report ID is 7Z5SXVBS

This includes three references, two of which have (two each) attachments.

testcase-new.bib is the file to be imported.

It is generated by the perl script from testcase.bib to unmuck the symlinks.

Let me know if there's anything else I can do to assist.

[retorquere]

Alright, try this

[maxheld83]

Fantastic, works great.

Except for this tiny hickup:

It appears that when there are >1 attachments, those are all called attachment in the Zotero UI.
The underlying assets are still properly named, as can be seen in the screenshot.

Items with one attachment import fine and display the proper filename instead of attachment, as can be seen in the screenshot.

[retorquere]

I'd be a little surprised if it were tied to the number of attachments; when no title is given in the bib file, the title defaults to 'Attachment'.

[maxheld83]

Mmh, I see. I'm not sure this (missing title in bib record) is that is going here.

The record above in question does have a title.

Also, scrolling through my import (>1000 records), I only find attachment named attachments for those few records that have more than 1 attachment and vice versa.

So empirically from my data, the link between attachment name and number of attachments is quite strong.

[retorquere]

I don't doubt your emperical results, but seeing as how I wrote the code... :) When I don't see a title, I default to "Attachment".

I'd theorize that your attachments that aren't named "Attachment" come from jabref imports, which often do have titles for the attachments. The bibdesk attachments don't appear to have a title.

[retorquere]

I currently don't treat the file name as a title, btw.

[maxheld83]

very sorry, I'm a bit confused. When you say attachment titles from jabref imports (never used it), do you mean some kind of metadata of the actual PDF, or inside the *.bib file= (or similar) record?

[maxheld83]

sorry, problem solved: the issue is on the side of the Zotfile addon, which I had also enabled. It is Zotfile that renamed those attachments, but choked on >1 attachments.

ZBB hardly ever created filenames for any of my data, because as you rightly pointed out, there is no title info for the attachment.

So, everything's dandy.

[maxheld83]

Sorry, hit the wrong button; will let you close if you see fit.

Thanks for being so generous with your time and work.

[retorquere]

I don't have access to the attachments during import, I can only tell Zotero where to find them so it will import it. That's really all that the importers can do; prep data and politely ask Zotero to import it. No file access, no database access. So I can't read the title from the attachment.

JabRef and some other bib managers do indeed write attachment metadata into the file field; they put a semicolon-separated list of attachments in there, where each attachment is either a file name, or a colon-separated tuple of filename:title:mimetype. Bare filenames are treated as if they specified the title "Attachment".

[retorquere]

Sorry, missed your last comment. If BBT now works as you want it to I'll merge to master and cut a release.

[maxheld83]

works like a charm; sorry about (my) confusion

[maxheld83]

just for the record, the zotfile issue with multiple attachments that here appeared is filed at jlegewie/zotfile#283

Not a BBT issue.

[maxheld83]

this is now merged, shipped in https://github.com/retorquere/zotero-better-bibtex/releases/tag/1.6.97 and closeable, correct?

[retorquere]

Correct.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.