Using --citeproc causes problem with commas or periods after quoted text

[gnpan]

pandoc 2.11.3.2
When using citeproc to format a bibliography and the file contains text within quotes followed by a comma or a period, in the resulting pdf the comma is placed within the quotes, not after as is should. This did not happen with previous versions using pandoc-citeproc. If citeproc is not used, then the comma appears correctly after the quote.

MWE:
File test.txt:

A citation: @author_test_2021. 

This is "quoted text", followed by a comma.

This is "quoted text". Followed by a period.

Bibliography:

Bibliography file test.bib:

@article{author_test_2021,
  title = Test,
  author = {Author, A.N.},
  date = {2021-01-01},
  journaltitle = {Journal},
  volume = {1},
  pages = {10--20}
}

Command:

pandoc --citeproc --bibliography=test.bib -o test.pdf test.txt

Result - note that the comma or period is before, not after the second quotation mark:

Result without citeproc - comma/period appear after the quotation, as intended (no bibliography of course):

[jgm]

CSL has a setting in locales determining whether punctuation should go inside quotes. In the csl locale files:

  <style-options punctuation-in-quote="true"/>

For en-US (default locale) this is set to true -- some CSL styles may also set it to true independent of the locale.

Pandoc currently applies this to the whole document, not just to citations. This is why you're seeing this behavior. If you're putting the punctuation outside of quotes because you're targeting British English, then try using lang: en-GB in your document (or --metadata lang=en-GB on command line).

We might think about doing this transformation only on citations, not on the whole document. (But I think maybe the reason we're doing it on the whole document is that we sometimes move punctuation around citations.)

[jgm]

We might think about doing this transformation only on citations, not on the whole document.

Endorsing this. I just ran into this issue myself, writing for a journal which wants punctuation outside of quotes for single quoted ("mentioned") items but inside quotes with doubled quotes. There's no way to handle this if the CSL setting is applied to the whole document.

[jgm]

It is tricky to do this, though. We need to use the citeproc locale-dependent function for moving punctuation inside quotes. This operates on an inline list. We can't restrict it to what's inside citations, because we need to handle the case (in note citations) where the citation and punctuation position have been flipped (so the note follows the punctuation) and the punctuation as a result is now right outside a quote.

[gnpan]

Using --metadata lang=en-GB works OK, but I found it is better to modify my csl file by changing default-locale="en-US" to default-locale="en-GB". In this way you get the required punctuation position but you can keep the default US format, e.g. for dates, in the whole document.
Thanks for your help!

[wlupton]

I followed @gnpan's advice and changed the default locale in the CSL file, but I found that this only works if the lang variable isn't set (previously it was set explicitly to en-US, but I've now commented this out).