BibAuthority: initial release

* BibAuthority: New module containing utility functions, configuration and tests. Adds demo records and new index tables. Adds HowTo guides for admin and hacking pages. (closes inveniosoftware#1602) * BibIndex: When indexing authority-controlled subfields of bibliographic records, enrich the index with terms from a configurable list of subfield values from the referenced authority record. * BibIndex: When looking for recently changed bibliographic records to re-index for a given authority-controlled subfield, reindex all bibliographic records that contain references towards recently updated authority records, even if the bibliographic records in questions have not been recently updated. * BibFormat: Simple HTML for authority record. Links for browsing between dependent authority and bibliographic records. * Other modules: some modifications in other modules as well. Co-authored-by: Grzegorz Szpura <[email protected]> Signed-off-by: Grzegorz Szpura <[email protected]> Tested-by: Tibor Simko <[email protected]>
kaplun · Oct 14, 2013 · f5969cf · f5969cf
1 parent d37ea7c
commit f5969cf
Show file tree

Hide file tree

Showing 59 changed files with 4,662 additions and 75 deletions.
diff --git a/configure.ac b/configure.ac
@@ -571,6 +571,13 @@ AC_CONFIG_FILES([config.nice  \
      modules/bibauthorid/etc/Makefile \
      modules/bibauthorid/etc/name_authority_files/Makefile \
      modules/bibauthorid/web/Makefile \
+     modules/bibauthority/Makefile \
+     modules/bibauthority/bin/Makefile \
+     modules/bibauthority/doc/Makefile \
+     modules/bibauthority/doc/admin/Makefile \
+     modules/bibauthority/doc/hacking/Makefile \
+     modules/bibauthority/lib/Makefile \
+     modules/bibauthority/web/Makefile \
      modules/bibcatalog/Makefile \
      modules/bibcatalog/doc/Makefile \
      modules/bibcatalog/doc/admin/Makefile \

diff --git a/modules/Makefile.am b/modules/Makefile.am
@@ -16,6 +16,7 @@
 ## 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.
 
 SUBDIRS = bibauthorid \
+          bibauthority \
           bibcatalog \
           bibcheck \
           bibcirculation \

diff --git a/modules/bibauthority/Makefile.am b/modules/bibauthority/Makefile.am
@@ -0,0 +1,21 @@
+##
+## This file is part of Invenio.
+## Copyright (C) 2011 CERN.
+##
+## Invenio is free software; you can redistribute it and/or
+## modify it under the terms of the GNU General Public License as
+## published by the Free Software Foundation; either version 2 of the
+## License, or (at your option) any later version.
+##
+## Invenio is distributed in the hope that it will be useful, but
+## WITHOUT ANY WARRANTY; without even the implied warranty of
+## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+## General Public License for more details.
+##
+## You should have received a copy of the GNU General Public License
+## along with Invenio; if not, write to the Free Software Foundation, Inc.,
+## 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.
+
+SUBDIRS = bin doc lib web
+
+CLEANFILES = *~
diff --git a/modules/bibauthority/bin/Makefile.am b/modules/bibauthority/bin/Makefile.am
@@ -0,0 +1,19 @@
+##
+## This file is part of Invenio.
+## Copyright (C) 2011 CERN.
+##
+## Invenio is free software; you can redistribute it and/or
+## modify it under the terms of the GNU General Public License as
+## published by the Free Software Foundation; either version 2 of the
+## License, or (at your option) any later version.
+##
+## Invenio is distributed in the hope that it will be useful, but
+## WITHOUT ANY WARRANTY; without even the implied warranty of
+## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+## General Public License for more details.
+##
+## You should have received a copy of the GNU General Public License
+## along with Invenio; if not, write to the Free Software Foundation, Inc.,
+## 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.
+
+CLEANFILES = *~ *.tmp
diff --git a/modules/bibauthority/doc/Makefile.am b/modules/bibauthority/doc/Makefile.am
@@ -0,0 +1,31 @@
+## This file is part of Invenio.
+## Copyright (C) 2011 CERN.
+##
+## Invenio is free software; you can redistribute it and/or
+## modify it under the terms of the GNU General Public License as
+## published by the Free Software Foundation; either version 2 of the
+## License, or (at your option) any later version.
+##
+## Invenio is distributed in the hope that it will be useful, but
+## WITHOUT ANY WARRANTY; without even the implied warranty of
+## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+## General Public License for more details.
+##
+## You should have received a copy of the GNU General Public License
+## along with Invenio; if not, write to the Free Software Foundation, Inc.,
+## 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.
+
+SUBDIRS = admin hacking
+
+#imgdir = $(localstatedir)/www/img/admin
+
+#img_DATA = authority-records-1.png \
+#           authority-records-2.png
+
+webdoclibdir = $(libdir)/webdoc/invenio/help
+
+#webdoclib_DATA = authority-records.webdoc
+
+#EXTRA_DIST = $(img_DATA) $(webdoclib_DATA)
+
+CLEANFILES = *~ *.tmp
diff --git a/modules/bibauthority/doc/admin/Makefile.am b/modules/bibauthority/doc/admin/Makefile.am
@@ -0,0 +1,24 @@
+## This file is part of Invenio.
+## Copyright (C) 2009, 2010, 2011 CERN.
+##
+## Invenio is free software; you can redistribute it and/or
+## modify it under the terms of the GNU General Public License as
+## published by the Free Software Foundation; either version 2 of the
+## License, or (at your option) any later version.
+##
+## Invenio is distributed in the hope that it will be useful, but
+## WITHOUT ANY WARRANTY; without even the implied warranty of
+## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+## General Public License for more details.
+##
+## You should have received a copy of the GNU General Public License
+## along with Invenio; if not, write to the Free Software Foundation, Inc.,
+## 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.
+
+webdoclibdir = $(libdir)/webdoc/invenio/admin
+
+webdoclib_DATA = bibauthority-admin-guide.webdoc
+
+EXTRA_DIST = $(webdoclib_DATA)
+
+CLEANFILES = *~ *.tmp
diff --git a/modules/bibauthority/doc/admin/bibauthority-admin-guide.webdoc b/modules/bibauthority/doc/admin/bibauthority-admin-guide.webdoc
@@ -0,0 +1,171 @@
+## This file is part of Invenio.
+## Copyright (C) 2010, 2011 CERN.
+##
+## Invenio is free software; you can redistribute it and/or
+## modify it under the terms of the GNU General Public License as
+## published by the Free Software Foundation; either version 2 of the
+## License, or (at your option) any later version.
+##
+## Invenio is distributed in the hope that it will be useful, but
+## WITHOUT ANY WARRANTY; without even the implied warranty of
+## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+## General Public License for more details.
+##
+## You should have received a copy of the GNU General Public License
+## along with Invenio; if not, write to the Free Software Foundation, Inc.,
+## 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.
+
+<!-- WebDoc-Page-Title: _(BibAuthority Admin Guide)_ -->
+<!-- WebDoc-Page-Navtrail: <a class="navtrail" href="<CFG_SITE_URL>/help/admin<lang:link/>">Admin Area</a> -->
+<!-- WebDoc-Page-Revision: $Id$ -->
+
+<h2>Introduction</h2>
+<p>The INVENIO admin can configure the various ways in which authority control works for INVENIO by means of the <code>bibauthority_config.py</code> file. The location and full contents of this configuration file with a commented example configuration are shown at the bottom of this page. Their functionality is explained in the following paragraphs.</p>
+<p><i>For examples of how Authority Control works in Invenio from a user's perspective, cf. <a href="howto-authority">_(HOWTO Manage Authority Records)_</a>.</i></p>
+
+<h2>Enforcing types of authority records</h2>
+<p>INVENIO is originally agnostic about the types of authority records it contains. Everything it needs to know about authority records comes, on the one hand, from the authority record types that are contained within the '980__a' fields, and from the configurations related to these types on the other hand. Whereas the '980__a' values are usually edited by the librarians, the INVENIO configuration is the responsibility of the administrator. It is important for librarians and administrators to communicate the exact authority record types as well as the desired functionality relative to the types for the various INVENIO modules.</p>
+
+<h2>BibEdit</h2>
+<p>As admin of an INVENIO instance, you have the possibility of configuring which fields are under authority control. In the “Configuration File Overview” at the end of this page you will find an example of a configuration which will enable the auto-complete functionality for the '100__a', '100__u', '110__a', '130__a', '150__a', '700__a' and '700__u' fields of a bibliographic record in BibEdit. The keys of the “CFG BIBAUTHORITY CONTROLLED FIELDS” dictionary indicate which bibliographic fields are under authority control. If the user types Ctrl-Shift-A while typing within one of these fields, they will propose an auto-complete dropdown list in BibEdit. The user still has the option to enter values manually without use of the drop-down list. The values associated with each key of the dictionary indicate which kind of authority record is to be associated with this field. In the example given, the '100__a' field is associated with the authority record type 'AUTHOR'.</p>
+<p>The “CFG BIBAUTHORITY AUTOSUGGEST OPTIONS” dictionary gives us the remaining configurations, specific only to the auto-suggest functionality. The value for the 'index' key determines which index type will be used find the authority records that will populate the drop-down with a list of suggestions (cf. the following paragraph on configuring the BibIndex for authority records). The value of the 'insert_here_field' determines which authority record field contains the value that should be used both for constructing the strings of the entries in the drop-down list as well as the value to be inserted directly into the edited subfield if the user clicks on one of the drop-down entries. <!-- The value of the 'sort' key tells INVENIO how to sort the entries in the drop-down list. If a popularity sort is chosen, the drop-down entries will be sorted according to how often the associated authority record is referenced in this particular INVENIO instance. -->Finally, the value for the 'disambiguation_fields' key is an ordered list of authority record fields that are used, in the order in which they appear in the list, to disambiguate between authority records with exactly the same value in their 'insert_here_field'.</p>
+
+<h2>BibIndex</h2>
+<p>As an admin of INVENIO, you have the possibility of configuring how indexing works in regards to authority records that are referenced by bibliographic records. When a bibliographic record is indexed for a particular index type, and if that index type contains MARC fields which are under authority control in this particular INVENIO instance (as configured by the, “CFG BIBAUTHORITY CONTROLLED FIELDS” dictionary in the bibauthority_config.py configuration file, mentioned above), then the indexer will include authority record data from specific MARC fields of these authority records in the same index. Which authority record fields are to be used to enrich the indexes for bibliographic records can be configured by the “CFG BIBAUTHORITY AUTHORITY SUBFIELDS TO INDEX” dictionary. In the example below each of the 4 authority record types ('AUTHOR', 'INSTITUTION', 'JOURNAL' and 'SUBJECT') is given a list of authority record MARC fields which are to be scanned for data that is to be included in the indexed terms of the dependent bibliographic records. For the 'AUTHOR' authority records, the example specifies that the values of the fields '100__a', '100__d', '100__q', '400__a', '400__d', and '400__q' (i.e. name, alternative names, and year of birth) should all be included in the data to be indexed for any bibliographic records referencing these authority records in their authority-controlled subfields.</p>
+
+<h2>Configuration File Overview</h2>
+<p>The configuration file for the BibAuthority module can be found at <code>invenio/lib/python/invenio/bibauthority_config.py</code>. Below is a commented example configuration to show how one would typically configure the parameters for BibAuthority. The details of how this works were explained in the paragraphs above.</p>
+<pre>
+&#35; CFG_BIBAUTHORITY_RECORD_CONTROL_NUMBER_FIELD
+&#35; the authority record field containing the authority record control number
+CFG_BIBAUTHORITY_RECORD_CONTROL_NUMBER_FIELD = '035__a'
+
+&#35; Separator to be used in control numbers to separate the authority type
+&#35; PREFIX (e.g. "INSTITUTION") from the control_no (e.g. "(CERN)abc123"
+CFG_BIBAUTHORITY_PREFIX_SEP = '|'
+
+&#35; the ('980__a') string that identifies an authority record
+CFG_BIBAUTHORITY_AUTHORITY_COLLECTION_IDENTIFIER = 'AUTHORITY'
+
+&#35; the name of the authority collection.
+&#35; This is needed for searching within the authority record collection.
+CFG_BIBAUTHORITY_AUTHORITY_COLLECTION_NAME = 'Authority Records'
+
+&#35; used in log file and regression tests
+CFG_BIBAUTHORITY_BIBINDEX_UPDATE_MESSAGE = \
+    "Indexing records dependent on modified authority records"
+
+&#35; CFG_BIBAUTHORITY_TYPE_NAMES
+&#35; Some administrators may want to be able to change the names used for the
+&#35; authority types. Although the keys of this dictionary are hard-coded into
+&#35; Invenio, the values are not and can therefore be changed to match whatever
+&#35; values are to be used in the MARC records.
+&#35; WARNING: These values shouldn't be changed on a running INVENIO installation
+&#35; ... since the same values are hard coded into the MARC data,
+&#35; ... including the 980__a subfields of all authority records
+&#35; ... and the $0 subfields of the bibliographic fields under authority control
+CFG_BIBAUTHORITY_TYPE_NAMES = {
+    'INSTITUTION': 'INSTITUTION',
+    'AUTHOR': 'AUTHOR',
+    'JOURNAL': 'JOURNAL',
+    'SUBJECT': 'SUBJECT',
+}
+
+&#35; CFG_BIBAUTHORITY_CONTROLLED_FIELDS_BIBLIOGRAPHIC
+&#35; 1. tells us which bibliographic subfields are under authority control
+&#35; 2. tells us which bibliographic subfields refer to which type of
+&#35; ... authority record (must conform to the keys of CFG_BIBAUTHORITY_TYPE_NAMES)
+CFG_BIBAUTHORITY_CONTROLLED_FIELDS_BIBLIOGRAPHIC = {
+    '100__a': 'AUTHOR',
+    '100__u': 'INSTITUTION',
+    '110__a': 'INSTITUTION',
+    '130__a': 'JOURNAL',
+    '150__a': 'SUBJECT',
+    '260__b': 'INSTITUTION',
+    '700__a': 'AUTHOR',
+    '700__u': 'INSTITUTION',
+}
+
+&#35; CFG_BIBAUTHORITY_CONTROLLED_FIELDS_AUTHORITY
+&#35; Tells us which authority record subfields are under authority control
+&#35; used by autosuggest feature in BibEdit
+&#35; authority record subfields use the $4 field for the control_no (not $0)
+CFG_BIBAUTHORITY_CONTROLLED_FIELDS_AUTHORITY = {
+    '500__a': 'AUTHOR',
+    '510__a': 'INSTITUTION',
+    '530__a': 'JOURNAL',
+    '550__a': 'SUBJECT',
+    '909C1u': 'INSTITUTION', # used in bfe_affiliation
+    '920__v': 'INSTITUTION', # used by FZ Juelich demo data
+}
+
+&#35; constants for CFG_BIBEDIT_AUTOSUGGEST_TAGS
+&#35; CFG_BIBAUTHORITY_AUTOSUGGEST_SORT_ALPHA for alphabetical sorting
+&#35; ... of drop-down suggestions
+&#35; CFG_BIBAUTHORITY_AUTOSUGGEST_SORT_POPULAR for sorting of drop-down
+&#35; ... suggestions according to a popularity ranking
+CFG_BIBAUTHORITY_AUTOSUGGEST_SORT_ALPHA = 'alphabetical'
+CFG_BIBAUTHORITY_AUTOSUGGEST_SORT_POPULAR = 'by popularity'
+
+&#35; CFG_BIBAUTHORITY_AUTOSUGGEST_CONFIG
+&#35; some additional configuration for auto-suggest drop-down
+&#35; 'field' : which logical or MARC field field to use for this
+&#35; ... auto-suggest type
+&#35; 'insert_here_field' : which authority record field to use
+&#35; ... for insertion into the auto-completed bibedit field
+&#35; 'disambiguation_fields': an ordered list of fields to use
+&#35; ... in case multiple suggestions have the same 'insert_here_field' values
+&#35; TODO: 'sort_by'. This has not been implemented yet !
+CFG_BIBAUTHORITY_AUTOSUGGEST_CONFIG = {
+    'AUTHOR': {
+        'field': 'authorityauthor',
+        'insert_here_field': '100__a',
+        'sort_by': CFG_BIBAUTHORITY_AUTOSUGGEST_SORT_POPULAR,
+        'disambiguation_fields': ['100__d', '270__m'],
+    },
+    'INSTITUTION':{
+        'field': 'authorityinstitution',
+        'insert_here_field': '110__a',
+        'sort_by': CFG_BIBAUTHORITY_AUTOSUGGEST_SORT_ALPHA,
+        'disambiguation_fields': ['270__b'],
+    },
+    'JOURNAL':{
+        'field': 'authorityjournal',
+        'insert_here_field': '130__a',
+        'sort_by': CFG_BIBAUTHORITY_AUTOSUGGEST_SORT_POPULAR,
+    },
+    'SUBJECT':{
+        'field': 'authoritysubject',
+        'insert_here_field': '150__a',
+        'sort_by': CFG_BIBAUTHORITY_AUTOSUGGEST_SORT_ALPHA,
+    },
+}
+
+&#35; list of authority record fields to index for each authority record type
+&#35; R stands for 'repeatable'
+&#35; NR stands for 'non-repeatable'
+CFG_BIBAUTHORITY_AUTHORITY_SUBFIELDS_TO_INDEX = {
+    'AUTHOR': [
+        '100__a', #Personal Name (NR, NR)
+        '100__d', #Year of birth or other dates (NR, NR)
+        '100__q', #Fuller form of name (NR, NR)
+        '400__a', #(See From Tracing) (R, NR)
+        '400__d', #(See From Tracing) (R, NR)
+        '400__q', #(See From Tracing) (R, NR)
+    ],
+    'INSTITUTION': [
+        '110__a', #(NR, NR)
+        '410__a', #(R, NR)
+    ],
+    'JOURNAL': [
+        '130__a', #(NR, NR)
+        '130__f', #(NR, NR)
+        '130__l', #(NR, NR)
+        '430__a', #(R, NR)
+    ],
+    'SUBJECT': [
+        '150__a', #(NR, NR)
+        '450__a', #(R, NR)
+    ],
+}
+</pre>
diff --git a/modules/bibauthority/doc/hacking/Makefile.am b/modules/bibauthority/doc/hacking/Makefile.am
@@ -0,0 +1,24 @@
+## This file is part of Invenio.
+## Copyright (C) 2011 CERN.
+##
+## Invenio is free software; you can redistribute it and/or
+## modify it under the terms of the GNU General Public License as
+## published by the Free Software Foundation; either version 2 of the
+## License, or (at your option) any later version.
+##
+## Invenio is distributed in the hope that it will be useful, but
+## WITHOUT ANY WARRANTY; without even the implied warranty of
+## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+## General Public License for more details.
+##
+## You should have received a copy of the GNU General Public License
+## along with Invenio; if not, write to the Free Software Foundation, Inc.,
+## 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.
+
+webdoclibdir = $(libdir)/webdoc/invenio/hacking
+
+webdoclib_DATA = bibauthority-internals.webdoc
+
+EXTRA_DIST = $(webdoclib_DATA)
+
+CLEANFILES = *~ *.tmp