Skip to content

Latest commit

 

History

History
executable file
·
18 lines (15 loc) · 3.02 KB

stack.md

File metadata and controls

executable file
·
18 lines (15 loc) · 3.02 KB
Raw

History of Stack: CMR, Solr, CAP, Metadatabus

The Metadabus was created to automate management of data that was being loaded into Solr for CAP.

  • Code for the Canadiana Access Portal (CAP) started earlier, but first commit to the Subversion repository (which was copied to Git and GitHub) was on April 26, 2010.
  • Version 1.0 (2010-08-20) of the Canadiana Metadata Repository (CMR) data schema was announced by William Wueppelmann on Mon, 20120-12-20 via a blog article, and made available on the webiste.
  • The Canadiana Discovery Portal (CDP) and its API was presented at the second Code4Lib North (McMaster University, May 5-6, 2011)
  • The Beta.canadiana.ca URL became Search.canadiana.ca, and a redirect was kept for many years (The word "beta" didn't have meaning outside software people, so many thought it was an official service).
  • CMR version 1.2 of the XML Schema Definition (XSD) is published November 2012.
  • The schema which Solr used to store and index data is CMR plus additional fields.
  • At this point CMR records were created and loaded via a large series of separate normalization/crosswalk scripts, sometimes several per metadata contributor. When the structure of an AIP was created as part of new OAIS archiving, a cmr.xml file (already crosswalked from the metadata record other Canadiana staff supplied) was part of the AIP (See AIP definition )
  • It was decided we wanted to automate the loading of the Solr core, and to have the metadata crosswalks run at load time rather than earlier to allow us to enhance the crosswalks and have that apply to content previously ingested into our OAIS platform.
  • As part of creating the Metadatabus, the CDP codebase and Solr core was separated from what was being used by all the other portals in the spring of 2016.
  • For efficiency, documents not needed for search were split out into a separate CouchDB database: cosearch was loaded into Solr, while copresentation was made available to CAP directly.
  • The CDP didn't receive new updates for several years, and eventually in 2018 it was decided to decomission that service.
  • While there have been other minor design changes, the core of CMR and CAP have remained: What is searchable via CAP, and the format that metadata is crosswalked to, remains based on CMR records loaded into a Solr database.