Fetcher manipulator: SpecificSet
Fetcher manipulator that that limits an MIK run to a specific set of objects based on a list of record keys. Useful for testing and QA purposes, or for regenerating a subset of Islandora ingest packages from a larger set of input data.
MIK's --limit parameter applies as if this manipulator were absent. In other words, if the identifiers listed in the input file match records retrieved within the limit, they are included in the set processed by MIK; if not, they are excluded from the set processed by MIK. Since the speicifc set is by definition a limit on how many records are processed, in practice the --limit parameter would not normally be used in conjuction with this manipulator.
Can be used within any toolchain (i.e., is not specific to CONTENTdm, CSV, etc.).
To register this manipulator in your toolchain, add the following line to the "[MANIPULATORS]" section of your .ini file:
fetchermanipulators[] = "SpecificSet|/tmp/record_ids.txt"
This manipulator takes a single mandatory parameter: the path to the file containing the identifiers for the records you want MIK to process.
An optional parameter, 'exclude', will cause the identifiers in the input file to be excluded from the fetcher. This may be useful if you want to process all but a specific set of objects. To use use this, append '|exclude' to the end of the path to the input file, e.g.,
fetchermanipulators[] = "SpecificSet|/tmp/record_ids.txt|exclude"
The identifiers for each record that are identified as 'record_key' in the .ini file are retrieved from a plain text file contianing one ID per line. The path to this file is this manipulator's first parameter, e.g., fetchermanipulators[] = "SpecificSet|/tmp/record_ids.txt".
The input file should contain a list of CONTENTdm pointers, CSV row IDs, or whatever field is defined in the .ini file as the record_key, one ID per line. Comments are allowed in this file, using a '#' at the start of comment lines. A sample file (for example, '/tmp/record_ids.txt' referenced in the previous paragraph) that contains four CONTENTdm pointers is:
# Sample file containing pointers to CONTENTdm objects.
3589
3590
3645
3649
If you want to reprocess a set of objects that were identified in the problem log, MIK provides a script (extras/scripts/specificsetfromproblemrecords.php) that will let you produce an input file for this fetcher manipulator from the problem log.
Content on the Move to Islandora Kit wiki is licensed under a Creative Commons Attribution 4.0 International License.
