examples PD_INSTR_DETECTOR

[rowlesmr]

from #124

should PD_INSTR_DETECTOR have _pd_instr_detector.instr_id to link the detectors to the instrument?

[jamesrhester]

As there could be multiple detectors on a single instrument, we have to link to instrument from detector. So yes, we definitely need this link for the examples to work.

[rowlesmr]

Added. Should it be a key?

Can the same detector (_pd_instr_detector.id) in a different instrument (_pd_instr.id) be considered the same detector? or is it a different detector?

I think we can infer from Brian's choice to use _pd_instr. data names for both PD_INSTR and PD_INSTR_DETECTOR, that the detector and instrument side of things are supposed to be parsed by the end-user as the same thing, so having it as a key ensures it is autopopulated, and doesn't need to be explicitly specified.

[jamesrhester]

As there could be multiple detectors on a single instrument, we have to link to instrument from detector. So yes, we definitely need this link for the examples to work.

I'm not sure this is true, if pd_instr_detector does not include the actual detector(s). Are there any cases that we know of with multiple detectors (in the DIFFRN_DETECTOR sense of a 2d module) and multiple distinct values in the pd_instr_detector category? e.g. differing slits/tertiary collimators based on 2theta angle? In which case we should link pd_instr_detector with a diffrn_detector_module.

[rowlesmr]

Excepting the choice to interpret detector_id as "channel no", I had envisioned detector_id refering to an entire flat panel, or linear PSD, or something of that like.

.

A complex detector setup that springs to mind is the JEEP beamline at diamond. with it's energy-dispersive detector

Here, there are 23 energy-sensitive detectors, all at the same 2Th angle, but at different azimuthal angles. So, you're combining many different physical detectors with the pdCIF choice of thinking of energy-dispersive channels as "detectors".

[jamesrhester]

A complex detector setup that springs to mind is the JEEP beamline at diamond. with it's energy-dispersive detector

Here, there are 23 energy-sensitive detectors, all at the same 2Th angle, but at different azimuthal angles. So, you're combining many different physical detectors with the pdCIF choice of thinking of energy-dispersive channels as "detectors".

I've taken the chance to figure out how raw data should be presented in the below as well.

What "Detector" means in the core CIF/mmCIF sense can be determined by the non-key data names - if any of them change, then it's a different detector. I see type, phosphor layer thickness, gain, deadtime, area resolution in pixels/mm are included in the diffrn_detector category by imgCIF+mmCIF. Detectors are composed of diffrn_detector_elements, meaning panels or modules. Binary data that make up the image are mapped to elements, these element images are composed into frames.

A complete post-mono instrumental description consists of 1 pd_instr, to which potentially multiple _pd_instr_detectors point, describing the post-sample optics before the potentially multiple diffrn_detectors, one per _pd_instr_detector.

I think we need:
_pd_instr_detector.instr_id - which instrument the detector is attached to, not a key data name
_pd_instr_detector.id - child of _diffrn_detector.id and also key to this category. Doing it this way is essentially the same as adding all of the _pd_instr_detector data names to diffrn_detector. These detectors may be composed of multiple elements as per imgCIF.
_pd_diffractogram.detector_id instead of _pd_diffractogram.instr_id (see below).

Answers to questions:

Can the same detector (_pd_instr_detector.id) in a different instrument (_pd_instr.id) be considered the same detector? or is it a different detector?

Yes. So a different pre-sample config, but the same post-sample config.

Modelling multi-analyser detectors:

This seems to be one of the thrusts of the original powder dictionary: analysers in front of scintillators for high-resolution work. In this case each of the analyser-detector combinations would be a separate detector as the analyser angles could be slightly different. Note that diffrn_detector is a Set category, so each detector description would, by default, be in a separate data block.

** Important change **

In the multi-analyser example, each scintillator will produce its own diffractogram. From this we deduce that each diffractogram is associated with a detector, not with an instrument. So we shouldn't have _pd_diffractogram.instr_id, instead we should have a _pd_diffractogram.detector_id. Otherwise we can't present raw data, only data after consolidation. This also mirrors the way in which imgCIF works, with the raw data associated with a detector element, not with a source or instrument.

** Modelling flat panel detectors **

Just use imgCIF, and the raw data are not diffractograms. A diffractogram can be associated with a detector (potentially composed of multiple elements) but it will be processed data.

Modelling TOF

Thinking of an instrument where there are multiple detectors at different azimuths.

If the detector modules all have the same _pd_instr_detector optics, then we could suppose there is one detector consisting of multiple elements. But that creates a problem: a TOF detector might produce 1D data, but that raw 1D data is associated with a detector element, not with a detector overall as per "important change" above. So we are forced to say that every TOF detector is individual, and then our _pd_diffractogram.detector_id link makes sense; otherwise we have to have a _pd_diffractogram.element_id everywhere instead. Therefore I conclude that multiple modules when they each produce 1D raw data may not be grouped together into a single detector.

[rowlesmr]

This will require thought on my behalf, but initial reaction, re:

** Important change **

In the multi-analyser example, each scintillator will produce its own diffractogram. From this we deduce that each diffractogram is associated with a detector, not with an instrument. So we shouldn't have _pd_diffractogram.instr_id, instead we should have a _pd_diffractogram.detector_id. Otherwise we can't present raw data, only data after consolidation. This also mirrors the way in which imgCIF works, with the raw data associated with a detector element, not with a source or instrument.

_pd_meas.detector_id exists. So you can do:

data_meas
_pd_diffractogram_id  MANY_DETECTORS

loop_
_pd_data.id
_pd_meas.detector_id
_pd_meas.2theta_scan
_pd_meas.counts_total
1   A   10.00   1234
2   B   10.00   1235
3   C   10.00   1324
4   A   10.01   1236
5   B   10.01   1324
6   C   10.01   1324
# ...

data_proc
_pd_diffractogram_id  MANY_DETECTORS

_pd_proc.info_data_reduction "averaged each detector"

loop_
_pd_data.id
_pd_proc.2theta_corrected
_pd_proc.intensity_total
01   10.00   1264.333
02   10.01   1295.667
# ...

I have a feeling that also alters the use of detector_id as a channel num proxy in energy-dispersive (or any type, really) detectors.

[rowlesmr]

My example is solely based on the multi-analyser example, but you could also consider combining a linear-PSD detector with an area detector - they could both sit on the same instrument at the same time, collecting data from the same incident beam and sample, but you'd be hard pressed to put that data in the same loop.

So yes, we do need to think on how the diffractogram, instr, and detector interact.

[jamesrhester]

_pd_meas.detector_id exists.

Oops. And it is supposed to point to _pd_instr_detector.id, which I'm proposing is a _diffrn_detector.id. The intent, as @rowlesmr says, is to capture multi-detector banks in a single diffractogram, but _diffrn_detector.id coming from a Set category would stop this happening.

So, if we want to preserve backwards compatibility, we're best off saying that pd_instr_detector may only be a 1D detector, and does not share a key data name with diffrn_detector. It is true enough that pd_meas can only be data from a 1D detector. Therefore, no _pd_instr_detector.id = _diffrn_detector.id.

We would still need to trace how a pd_proc diffractogram arose from a 2D detector, so perhaps we have a _diffrn_detector.diffractogram_id pointing to the diffractogram that has been derived from this detector. That way we're allowing more than one 2D detector to contribute to a single diffractogram, just in case those TOF bods decide to record 2D TOF (therefore a 3D block of data). Is it possible for more than one diffractogram to be derived from a single detector?

[jamesrhester]

My example is solely based on the multi-analyser example, but you could also consider combining a linear-PSD detector with an area detector - they could both sit on the same instrument at the same time, collecting data from the same incident beam and sample, but you'd be hard pressed to put that data in the same loop.

Yes, the original design for the Australia Synchrotron PD beamline had both Mythen and analyser-scintillators potentially operating simultaneously. As all of those are 1D detectors, the data from them would be tabulated together under our current proposal, which is OK.

[rowlesmr]

So, if we want to preserve backwards compatibility, we're best off saying that pd_instr_detector may only be a 1D detector, and does not share a key data name with diffrn_detector. It is true enough that pd_meas can only be data from a 1D detector. Therefore, no _pd_instr_detector.id = _diffrn_detector.id.

re your last sentence, you mean _pd_instr_detector.id != _diffrn_detector.id ?

Indeed, re your desire to say that pd_instr_detector may only be a 1D detector, in the powder CIF chapter, Brian says:

The CIF syntax, as opposed to the STAR File syntax, is not well suited to han-
dling large multi-dimensional data sets. For some two-dimensional image formats,
this deficiency was addressed by the development of imgCIF (see Chapters 2.3 and
3.12). It is also true that the CIF syntax is not well suited to storing unprocessed
powder-diffraction measurements from the many instruments that use area detectors
(particularly for the case of the three-dimensional data structures needed for modern
TOF instruments). Even in these cases, however, diffraction intensities are commonly
reduced to simpler representations, such as might be input to a Rietveld refinement
program. The pdCIF definitions are intended for use with these reduced data sets.

[rowlesmr]

Just flagging that _diffrn_detector.* data names are mentioned a few times in the powder chapter.