test coverage for slow evotype classes

[rileyjmurray]

As part of PR #452 we noticed that several "slow" evotype objects had an implementation for __reduce__ that was commented out. Some of these commented out definitions indicated a need to be fixed. That raised the question of if the existing slow evotype code is broken in some way. To start the discussion on this point I ran our unit tests with code coverage enabled. Zooming in on results for evotypes, I get the following

Note: I generated this report by running

pytest -n auto --cov=pygsti --cov-report=html unit

inside the test directory.

Tests results when Evotype.default_evotype = "densitymx_slow"

The default evotype is densitymx as long as the cython extensions correctly compiled. Since we're trying to understand possible bugs in the slow evotypes I re-ran with densitymx_slow as the default evotype. This resulted in four failed unit tests.

============================================================================== short test summary info ==============================================================================
FAILED unit/objects/test_localnoisemodel.py::LocalNoiseModelInstanceTester::test_marginalized_povm - ValueError: substring not found
FAILED unit/objects/test_circuit.py::CircuitMethodTester::test_simulate - ValueError: substring not found
FAILED unit/objects/test_circuit.py::CircuitMethodTester::test_simulate_marginalization - ValueError: substring not found
FAILED unit/protocols/test_vb.py::TestPeriodicMirrorCircuitsDesign::test_design_construction - ValueError: substring not found
ERROR unit/extras/interpygate/test_construction.py::InterpygateGSTTester::test_circuit_probabilities - ValueError: Cannot add an object with evolution type 'densitymx' to a model with one of 'densitymx_slow'
ERROR unit/extras/interpygate/test_construction.py::InterpygateGSTTester::test_germ_selection - ValueError: Cannot add an object with evolution type 'densitymx' to a model with one of 'densitymx_slow'
ERROR unit/extras/interpygate/test_construction.py::InterpygateGSTTester::test_gpindices - ValueError: Cannot add an object with evolution type 'densitymx' to a model with one of 'densitymx_slow'
================================================= 4 failed, 1831 passed, 75 skipped, 11769 warnings, 3 errors in 162.96s (0:02:42) ==================================================

Here's a comparison of coverage when using the densitymx_slow by default (top window) vs the plain densitymx evotype (bottom window):

Click here for details on test failures

====================================================================================== ERRORS =======================================================================================
_________________________________________________________ ERROR at setup of InterpygateGSTTester.test_circuit_probabilities _________________________________________________________
[gw9] darwin -- Python 3.10.13 /Users/rjmurr/miniconda3/envs/pg310/bin/python

cls = <class 'test.unit.extras.interpygate.test_construction.InterpygateGSTTester'>

    @classmethod
    def setUpClass(cls):
        super(InterpygateGSTTester, cls).setUpClass()
        target_op = SingleQubitTargetOp()
        param_ranges = [(0.9,1.1,3)]
        arg_ranges = [2*np.pi*(1+np.cos(np.linspace(np.pi,0, 7)))/2,
                  (0, np.pi, 3)]
        arg_indices = [0,1]
        gate_process = SingleQubitGate(num_params = 3,num_params_evaluated_as_group = 1)
        opfactory = interp.InterpolatedOpFactory.create_by_interpolating_physical_process(
                                target_op, gate_process, argument_ranges=arg_ranges,
                                parameter_ranges=param_ranges, argument_indices=arg_indices,
                                interpolator_and_args='linear')
        x_gate = opfactory.create_op([0,np.pi/4])
        y_gate = opfactory.create_op([np.pi/2,np.pi/4])
    
        cls.model = pygsti.models.ExplicitOpModel([0],'pp')
        cls.model['rho0'] = [ 1/np.sqrt(2), 0, 0, 1/np.sqrt(2) ] # density matrix [[1, 0], [0, 0]] in Pauli basis
        cls.model['Mdefault'] = pygsti.modelmembers.povms.UnconstrainedPOVM(
            {'0': [ 1/np.sqrt(2), 0, 0, 1/np.sqrt(2) ],   # projector onto [[1, 0], [0, 0]] in Pauli basis
             '1': [ 1/np.sqrt(2), 0, 0, -1/np.sqrt(2) ] }, evotype="default") # projector onto [[0, 0], [0, 1]] in Pauli basis
>       cls.model['Gxpi2',0] = x_gate

unit/extras/interpygate/test_construction.py:166: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../pygsti/models/explicitmodel.py:299: in __setitem__
    self.operations[label] = value
../pygsti/models/memberdict.py:302: in __setitem__
    self._check_evotype(value._evotype)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = OrderedMemberDict(), evotype = <pygsti.evotypes.evotype.Evotype object at 0x340a5ad10>

    def _check_evotype(self, evotype):
        if not self.flags['match_parent_evotype']: return  # no check
        if self.parent is None: return
        elif self.parent._evotype != evotype:
>           raise ValueError(("Cannot add an object with evolution type"
                              " '%s' to a model with one of '%s'") %
                             (evotype, self.parent._evotype))
E           ValueError: Cannot add an object with evolution type 'densitymx' to a model with one of 'densitymx_slow'

../pygsti/models/memberdict.py:190: ValueError
____________________________________________________________ ERROR at setup of InterpygateGSTTester.test_germ_selection _____________________________________________________________
[gw9] darwin -- Python 3.10.13 /Users/rjmurr/miniconda3/envs/pg310/bin/python

cls = <class 'test.unit.extras.interpygate.test_construction.InterpygateGSTTester'>

    @classmethod
    def setUpClass(cls):
        super(InterpygateGSTTester, cls).setUpClass()
        target_op = SingleQubitTargetOp()
        param_ranges = [(0.9,1.1,3)]
        arg_ranges = [2*np.pi*(1+np.cos(np.linspace(np.pi,0, 7)))/2,
                  (0, np.pi, 3)]
        arg_indices = [0,1]
        gate_process = SingleQubitGate(num_params = 3,num_params_evaluated_as_group = 1)
        opfactory = interp.InterpolatedOpFactory.create_by_interpolating_physical_process(
                                target_op, gate_process, argument_ranges=arg_ranges,
                                parameter_ranges=param_ranges, argument_indices=arg_indices,
                                interpolator_and_args='linear')
        x_gate = opfactory.create_op([0,np.pi/4])
        y_gate = opfactory.create_op([np.pi/2,np.pi/4])
    
        cls.model = pygsti.models.ExplicitOpModel([0],'pp')
        cls.model['rho0'] = [ 1/np.sqrt(2), 0, 0, 1/np.sqrt(2) ] # density matrix [[1, 0], [0, 0]] in Pauli basis
        cls.model['Mdefault'] = pygsti.modelmembers.povms.UnconstrainedPOVM(
            {'0': [ 1/np.sqrt(2), 0, 0, 1/np.sqrt(2) ],   # projector onto [[1, 0], [0, 0]] in Pauli basis
             '1': [ 1/np.sqrt(2), 0, 0, -1/np.sqrt(2) ] }, evotype="default") # projector onto [[0, 0], [0, 1]] in Pauli basis
>       cls.model['Gxpi2',0] = x_gate

unit/extras/interpygate/test_construction.py:166: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../pygsti/models/explicitmodel.py:299: in __setitem__
    self.operations[label] = value
../pygsti/models/memberdict.py:302: in __setitem__
    self._check_evotype(value._evotype)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = OrderedMemberDict(), evotype = <pygsti.evotypes.evotype.Evotype object at 0x340a5ad10>

    def _check_evotype(self, evotype):
        if not self.flags['match_parent_evotype']: return  # no check
        if self.parent is None: return
        elif self.parent._evotype != evotype:
>           raise ValueError(("Cannot add an object with evolution type"
                              " '%s' to a model with one of '%s'") %
                             (evotype, self.parent._evotype))
E           ValueError: Cannot add an object with evolution type 'densitymx' to a model with one of 'densitymx_slow'

../pygsti/models/memberdict.py:190: ValueError
_______________________________________________________________ ERROR at setup of InterpygateGSTTester.test_gpindices _______________________________________________________________
[gw9] darwin -- Python 3.10.13 /Users/rjmurr/miniconda3/envs/pg310/bin/python

cls = <class 'test.unit.extras.interpygate.test_construction.InterpygateGSTTester'>

    @classmethod
    def setUpClass(cls):
        super(InterpygateGSTTester, cls).setUpClass()
        target_op = SingleQubitTargetOp()
        param_ranges = [(0.9,1.1,3)]
        arg_ranges = [2*np.pi*(1+np.cos(np.linspace(np.pi,0, 7)))/2,
                  (0, np.pi, 3)]
        arg_indices = [0,1]
        gate_process = SingleQubitGate(num_params = 3,num_params_evaluated_as_group = 1)
        opfactory = interp.InterpolatedOpFactory.create_by_interpolating_physical_process(
                                target_op, gate_process, argument_ranges=arg_ranges,
                                parameter_ranges=param_ranges, argument_indices=arg_indices,
                                interpolator_and_args='linear')
        x_gate = opfactory.create_op([0,np.pi/4])
        y_gate = opfactory.create_op([np.pi/2,np.pi/4])
    
        cls.model = pygsti.models.ExplicitOpModel([0],'pp')
        cls.model['rho0'] = [ 1/np.sqrt(2), 0, 0, 1/np.sqrt(2) ] # density matrix [[1, 0], [0, 0]] in Pauli basis
        cls.model['Mdefault'] = pygsti.modelmembers.povms.UnconstrainedPOVM(
            {'0': [ 1/np.sqrt(2), 0, 0, 1/np.sqrt(2) ],   # projector onto [[1, 0], [0, 0]] in Pauli basis
             '1': [ 1/np.sqrt(2), 0, 0, -1/np.sqrt(2) ] }, evotype="default") # projector onto [[0, 0], [0, 1]] in Pauli basis
>       cls.model['Gxpi2',0] = x_gate

unit/extras/interpygate/test_construction.py:166: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../pygsti/models/explicitmodel.py:299: in __setitem__
    self.operations[label] = value
../pygsti/models/memberdict.py:302: in __setitem__
    self._check_evotype(value._evotype)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = OrderedMemberDict(), evotype = <pygsti.evotypes.evotype.Evotype object at 0x340a5ad10>

    def _check_evotype(self, evotype):
        if not self.flags['match_parent_evotype']: return  # no check
        if self.parent is None: return
        elif self.parent._evotype != evotype:
>           raise ValueError(("Cannot add an object with evolution type"
                              " '%s' to a model with one of '%s'") %
                             (evotype, self.parent._evotype))
E           ValueError: Cannot add an object with evolution type 'densitymx' to a model with one of 'densitymx_slow'

../pygsti/models/memberdict.py:190: ValueError
===================================================================================== FAILURES ======================================================================================
_______________________________________________________________ LocalNoiseModelInstanceTester.test_marginalized_povm ________________________________________________________________
[gw2] darwin -- Python 3.10.13 /Users/rjmurr/miniconda3/envs/pg310/bin/python

self = <test.unit.objects.test_localnoisemodel.LocalNoiseModelInstanceTester testMethod=test_marginalized_povm>

    def test_marginalized_povm(self):
        mdl_local = create_crosstalk_free_model(self.pspec_4Q,
                                                ideal_gate_type='H+S', independent_gates=True,
                                                ensure_composed_gates=False)
    
        c = Circuit( [('Gx','qb0'),('Gx','qb1'),('Gx','qb2'),('Gx','qb3')], num_lines=4)
        prob = mdl_local.probabilities(c)
        self.assertEqual(len(prob), 16) # Full 4 qubit space
    
        c2 = Circuit( [('Gx','qb0'),('Gx','qb1')], num_lines=2)
>       prob2 = mdl_local.probabilities(c2)

unit/objects/test_localnoisemodel.py:124: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../pygsti/models/model.py:2055: in probabilities
    return self.sim.probs(circuit, outcomes, time)
../pygsti/forwardsims/forwardsim.py:198: in probs
    self.bulk_fill_probs(probs_array, copa_layout)
../pygsti/forwardsims/forwardsim.py:516: in bulk_fill_probs
    return self._bulk_fill_probs(array_to_fill, layout)
../pygsti/forwardsims/distforwardsim.py:99: in _bulk_fill_probs
    self._bulk_fill_probs_atom(array_to_fill[atom.element_slice], atom, atom_resource_alloc)
../pygsti/forwardsims/mapforwardsim.py:350: in _bulk_fill_probs_atom
    self.calclib.mapfill_probs_atom(self, array_to_fill, slice(0, array_to_fill.shape[0]),  # all indices
../pygsti/forwardsims/mapforwardsim_calc_generic.py:45: in mapfill_probs_atom
    povmreps = {plbl: fwdsim.model._circuit_layer_operator(plbl, 'povm')._rep for plbl in layout_atom.povm_labels}
../pygsti/forwardsims/mapforwardsim_calc_generic.py:45: in <dictcomp>
    povmreps = {plbl: fwdsim.model._circuit_layer_operator(plbl, 'povm')._rep for plbl in layout_atom.povm_labels}
../pygsti/models/model.py:2011: in _circuit_layer_operator
    return fns[typ](self, layerlbl, self._opcaches)
../pygsti/models/localnoisemodel.py:516: in povm_layer_operator
    povmName = _ot.effect_label_to_povm(layerlbl)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

povm_and_effect_lbl = Label(('Mdefault', 'qb0', 'qb1'))

    def effect_label_to_povm(povm_and_effect_lbl):
        """
        Extract the POVM label from a "simplified" effect label.
    
        Simplified effect labels are not themselves so simple.  They
        combine POVM and effect labels so that accessing any given effect
        vector is simpler.
    
        If `povm_and_effect_lbl` is `None` then `"NONE"` is returned.
    
        Parameters
        ----------
        povm_and_effect_lbl : Label
            Simplified effect vector.
    
        Returns
        -------
        str
        """
        # Helper fn: POVM_ELbl:sslbls -> POVM mapping
        if povm_and_effect_lbl is None:
            return "NONE"  # Dummy label for placeholding
        else:
            if isinstance(povm_and_effect_lbl, _Label):
>               last_underscore = povm_and_effect_lbl.name.rindex('_')
E               ValueError: substring not found

../pygsti/tools/optools.py:2489: ValueError
_________________________________________________________________________ CircuitMethodTester.test_simulate _________________________________________________________________________
[gw4] darwin -- Python 3.10.13 /Users/rjmurr/miniconda3/envs/pg310/bin/python

self = <test.unit.objects.test_circuit.CircuitMethodTester testMethod=test_simulate>

    def test_simulate(self):
        # TODO optimize
        # Create a pspec, to test the circuit simulator.
        n = 4
        qubit_labels = ['Q' + str(i) for i in range(n)]
        gate_names = ['Gh', 'Gp', 'Gxpi', 'Gpdag', 'Gcnot']  # 'Gi',
        ps = QubitProcessorSpec(n, gate_names=gate_names, qubit_labels=qubit_labels, geometry='line')
    
        # Tests the circuit simulator
        mdl = mc.create_crosstalk_free_model(ps)
        c = circuit.Circuit(layer_labels=[Label('Gh', 'Q0'), Label('Gcnot', ('Q0', 'Q1'))], line_labels=['Q0', 'Q1'])
>       out = c.simulate(mdl)

unit/objects/test_circuit.py:625: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../pygsti/tools/legacytools.py:57: in _inner
    return fn(*args, **kwargs)
../pygsti/circuits/circuit.py:4419: in simulate
    results = model.probabilities(self)
../pygsti/models/model.py:2055: in probabilities
    return self.sim.probs(circuit, outcomes, time)
../pygsti/forwardsims/forwardsim.py:198: in probs
    self.bulk_fill_probs(probs_array, copa_layout)
../pygsti/forwardsims/forwardsim.py:516: in bulk_fill_probs
    return self._bulk_fill_probs(array_to_fill, layout)
../pygsti/forwardsims/distforwardsim.py:99: in _bulk_fill_probs
    self._bulk_fill_probs_atom(array_to_fill[atom.element_slice], atom, atom_resource_alloc)
../pygsti/forwardsims/mapforwardsim.py:350: in _bulk_fill_probs_atom
    self.calclib.mapfill_probs_atom(self, array_to_fill, slice(0, array_to_fill.shape[0]),  # all indices
../pygsti/forwardsims/mapforwardsim_calc_generic.py:45: in mapfill_probs_atom
    povmreps = {plbl: fwdsim.model._circuit_layer_operator(plbl, 'povm')._rep for plbl in layout_atom.povm_labels}
../pygsti/forwardsims/mapforwardsim_calc_generic.py:45: in <dictcomp>
    povmreps = {plbl: fwdsim.model._circuit_layer_operator(plbl, 'povm')._rep for plbl in layout_atom.povm_labels}
../pygsti/models/model.py:2011: in _circuit_layer_operator
    return fns[typ](self, layerlbl, self._opcaches)
../pygsti/models/localnoisemodel.py:516: in povm_layer_operator
    povmName = _ot.effect_label_to_povm(layerlbl)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

povm_and_effect_lbl = Label(('Mdefault', 'Q0', 'Q1'))

    def effect_label_to_povm(povm_and_effect_lbl):
        """
        Extract the POVM label from a "simplified" effect label.
    
        Simplified effect labels are not themselves so simple.  They
        combine POVM and effect labels so that accessing any given effect
        vector is simpler.
    
        If `povm_and_effect_lbl` is `None` then `"NONE"` is returned.
    
        Parameters
        ----------
        povm_and_effect_lbl : Label
            Simplified effect vector.
    
        Returns
        -------
        str
        """
        # Helper fn: POVM_ELbl:sslbls -> POVM mapping
        if povm_and_effect_lbl is None:
            return "NONE"  # Dummy label for placeholding
        else:
            if isinstance(povm_and_effect_lbl, _Label):
>               last_underscore = povm_and_effect_lbl.name.rindex('_')
E               ValueError: substring not found

../pygsti/tools/optools.py:2489: ValueError
_________________________________________________________________ CircuitMethodTester.test_simulate_marginalization _________________________________________________________________
[gw4] darwin -- Python 3.10.13 /Users/rjmurr/miniconda3/envs/pg310/bin/python

self = <test.unit.objects.test_circuit.CircuitMethodTester testMethod=test_simulate_marginalization>

    def test_simulate_marginalization(self):
        pspec = QubitProcessorSpec(4, ['Gx', 'Gy'], geometry='line')
        mdl = mc.create_crosstalk_free_model(pspec)
    
        #Same circuit with different line labels
        c = circuit.Circuit("Gx:0Gy:0", line_labels=(0,1,2,3))
        cp = circuit.Circuit("Gx:0Gy:0", line_labels=(1,2,0,3))
        c01 = circuit.Circuit("Gx:0Gy:0", line_labels=(0,1))
        c10 = circuit.Circuit("Gx:0Gy:0", line_labels=(1,0))
        c0 = circuit.Circuit("Gx:0Gy:0", line_labels=(0,))
    
        #Make sure mdl.probabilities and circuit.simulate give us the correct answers
        pdict = mdl.probabilities(c)
        self.assertEqual(len(pdict), 16)  # all of 0000 -> 1111
        self.assertAlmostEqual(pdict['0000'], 0.5)
        self.assertAlmostEqual(pdict['1000'], 0.5)
    
>       pdict = mdl.probabilities(cp)

unit/objects/test_circuit.py:646: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../pygsti/models/model.py:2055: in probabilities
    return self.sim.probs(circuit, outcomes, time)
../pygsti/forwardsims/forwardsim.py:198: in probs
    self.bulk_fill_probs(probs_array, copa_layout)
../pygsti/forwardsims/forwardsim.py:516: in bulk_fill_probs
    return self._bulk_fill_probs(array_to_fill, layout)
../pygsti/forwardsims/distforwardsim.py:99: in _bulk_fill_probs
    self._bulk_fill_probs_atom(array_to_fill[atom.element_slice], atom, atom_resource_alloc)
../pygsti/forwardsims/mapforwardsim.py:350: in _bulk_fill_probs_atom
    self.calclib.mapfill_probs_atom(self, array_to_fill, slice(0, array_to_fill.shape[0]),  # all indices
../pygsti/forwardsims/mapforwardsim_calc_generic.py:45: in mapfill_probs_atom
    povmreps = {plbl: fwdsim.model._circuit_layer_operator(plbl, 'povm')._rep for plbl in layout_atom.povm_labels}
../pygsti/forwardsims/mapforwardsim_calc_generic.py:45: in <dictcomp>
    povmreps = {plbl: fwdsim.model._circuit_layer_operator(plbl, 'povm')._rep for plbl in layout_atom.povm_labels}
../pygsti/models/model.py:2011: in _circuit_layer_operator
    return fns[typ](self, layerlbl, self._opcaches)
../pygsti/models/localnoisemodel.py:516: in povm_layer_operator
    povmName = _ot.effect_label_to_povm(layerlbl)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

povm_and_effect_lbl = Label(('Mdefault', 1, 2, 0, 3))

    def effect_label_to_povm(povm_and_effect_lbl):
        """
        Extract the POVM label from a "simplified" effect label.
    
        Simplified effect labels are not themselves so simple.  They
        combine POVM and effect labels so that accessing any given effect
        vector is simpler.
    
        If `povm_and_effect_lbl` is `None` then `"NONE"` is returned.
    
        Parameters
        ----------
        povm_and_effect_lbl : Label
            Simplified effect vector.
    
        Returns
        -------
        str
        """
        # Helper fn: POVM_ELbl:sslbls -> POVM mapping
        if povm_and_effect_lbl is None:
            return "NONE"  # Dummy label for placeholding
        else:
            if isinstance(povm_and_effect_lbl, _Label):
>               last_underscore = povm_and_effect_lbl.name.rindex('_')
E               ValueError: substring not found

../pygsti/tools/optools.py:2489: ValueError
_____________________________________________________________ TestPeriodicMirrorCircuitsDesign.test_design_construction _____________________________________________________________
[gw9] darwin -- Python 3.10.13 /Users/rjmurr/miniconda3/envs/pg310/bin/python

self = <test.unit.protocols.test_vb.TestPeriodicMirrorCircuitsDesign testMethod=test_design_construction>

    def test_design_construction(self):
    
        n = 4
        qs = ['Q'+str(i) for i in range(n)]
        ring = [('Q'+str(i),'Q'+str(i+1)) for i in range(n-1)]
    
        gateset1 = ['Gcphase'] + ['Gc'+str(i) for i in range(24)]
        pspec1 = QPS(n, gateset1, availability={'Gcphase':ring}, qubit_labels=qs)
        tmodel1 = pygsti.models.create_crosstalk_free_model(pspec1)
    
        depths = [0, 2, 8]
        q_set = ('Q0', 'Q1', 'Q2')
    
        clifford_compilations = {'absolute': CCR.create_standard(pspec1, 'absolute', ('paulis', '1Qcliffords'), verbosity=0)}
    
        design1 = _vb.PeriodicMirrorCircuitDesign (pspec1, depths, 3, qubit_labels=q_set,
                                        clifford_compilations=clifford_compilations, sampler='edgegrab', samplerargs=(0.25,))
    
>       [[self.assertAlmostEqual(c.simulate(tmodel1)[bs],1.) for c, bs in zip(cl, bsl)] for cl, bsl in zip(design1.circuit_lists, design1.idealout_lists)]

unit/protocols/test_vb.py:28: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
unit/protocols/test_vb.py:28: in <listcomp>
    [[self.assertAlmostEqual(c.simulate(tmodel1)[bs],1.) for c, bs in zip(cl, bsl)] for cl, bsl in zip(design1.circuit_lists, design1.idealout_lists)]
unit/protocols/test_vb.py:28: in <listcomp>
    [[self.assertAlmostEqual(c.simulate(tmodel1)[bs],1.) for c, bs in zip(cl, bsl)] for cl, bsl in zip(design1.circuit_lists, design1.idealout_lists)]
../pygsti/tools/legacytools.py:57: in _inner
    return fn(*args, **kwargs)
../pygsti/circuits/circuit.py:4419: in simulate
    results = model.probabilities(self)
../pygsti/models/model.py:2055: in probabilities
    return self.sim.probs(circuit, outcomes, time)
../pygsti/forwardsims/forwardsim.py:198: in probs
    self.bulk_fill_probs(probs_array, copa_layout)
../pygsti/forwardsims/forwardsim.py:516: in bulk_fill_probs
    return self._bulk_fill_probs(array_to_fill, layout)
../pygsti/forwardsims/distforwardsim.py:99: in _bulk_fill_probs
    self._bulk_fill_probs_atom(array_to_fill[atom.element_slice], atom, atom_resource_alloc)
../pygsti/forwardsims/mapforwardsim.py:350: in _bulk_fill_probs_atom
    self.calclib.mapfill_probs_atom(self, array_to_fill, slice(0, array_to_fill.shape[0]),  # all indices
../pygsti/forwardsims/mapforwardsim_calc_generic.py:45: in mapfill_probs_atom
    povmreps = {plbl: fwdsim.model._circuit_layer_operator(plbl, 'povm')._rep for plbl in layout_atom.povm_labels}
../pygsti/forwardsims/mapforwardsim_calc_generic.py:45: in <dictcomp>
    povmreps = {plbl: fwdsim.model._circuit_layer_operator(plbl, 'povm')._rep for plbl in layout_atom.povm_labels}
../pygsti/models/model.py:2011: in _circuit_layer_operator
    return fns[typ](self, layerlbl, self._opcaches)
../pygsti/models/localnoisemodel.py:516: in povm_layer_operator
    povmName = _ot.effect_label_to_povm(layerlbl)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

povm_and_effect_lbl = Label(('Mdefault', 'Q0', 'Q1', 'Q2'))

    def effect_label_to_povm(povm_and_effect_lbl):
        """
        Extract the POVM label from a "simplified" effect label.
    
        Simplified effect labels are not themselves so simple.  They
        combine POVM and effect labels so that accessing any given effect
        vector is simpler.
    
        If `povm_and_effect_lbl` is `None` then `"NONE"` is returned.
    
        Parameters
        ----------
        povm_and_effect_lbl : Label
            Simplified effect vector.
    
        Returns
        -------
        str
        """
        # Helper fn: POVM_ELbl:sslbls -> POVM mapping
        if povm_and_effect_lbl is None:
            return "NONE"  # Dummy label for placeholding
        else:
            if isinstance(povm_and_effect_lbl, _Label):
>               last_underscore = povm_and_effect_lbl.name.rindex('_')
E               ValueError: substring not found

../pygsti/tools/optools.py:2489: ValueError

[sserita]

OK I can confirm the second error message is a densitymx_slow evotype bug. The bug has to do with the case where a circuit is implicitly using a MarginalizedPOVM, i.e. it is using measuring a subset of the qubits defined in the model. This is a rarely used edge case, so not surprised it slipped under the radar until now.

Basically the difference is between the Cython directly computing effect reps:

pyGSTi/pygsti/forwardsims/mapforwardsim_calc_densitymx.pyx

Line 327 in 916877c

ereps = [fwdsim.model._circuit_layer_operator(elbl, 'povm')._rep for elbl in layout_atom.full_effect_labels] # cache these in future

vs the Python computing for the general POVM effect first.

pyGSTi/pygsti/forwardsims/mapforwardsim_calc_generic.py

Lines 45 to 50 in 916877c

	povmreps = {plbl: fwdsim.model._circuit_layer_operator(plbl, 'povm')._rep for plbl in layout_atom.povm_labels}
	if any([(povmrep is None) for povmrep in povmreps.values()]):
	effectreps = {i: fwdsim.model._circuit_layer_operator(Elbl, 'povm')._rep
	for i, Elbl in enumerate(layout_atom.full_effect_labels)} # cache these in future
	else:
	effectreps = None # not needed, as we use povm reps directly

I'll use test_simulate from test/unit/objects/test_circuit.py to illustrate the issue.

pyGSTi/test/unit/objects/test_circuit.py

Lines 614 to 627 in 916877c

	def test_simulate(self):
	# TODO optimize
	# Create a pspec, to test the circuit simulator.
	n = 4
	qubit_labels = ['Q' + str(i) for i in range(n)]
	gate_names = ['Gh', 'Gp', 'Gxpi', 'Gpdag', 'Gcnot'] # 'Gi',
	ps = QubitProcessorSpec(n, gate_names=gate_names, qubit_labels=qubit_labels, geometry='line')
	# Tests the circuit simulator
	mdl = mc.create_crosstalk_free_model(ps)
	c = circuit.Circuit(layer_labels=[Label('Gh', 'Q0'), Label('Gcnot', ('Q0', 'Q1'))], line_labels=['Q0', 'Q1'])
	out = c.simulate(mdl)
	self.assertLess(abs(out['00'] - 0.5), 10**-10)
	self.assertLess(abs(out['11'] - 0.5), 10**-10)

This test uses a circuit defined on two qubits, but the model is defined on 4 qubits, so this is an implicit marginalized POVM.
The test constructs a LocalNoiseModel, so this is the relevant POVM lookup function:

pyGSTi/pygsti/models/localnoisemodel.py

Lines 495 to 530 in 916877c

	def povm_layer_operator(self, model, layerlbl, caches):
	"""
	Create the operator corresponding to `layerlbl`.
	Parameters
	----------
	layerlbl : Label
	A circuit layer label.
	Returns
	-------
	POVM or POVMEffect
	"""
	# caches['povm-layers'] *are* just complete layers
	if layerlbl in caches['povm-layers']: return caches['povm-layers'][layerlbl]
	if layerlbl in model.povm_blks['layers']:
	return model.povm_blks['layers'][layerlbl]
	else:
	# See if this effect label could correspond to a *marginalized* POVM, and
	# if so, create the marginalized POVM and add its effects to model.effect_blks['layers']
	#assert(isinstance(layerlbl, _Lbl)) # Sanity check
	povmName = _ot.effect_label_to_povm(layerlbl)
	if povmName in model.povm_blks['layers']:
	# implicit creation of marginalized POVMs whereby an existing POVM name is used with sslbls that
	# are not present in the stored POVM's label.
	mpovm = _povm.MarginalizedPOVM(model.povm_blks['layers'][povmName],
	model.state_space, layerlbl.sslbls) # cache in FUTURE
	mpovm_lbl = _Lbl(povmName, layerlbl.sslbls)
	simplified_effects = mpovm.simplify_effects(mpovm_lbl)
	assert(layerlbl in simplified_effects), "Failed to create marginalized effect!"
	if self.use_op_caching:
	caches['povm-layers'].update(simplified_effects)
	return simplified_effects[layerlbl]
	else:
	#raise KeyError(f"Could not build povm/effect for {layerlbl}!")
	raise KeyError("Could not build povm/effect for %s!" % str(layerlbl))

Both will fail the layerlbl in povm_blks check, but the else assumes that layerlbl is an effect label (e.g. "Mdefault_00"). This is true in the Cython (evotype "densitymx") case, but not in the Python (evotype "densitymx_slow") case. In fact, we are failing at the povmreps construction line just before the effect labels would be passed in.

Two possible solutions:

1. Wrap the povmreps construction in a try/except that then moves onto the effect labels which should work (since this is the codepath "densitymx" is using).
2. Fix the LocalNoiseModel layer rules such that the POVM label is handled properly.

Solution 1 has the advantage that it is model/layer rule independent, but I kind of like Solution 2 more. I personally prefer not having try/excepts unless there really is no reasonable alternative. We will just have to make sure that every layer rule handles both the POVM and effect label cases. Thoughts?