Support direct Target generation and lazy Pulse Qobj conversion

[nkanazawa1989]

Summary

This PR has dependency on Qiskit/qiskit#8885

This PR allows V2 backend to bypass generation of PulseDefaults and BackendProperties to generate target, in addition to support of lazy Pulse Qobj conversion mechanism for speedup. Roughly x100 speed up in loading the target of ibm_washington.

Details and comments

1. Benchmarking this branch with Qiskit/qiskit#8885

To avoid taking network latency into account, target generation speed is measured by preloaded defaults and property dictionary.

from qiskit_ibm_provider import IBMProvider
from cProfile import Profile

from qiskit_ibm_provider.utils.json_decoder import target_from_server_data

def get_target(configuration, defaults, properties):

    target = target_from_server_data(
        configuration=configuration,
        properties=properties,
        pulse_defaults=defaults,
    )


def main():
    provider = IBMProvider()
    backend = provider.get_backend("ibm_washington")
    client = backend._api_client
    defaults = client.backend_pulse_defaults("ibm_washington")
    properties = client.backend_properties("ibm_washington")
    
    p = Profile()
    p.runcall(get_target, backend._configuration, defaults, properties)
    p.dump_stats("load_washington_lazy.profile")

2. Benchmarking main branch

In the same way target "conversion" speed is measured.

from qiskit_ibm_provider import IBMProvider
from cProfile import Profile

from qiskit_ibm_provider.utils.json_decoder import properties_from_server_data
from qiskit_ibm_provider.utils.json_decoder import defaults_from_server_data
from qiskit_ibm_provider.utils.backend_converter import convert_to_target


def get_target(configuration, defaults, properties):

    target = convert_to_target(
        configuration=configuration,
        properties=properties_from_server_data(properties),
        defaults=defaults_from_server_data(defaults),
    )


def main():
    provider = IBMProvider()
    backend = provider.get_backend("ibm_washington")
    client = backend._api_client
    defaults = client.backend_pulse_defaults("ibm_washington")
    properties = client.backend_properties("ibm_washington")
    
    p = Profile()
    p.runcall(get_target, backend._configuration, defaults, properties)
    p.dump_stats("load_washington_main.profile")

[coveralls]

Pull Request Test Coverage Report for Build 4026788784

• 17 of 111 (15.32%) changed or added relevant lines in 2 files are covered.
• 1 unchanged line in 1 file lost coverage.
• Overall coverage increased (+0.09%) to 50.211%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
qiskit_ibm_provider/ibm_backend.py	1	6	16.67%
qiskit_ibm_provider/utils/json_decoder.py	16	105	15.24%

Files with Coverage Reduction	New Missed Lines	%
qiskit_ibm_provider/ibm_backend.py	1	61.97%

Totals
Change from base Build 4019692203:	0.09%
Covered Lines:	2862
Relevant Lines:	5700

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 4726014865

• 20 of 128 (15.63%) changed or added relevant lines in 2 files are covered.
• 1 unchanged line in 1 file lost coverage.
• Overall coverage increased (+0.05%) to 50.38%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
qiskit_ibm_provider/ibm_backend.py	2	17	11.76%
qiskit_ibm_provider/utils/json_decoder.py	18	111	16.22%

Files with Coverage Reduction	New Missed Lines	%
qiskit_ibm_provider/ibm_backend.py	1	65.73%

Totals
Change from base Build 4725435778:	0.05%
Covered Lines:	2982
Relevant Lines:	5919

---

💛 - Coveralls

[kt474]

Mostly looks good to me, looks like lots of integration tests are failing with the following errors:

test_ibm_integration, test_ibm_qasm_simulator

File "/Users/kt/opt/anaconda3/envs/dev/lib/python3.9/site-packages/qiskit/circuit/gate.py", line 237, in validate_parameter
    raise CircuitError(f"Invalid param type {type(parameter)} for gate {self.name}.")
qiskit.circuit.exceptions.CircuitError: "Invalid param type <class 'str'> for gate kraus."

test_ibm_job, test_ibm_job_attributes

 File "/Users/kt/opt/anaconda3/envs/dev/lib/python3.9/site-packages/qiskit/circuit/gate.py", line 237, in validate_parameter
    raise CircuitError(f"Invalid param type {type(parameter)} for gate {self.name}.")
qiskit.circuit.exceptions.CircuitError: "Invalid param type <class 'str'> for gate mcu3."

I'm using terra 0.23.1

[mtreinish]

Overall this LGTM, I like the new structure for the target generation, it also makes a lot more sense in the json utils because that's what it's doing (the old file was called backend_converter.py because that was the filename of my local test script for going v1->v2 that it was based on). I just left two small inline comments about some edge case handling in the target generation.

The other thing is I think this needs a rebase, it conflicts with some recent changes made to the backends to do more lazy loading in general.

[mtreinish]

Shouldn't we look this up in qiskit_inst_mapping first instead of assigning it an opaque gate?

[nkanazawa1989]

I decided to ignore this in a67315e because this must be an edge case of misconfiguration, i.e. gate property is provided for gates which is not included in basis gates.

[nkanazawa1989]

Thanks Matthew for reviewing. I added logger to the decoder module and add more description for missing entries (I hope this will help deployment work). The criteria of error is bit unclear, as you suggested to raise in this comment. This case could be just a info log since missing calibration (especially for default gates) is not significant matter. On the other hand, missing Qiskit gate representation for given name is much more problematic and we could raise an error.

I completed integration tests locally and encountered following errors, but likely these are not relevant to this PR. I didn't complete make e2e-test because of long queueing through open provider.

======================================================================
ERROR: test_backend_options (test.integration.test_backend.TestIBMBackend)
Test backend options.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/knzwnao/Develop/qiskit/qiskit-ibm-provider/test/integration/test_backend.py", line 96, in test_backend_options
    job = backend.run(get_pulse_schedule(backend), meas_level=1, foo="foo")
  File "/Users/knzwnao/Develop/qiskit/qiskit-ibm-provider/qiskit_ibm_provider/ibm_backend.py", line 486, in run
    return self._runtime_run(
  File "/Users/knzwnao/Develop/qiskit/qiskit-ibm-provider/qiskit_ibm_provider/ibm_backend.py", line 503, in _runtime_run
    response = self.provider._runtime_client.program_run(
  File "/Users/knzwnao/Develop/qiskit/qiskit-ibm-provider/qiskit_ibm_provider/api/clients/runtime.py", line 137, in program_run
    return self._api.program_run(
  File "/Users/knzwnao/Develop/qiskit/qiskit-ibm-provider/qiskit_ibm_provider/api/rest/runtime.py", line 116, in program_run
    data = json.dumps(payload, cls=RuntimeEncoder)
  File "/usr/local/Cellar/python@3.10/3.10.10_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/local/Cellar/python@3.10/3.10.10_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/local/Cellar/python@3.10/3.10.10_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/Users/knzwnao/Develop/qiskit/qiskit-ibm-provider/qiskit_ibm_provider/utils/json.py", line 257, in default
    return super().default(obj)
  File "/usr/local/Cellar/python@3.10/3.10.10_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type Schedule is not JSON serializable

======================================================================
ERROR: test_provider_backend (test.integration.test_ibm_provider.TestIBMProviderServices)
Test provider backend has correct attributes.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/knzwnao/Develop/qiskit/qiskit-ibm-provider/test/integration/test_ibm_provider.py", line 308, in test_provider_backend
    backends = {
  File "/Users/knzwnao/Develop/qiskit/qiskit-ibm-provider/test/integration/test_ibm_provider.py", line 309, in <setcomp>
    back.name.lower()
AttributeError: 'NoneType' object has no attribute 'name'

======================================================================
FAIL: test_transpile_converts_id (test.integration.test_backend.TestIBMBackend)
Test that when targeting an IBM backend id is translated to delay.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/knzwnao/Develop/qiskit/qiskit-ibm-provider/test/integration/test_backend.py", line 169, in test_transpile_converts_id
    self.assertIn("delay", op_counts)
AssertionError: 'delay' not found in OrderedDict()
----------------------------------------------------------------------

[mtreinish]

My thinking on that comment at the time was if the defaults payload contained a gate not mentioned in the configuration or properties it was malformed and that should be an error. But, the two things I hadn't thought of at the time but on reflection are important to discuss now are:

• Could this error be triggered as part of a custom calibration job/environment ?
• An error would not be user actionable and not something an end user can control

So given that I think assuming this isn't an edge case that could come up during calibrations I think you're correct we should just log this and not raise an exception. Since you're correct a user would have no control over the payload being malformed as it would require changes in the backend to fix it. So I think your suggestion to just emit a log message (maybe at warning level instead of info so it's seen by default) so users can report it's happening and just ignore the out of spec gate.

[nkanazawa1989]

Thanks Matthew. That's correct. There is nothing actionable for end-users. I chose the info level since IBM backend still reports U gate calibrations and it generates very long lines of warning message which is quite annoying. I turned the error into log message and set warning level only to missing Qiskit gate 939ca42. I think now this PR is ready for the final review/approval.

[mtreinish]

This LGTM, thanks for making this change