Support Asynchronous Execution with Compatibility Layer

.gitlab-ci.yml

@@ -75,6 +75,7 @@ Test Unit (3.7):
 
 Test e2e QPU (3.7):
   stage: test
+  allow_failure: true
   image: python:3.7
   coverage: '/TOTAL.*?(\d+)\%/'
   script:
@@ -101,6 +102,7 @@ Test Unit (3.8):
 
 Test e2e QPU (3.8):
   stage: test
+  allow_failure: true
   image: python:3.8
   coverage: '/TOTAL.*?(\d+)\%/'
   script:
@@ -127,6 +129,7 @@ Test Unit (3.9):
 
 Test e2e QPU (3.9):
   stage: test
+  allow_failure: true
   image: python:3.9
   coverage: '/TOTAL.*?(\d+)\%/'
   script:

CHANGELOG.md

@@ -6,8 +6,15 @@ Changelog
 
 ### Announcements
 
-- Python 3.6 is no longer supported. Python 3.7, 3.8, and 3.9 are supported.
+- pyQuil now directly supports the QCS API v1.0, offering you better performance and more
+  granular data about QCS quantum processors.
 
+- Python 3.6 is no longer supported. Python 3.7, 3.8, and 3.9 are supported.
+
+- `pyquil.compatibility.v2` provides a number of classes which support the pyQuil v2 API, such as
+  `QuantumComputer` and `get_qc`; `pyquil.compatibility.v2.api` offers `QPU` and `QVM`. These may be
+  used to incrementally migrate from v2 to v3, but should not be relied on indefinitely, as the
+  underlying mechanics of these two versions continue to diverge in the future.
 ### Improvements and Changes
 
 - Added support and documentation for concurrent compilation and execution (see "Advanced Usage" in docs)
@@ -24,8 +31,7 @@ Changelog
 - `ForestConnection` and `ForestSession` have been removed. Connection information is now managed via `api.QCSClientConfiguration`
   and `api.EngagementManager`.
 
-- `QVMCompiler` now produces a `Program` instead of a `PyQuilExecutableResponse`. As a result, `QVM.load()` only
-  accepts a `Program`, and `QVM.requires_executable` has been removed.
+- `QVMCompiler` now produces a `Program` instead of a `PyQuilExecutableResponse`.
 
 - `QPU.get_version_info()` has been removed.
 
@@ -84,7 +90,7 @@ Changelog
 
 - `get_qc()` no longer accepts `"9q-generic"` (deprecated). Use `"9q-square"` instead.
 
-- Removed `QAM.read_from_memory_region()` (deprecated). Use `QAM.read_memory()` instead.
+- Removed `QAM.read_from_memory_region()` (deprecated). Use `QAMExecutionResult.readout_data.get(region_name)` instead.
 
 - Removed `local_qvm()` (deprecated). Use `local_forest_runtime()` instead.
 
@@ -119,6 +125,37 @@ Changelog
   are passed to the QVM by `WavefunctionSimulator.run_and_measure()` and `WavefunctionSimulator.wavefunction()`.
 
 - `noise.estimate_assignment_probs()` now accepts a `QuantumComputer` instead of `QVMConnection`.
+
+- `QAM` and its subclasses (such as `QPU` and `QVM`) do not store any information specific to the state
+  of execution requests, and thus are safe to be used concurrently by different requests. `QAM.run`
+  is now composed of two intermediate calls:
+  -  `QAM.execute` starts execution of the provided executable, returning an opaque handle.
+  -  `QAM.get_result` uses the opaque handle returned by `execute` to retrieve the result values.
+
+  These new calls can be used to enqueue multiple programs for execution prior to retrieving
+  results for any of them. Note that this new pattern means that `QAM.load`, `QAM.reset`, and
+  `QAM.wait` no longer exist.
+
+- `QAM.run` no longer accepts a `memory_map` argument. Memory values must be written onto
+  executable directly with `Program.write_memory()` and `EncryptedProgram.write_memory()` instead. 
+
+- `QuantumComputer`, `QAM`, `QPU`, and `QVM` are now safe to share across threads and processes,
+  as they no longer store request-related state.
+
+- `PyQVM.execute` has been renamed to `PyQVM.execute_once` to execute a single program from start
+  to finish within the context of the existing `PyQVM` state. `PyQVM` is the only stateful `QAM`.
+  `PyQVM.execute` now implements `QAM.execute` and resets the `PyQVM` state prior to program execution.
+
+- `QuantumComputer.experiment` has been renamed to `QuantumComputer.run_experiment`.
+
+- Results returned from execution are now referred to as `readout_data` rather than `memory`, reflecting the reality
+  that the memory of the QAM is not currently exposed to the user. The exception to this rule is the stateful `PyQVM`,
+  whose state is maintained within the pyQuil process and whose memory _may truly be inspected._ For that,
+  `PyQVM.read_memory` remains available.
+
+- `QuantumComputer.run` now returns a `QAMExecutionResult` rather than the readout data from the `ro` readout
+  source. To access those same readout results, use `qc.run().readout_data.get('ro')`. This allows access to other
+  execution-related information and other readout sources.
 
 - Simultaneous, rather than independent, random benchmark scores are passed to quilc as the gate fidelity for RX and RZ operations.
 

README.md

@@ -79,8 +79,8 @@ ro = p.declare('ro', 'BIT', 2)
 p += MEASURE(0, ro[0])
 p += MEASURE(1, ro[1])
 p.wrap_in_numshots_loop(10)
- 
-qvm.run(p).tolist()
+
+qvm.run(p).readout_data['ro'].tolist()
 ```
 
 The output of the above program should look something like the following,

docs/source/advanced_usage.rst

@@ -35,21 +35,21 @@ locally):
 Concurrency
 ~~~~~~~~~~~
 
-Using pyQuil for concurrent programming is as simple as calling :py:func:`~pyquil.get_qc()` from within a given thread
-or process, then using the returned :py:class:`~pyquil.api.QuantumComputer` as usual. While
-:py:class:`~pyquil.api.QuantumComputer` objects as a whole are not safe to share between threads or processes (due to
-state related to currently-running compilation or execution requests), some information they use is. Information related
-to client configuration (:py:class:`~pyquil.api.QCSClientConfiguration`) and QPU auth
-(:py:class:`~pyquil.api.EngagementManager`) can be safely extracted and shared among
-:py:class:`~pyquil.api.QuantumComputer` instances, as shown below, to save your code from redundant disk reads and
-auth-related HTTP requests.
+:py:class:`~pyquil.api.QuantumComputer` objects are safe to share between threads or processes,
+enabling you to execute and retrieve results for multiple programs or parameter values at once.
+Note that :py:class`~pyquil.Program` and :py:class`~pyquil.api.EncryptedProgram` are **not**
+thread-safe, and should be copied (with ``copy()``) before use in a concurrent context.
 
 .. note::
     The QVM processes incoming requests in parallel, while a QPU may process them sequentially or in parallel
     (depending on the qubits used). If you encounter timeouts while trying to run large numbers of programs against a
     QPU, try increasing the ``execution_timeout`` parameter on calls  to :py:func:`~pyquil.get_qc()` (specified in
     seconds).
 
+.. note::
+    We suggest running jobs with a minimum of 2x parallelism, so that the QVM or QPU
+    is fully occupied while your program runs and no time is wasted in between jobs.
+
 Using Multithreading
 --------------------
 
@@ -58,16 +58,14 @@ Using Multithreading
     from multiprocessing.pool import ThreadPool
 
     from pyquil import get_qc, Program
-    from pyquil.api import EngagementManager, QCSClientConfiguration
-
+    from pyquil.api import QCSClientConfiguration
 
     configuration = QCSClientConfiguration.load()
-    engagement_manager = EngagementManager(client_configuration=configuration)
+    qc = get_qc("Aspen-8", client_configuration=configuration)
 
 
     def run(program: Program):
-        qc = get_qc("Aspen-8", client_configuration=configuration, engagement_manager=engagement_manager)
-        return qc.run(qc.compile(program))
+        return qc.run(qc.compile(program)).readout_data.get("ro")
 
 
     programs = [Program("DECLARE ro BIT", "RX(pi) 0", "MEASURE 0 ro").wrap_in_numshots_loop(10)] * 20
@@ -86,16 +84,15 @@ Using Multiprocessing
     from multiprocessing.pool import Pool
 
     from pyquil import get_qc, Program
-    from pyquil.api import EngagementManager, QCSClientConfiguration
+    from pyquil.api import QCSClientConfiguration
 
 
     configuration = QCSClientConfiguration.load()
-    engagement_manager = EngagementManager(client_configuration=configuration)
+    qc = get_qc("Aspen-8", client_configuration=configuration)
 
 
     def run(program: Program):
-        qc = get_qc("Aspen-8", client_configuration=configuration, engagement_manager=engagement_manager)
-        return qc.run(qc.compile(program))
+        return qc.run(qc.compile(program)).readout_data.get("ro")
 
 
     programs = [Program("DECLARE ro BIT", "RX(pi) 0", "MEASURE 0 ro").wrap_in_numshots_loop(10)] * 20

docs/source/apidocs/quantum_computer.rst

@@ -17,7 +17,6 @@ Quantum Computer
 
         ~QuantumComputer.run
         ~QuantumComputer.calibrate
-        ~QuantumComputer.experiment
         ~QuantumComputer.run_symmetrized_readout
         ~QuantumComputer.qubits
         ~QuantumComputer.qubit_topology

docs/source/basics.rst

@@ -80,7 +80,8 @@ program on the Quantum Virtual Machine (QVM). We just have to add a few lines to
     qc = get_qc('1q-qvm')  # You can make any 'nq-qvm' this way for any reasonable 'n'
     executable = qc.compile(p)
     result = qc.run(executable)
-    print(result)
+    bitstrings = result.readout_data.get('ro')
+    print(bitstrings)
 
 Congratulations! You just ran your program on the QVM. The returned value should be:
 
@@ -299,8 +300,12 @@ filled in for, say, 200 values between :math:`0` and :math:`2\pi`. We demonstrat
     parametric_measurements = []
 
     for theta in np.linspace(0, 2 * np.pi, 200):
+        # Set the desired parameter value in executable memory
+        executable.write_memory(region_name='theta', value=theta)
+
         # Get the results of the run with the value we want to execute with
-        bitstrings = qc.run(executable, {'theta': [theta]})
+        bitstrings = qc.run(executable).readout_data.get("ro")
+
         # Store our results
         parametric_measurements.append(bitstrings)
 

docs/source/intro.rst

@@ -597,7 +597,7 @@ measurements. This functionality is emulated by :py:func:`QuantumComputer.run`.
 
 
     qc = get_qc('9q-square-qvm')
-    print (qc.run(qc.compile(p)))
+    print (qc.run(qc.compile(p)).readout_data.get("ro"))
 
 
 .. parsed-literal::
@@ -615,7 +615,7 @@ qubit before measurement then we obtain:
     p += Program(X(0))   # Flip the qubit
     p.measure(0, classical_register[0])   # Measure the qubit
 
-    print (qc.run(qc.compile(p)))
+    print (qc.run(qc.compile(p)).readout_data.get("ro"))
 
 
 .. parsed-literal::
@@ -636,7 +636,7 @@ times then we always get the same outcome:
     trials = 10
     p.wrap_in_numshots_loop(shots=trials)
 
-    print (qc.run(qc.compile(p)))
+    print (qc.run(qc.compile(p)).readout_data.get("ro"))
 
 
 .. parsed-literal::
@@ -705,7 +705,7 @@ extra power over regular bits.
     p.wrap_in_numshots_loop(shots=10)
 
     # We see probabilistic results of about half 1's and half 0's
-    print (qc.run(qc.compile(p)))
+    print (qc.run(qc.compile(p)).readout_data.get("ro"))
 
 
 .. parsed-literal::
@@ -799,7 +799,7 @@ quantum operations to run.
     p.measure(7, ro[7])
 
     # Run and check register [7]
-    print (qc.run(qc.compile(p)))
+    print (qc.run(qc.compile(p)).readout_data.get("ro"))
 
 
 .. parsed-literal::
@@ -824,7 +824,7 @@ increasing chance of halting, but that may run forever!
     p.inst(X(0)).while_do(ro[0], inside_loop)
 
     qc = get_qc('9q-square-qvm')
-    print (qc.run(qc.compile(p)))
+    print (qc.run(qc.compile(p)).readout_data.get("ro"))
 
 
 .. parsed-literal::

docs/source/migration.rst

@@ -9,6 +9,89 @@ to the code affected. Most users should only need to make minimal changes.
 If you've supplied ``PyquilConfig`` objects to functions (or used the ``QVM_URL`` and ``QUILC_URL`` environment variables)
 to override configuration, see :ref:`pyquil_configuration`.
 
-Lastly, pyQuil v3 relies on an updated authentication model. To get going smoothly, you should install the new `QCS CLI
-<https://docs.rigetti.com/en/command-line-interface/command-line-interface>`_ and log in with it before using pyQuil v3
-against real QPUs.
+Authentication
+--------------
+
+pyQuil v3 relies on an updated authentication model. To get started, install the new `QCS CLI
+<https://docs.rigetti.com/>`_  and
+log in with it before using pyQuil v3 with QCS and live QPUs.
+
+
+Parameters & Memory
+-------------------
+
+In order to give the user more control over and visibility into program execution, especially in
+parallel, objects such as ``QuantumComputer``, ``QAM``, ``QPU``, and ``QVM`` are no longer stateful
+with respect to individual programs. These objects are now safe to
+share among different threads, so you can execute and retrieve results in parallel for even better
+performance. (See :doc:`advanced_usage` for more information).
+
+However, this required three small but important changes:
+
+1. ``write_memory`` is no longer a method on ``QAM`` but rather on ``Program`` and ``EncryptedProgram``.
+2. ``qc.run()`` no longer accepts a ``memory_map`` argument. All memory values must be set directly on the ``Program`` or ``EncryptedProgram`` using ``write_memory``.
+3. ``QAM.load()``, ``QAM.wait()``, and ``QAM.reset()`` no longer exist, because the ``QAM`` no longer "stores" program state.
+
+This means that you should now execute your programs using one of these options:
+
+.. code:: python
+
+   qc = get_qc("Aspen-X")
+   program = Program()
+   theta = program.declare('theta', 'REAL')
+   program += RZ(theta, 0)
+   exe = qc.compile(program)
+
+   # Previously, we would have called ``qc.qam.write_memory`` instead
+   exe.write_memory(region_name='theta', value=np.pi)
+
+   # Option 1
+   result = qc.run(exe)
+
+   # Option 2
+   result = qc.qam.run(exe)
+
+   # Option 3
+   job = qc.qam.execute(exe)
+   result = qc.qam.get_result(job)
+
+   # Run our program 10 times, enqueuing all the programs before retrieving results for any of them
+   jobs = [qc.qam.execute(exe) for _ in range(10)]
+   results = [qc.qam.get_result(job) for job in jobs]
+
+Additionally, ``QuantumComputer.run()`` and ``QAM.run()`` now both return ``QAMExecutionResult``, rather
+than the bitstrings which were read out following execution. So, to retrieve the bitstrings from region ``ro``
+like you would have in pyQuil v2, query the readout data:
+
+.. code:: python
+
+   result = qc.run(exe)
+   bitstrings = result.readout_data.get('ro')
+
+Note that there is no "memory" in the result shape -- yet. Readout data is inherently collected and returned
+differently than memory values are, and that nuance is no longer masked in pyQuil v3. All of the results
+that, in pyQuil v2, you would have expected to retrieve using ``QAM.read_memory()`` are now present in
+``QAMExecutionResult.readout_data``.
+
+The notable exception to this is ``PyQVM``, which remains stateful, and whose memory you can still inspect using
+``PyQVM.read_memory()``.
+
+
+Compatibility Utilities
+-----------------------
+
+We understand that the changes above regarding parameters might cause difficulty in migration,
+especially for large projects and lengthy scripts. So, to ease the migration path, we've added
+the following utility classes which allow you to upgrade your pyQuil projects without having to
+change any code.
+
+.. code:: python
+
+   from pyquil.compatibility.v2 import get_qc, QuantumComputer
+   from pyquil.compatibility.v2.api import QAM, QVM, QPU
+
+You can use these imported objects similarly how you use their counterparts in pyQuil v2.
+Once you've verified that your scripts still work with v3, we recommend that you gradually convert
+them to use the new versions of each object. This compatibility layer won't see any new
+development, and without fully upgrading you'd miss out on all the new features to come in the
+future.