Skip to content

Commit 61d1337

Browse files
mccoypiscai-msft
mccoyp
authored and
iscai-msft
committed
[Test proxy] Add migration guide (#20469)
1 parent 1681569 commit 61d1337

File tree

2 files changed

+180
-0
lines changed

2 files changed

+180
-0
lines changed

‎doc/dev/test_proxy_migration_guide.md‎

Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
# Guide for migrating to the test proxy from vcrpy
2+
3+
This guide describes the changes that service SDKs should make to their test frameworks in order to take advantage of
4+
the Azure SDK test proxy.
5+
6+
Documentation of the motivations and goals of the test proxy can be found [here][general_docs] in the azure-sdk-tools
7+
GitHub repository, and documentation of how to set up and use the proxy can be found [here][detailed_docs].
8+
9+
## Update existing tests
10+
11+
### Current test structure
12+
13+
Test classes currently inherit from AzureTestCase, and test methods can optionally use decorators:
14+
15+
```py
16+
from devtools_testutils import AzureTestCase
17+
18+
class TestExample(AzureTestCase):
19+
20+
def test_example(self):
21+
...
22+
23+
@ExamplePreparer()
24+
def test_example_with_preparer(self):
25+
...
26+
```
27+
28+
### New test structure
29+
30+
To use the proxy, test classes should inherit from AzureRecordedTestCase and recorded test methods should use a
31+
RecordedByProxy decorator:
32+
33+
```py
34+
from devtools_testutils import AzureRecordedTestCase, RecordedByProxy
35+
36+
class TestExample(AzureRecordedTestCase):
37+
38+
@RecordedByProxy
39+
def test_example(self):
40+
...
41+
42+
@ExamplePreparer()
43+
@RecordedByProxy
44+
def test_example_with_preparer(self):
45+
...
46+
```
47+
48+
For async tests, import the RecordedByProxyAsync decorator from `devtools_testutils.aio` and use it in the same
49+
way as RecordedByProxy.
50+
51+
> **Note:** since AzureRecordedTestCase doesn't inherit from `unittest.TestCase`, test class names need to start
52+
> with "Test" in order to be properly collected by pytest by default. For more information, please refer to
53+
> [pytest's documentation][pytest_collection].
54+
55+
## Run the tests
56+
57+
### Perform one-time setup
58+
59+
The test proxy is made available for your tests via a Docker container. Some tests require an SSL connection to work, so
60+
the Docker image used for the container has a certificate imported that you need to trust on your machine. Instructions
61+
on how to do so can be found [here][proxy_cert_docs].
62+
63+
### Start the proxy server
64+
65+
There is a [PowerShell script][docker_start_proxy] in `eng/common/testproxy` that will fetch the proxy Docker image if
66+
you don't already have it, and will start or stop a container running the image for you. You can run the following
67+
command from the root of the `azure-sdk-for-python` directory to start the container whenever you want to make the test
68+
proxy available for running tests:
69+
70+
```powershell
71+
.\eng\common\testproxy\docker-start-proxy.ps1 "start"
72+
```
73+
74+
Note that the proxy is available as long as the container is running. In other words, you don't need to start and
75+
stop the container for each test run or between tests for different SDKs. You can run the above command in the morning
76+
and just stop the container whenever you'd like. To stop the container, run the same command but with `"stop"` in place
77+
of `"start"`. In the future, the proxy container will be set up and started automatically when tests are run, and
78+
starting it manually will be optional.
79+
80+
For more details on proxy startup, please refer to the [proxy documentation][detailed_docs].
81+
82+
### Record or play back tests
83+
84+
Configuring live and playback tests is done with the `AZURE_TEST_RUN_LIVE` environment variable. When this variable is
85+
set to "true" or "yes", live tests will run and produce recordings. When this variable is set to "false" or "no", or
86+
not set at all, tests will run in playback mode and attempt to match existing recordings.
87+
88+
Recordings for a given package will end up in that package's `/tests/recordings` directory, just like they currently
89+
do.
90+
91+
> **Note:** at this time, support for configuring live or playback tests with a `testsettings_local.cfg` file has been
92+
> deprecated in favor of using just `AZURE_TEST_RUN_LIVE`.
93+
94+
### Register sanitizers
95+
96+
Since the test proxy doesn't use [`vcrpy`][vcrpy], tests don't use a scrubber to sanitize values in recordings.
97+
Instead, sanitizers (as well as matchers and transforms) can be registered on the proxy as detailed in
98+
[this][sanitizers] section of the proxy documentation. At the time of writing, sanitizers can be registered via the
99+
`add_sanitizer` method in `devtools_testutils`.
100+
101+
Sanitizers, matchers, and transforms remain registered until the proxy container is stopped, so for any sanitizers that
102+
are shared by different tests, using a session fixture declared in a `conftest.py` file is recommended. Please refer to
103+
[pytest's scoped fixture documentation][pytest_fixtures] for more details.
104+
105+
For example, to sanitize URIs in recordings, you can set up a URI sanitizer for all tests in the pytest session by
106+
adding something like the following in the package's `conftest.py` file:
107+
108+
```python
109+
from devtools_testutils import add_sanitizer
110+
111+
@pytest.fixture(scope="session")
112+
def sanitize_uris():
113+
add_sanitizer(ProxyRecordingSanitizer.URI, value="fakeendpoint")
114+
```
115+
116+
`add_sanitizer` accepts a sanitizer, matcher, or transform type from the ProxyRecordingSanitizer enum as a required
117+
parameter. Keyword-only arguments can be provided to customize the sanitizer; for example, in the snippet above, any
118+
request URIs that match the default URI regular expression will have their domain name replaced with "fakeendpoint". A
119+
request made to `https://tableaccount.table.core.windows.net` will be recorded as being made to
120+
`https://fakeendpoint.table.core.windows.net`.
121+
122+
## Implementation details
123+
124+
### What does the test proxy do?
125+
126+
The gist of the test proxy is that it stands in between your tests and the service. What this means is that test
127+
requests which would usually go straight to the service should instead point to the locally-hosted test proxy.
128+
129+
For example, if an operation would typically make a GET request to
130+
`https://fakeazsdktestaccount.table.core.windows.net/Tables`, that operation should now be sent to
131+
`https://localhost:5001/Tables` instead. The original endpoint should be stored in an `x-recording-upstream-base-uri` --
132+
the proxy will send the original request and record the result.
133+
134+
The RecordedByProxy and RecordedByProxyAsync decorators patch test requests to do this for you.
135+
136+
### How does the test proxy know when and what to record or play back?
137+
138+
This is achieved by making POST requests to the proxy server that say whether to start or stop recording or playing
139+
back, as well as what test is being run.
140+
141+
To start recording a test, the server should be primed with a POST request:
142+
143+
```
144+
URL: https://localhost:5001/record/start
145+
headers {
146+
"x-recording-file": "<path-to-test>/recordings/<testfile>.<testname>"
147+
}
148+
```
149+
150+
This will return a recording ID in an `x-recording-id` header. This ID should be sent as an `x-recording-id` header in
151+
all further requests during the test.
152+
153+
After the test has finished, a POST request should be sent to indicate that recording is complete:
154+
155+
```
156+
URL: https://localhost:5001/record/stop
157+
headers {
158+
"x-recording-id": "<x-recording-id>"
159+
}
160+
```
161+
162+
Running tests in playback follows the same pattern, except that requests will be sent to `/playback/start` and
163+
`/playback/stop` instead. A header, `x-recording-mode`, should be set to `record` for all requests when recording and
164+
`playback` when playing recordings back. More details can be found [here][detailed_docs].
165+
166+
The RecordedByProxy and RecordedByProxyAsync decorators send the appropriate requests at the start and end of each test
167+
case.
168+
169+
[detailed_docs]: https://github.com/Azure/azure-sdk-tools/tree/main/tools/test-proxy/Azure.Sdk.Tools.TestProxy/README.md
170+
[docker_start_proxy]: https://github.com/Azure/azure-sdk-for-python/blob/main/eng/common/testproxy/docker-start-proxy.ps1
171+
[general_docs]: https://github.com/Azure/azure-sdk-tools/blob/main/tools/test-proxy/README.md
172+
[proxy_cert_docs]: https://github.com/Azure/azure-sdk-tools/blob/main/tools/test-proxy/documentation/trusting-cert-per-language.md
173+
[pytest_collection]: https://docs.pytest.org/latest/goodpractices.html#test-discovery
174+
[pytest_fixtures]: https://docs.pytest.org/latest/fixture.html#scope-sharing-fixtures-across-classes-modules-packages-or-session
175+
[sanitizers]: https://github.com/Azure/azure-sdk-tools/blob/main/tools/test-proxy/Azure.Sdk.Tools.TestProxy/README.md#session-and-test-level-transforms-sanitiziers-and-matchers
176+
[vcrpy]: https://vcrpy.readthedocs.io

‎tools/azure-sdk-tools/devtools_testutils/proxy_testcase.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,10 @@
2424
from .config import PROXY_URL
2525

2626

27+
# To learn about how to migrate SDK tests to the test proxy, please refer to the migration guide at
28+
# https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/test_proxy_migration_guide.md
29+
30+
2731
# defaults
2832
RECORDING_START_URL = "{}/record/start".format(PROXY_URL)
2933
RECORDING_STOP_URL = "{}/record/stop".format(PROXY_URL)

0 commit comments

Comments
 (0)