Skip to content
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
0616843
throttling policy fix in core-http
HarshaNalluru Jun 17, 2021
10a3816
remove comment
HarshaNalluru Jun 17, 2021
8bf61fe
typo
HarshaNalluru Jun 17, 2021
d876b54
import core-amqp delay from core-util
HarshaNalluru Jun 17, 2021
db74514
import core-util
HarshaNalluru Jun 17, 2021
01bd7c4
Merge branch 'master' of https://github.com/Azure/azure-sdk-for-js in…
HarshaNalluru Jun 17, 2021
a0307af
import { delay } from "@azure/core-util";
HarshaNalluru Jun 17, 2021
78814aa
changelog
HarshaNalluru Jun 17, 2021
ff9cb2a
throw abort error check
HarshaNalluru Jun 17, 2021
d003ea8
revert core-rest-pipeline
HarshaNalluru Jun 17, 2021
15f9428
Merge branch 'main' of https://github.com/Azure/azure-sdk-for-js into…
HarshaNalluru Jun 18, 2021
c774368
import { delay } from "@azure/core-util";
HarshaNalluru Jun 18, 2021
daedfbb
fix linter errors
HarshaNalluru Jun 18, 2021
0dd2974
fix build failures
HarshaNalluru Jun 18, 2021
6c4ed84
export { delay } from "@azure/core-util";
HarshaNalluru Jun 18, 2021
0ad0a26
API Report
HarshaNalluru Jun 18, 2021
f7e294a
Merge branch 'main' of https://github.com/Azure/azure-sdk-for-js into…
HarshaNalluru Jun 23, 2021
d80e0d9
Merge branch 'main' of https://github.com/Azure/azure-sdk-for-js into…
HarshaNalluru Jun 23, 2021
97c6f0a
have 2 copies of delay
HarshaNalluru Jun 23, 2021
99f4d41
keep duplicates
HarshaNalluru Jun 23, 2021
8d77016
API Report
HarshaNalluru Jun 23, 2021
ed5e621
retain core-util
HarshaNalluru Jun 23, 2021
8977a42
get rid of circular dependencies
HarshaNalluru Jun 24, 2021
3029418
value should be the second argument
HarshaNalluru Jun 24, 2021
75ad58e
update delay to have options bag and additional feedback
HarshaNalluru Jun 24, 2021
6ede956
make async functions and typeguard feedback
HarshaNalluru Jun 24, 2021
37018a1
update app-config and API Report
HarshaNalluru Jun 24, 2021
1a6555b
HttpMockFacade
HarshaNalluru Jun 24, 2021
58b5855
should honor the abort signal passed
HarshaNalluru Jun 24, 2021
a3a0a04
remove nock
HarshaNalluru Jun 24, 2021
9b7f233
satisfy linter
HarshaNalluru Jun 24, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions sdk/appconfiguration/app-configuration/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@
"@azure/core-http": "^1.2.0",
"@azure/core-paging": "^1.1.1",
"@azure/core-tracing": "1.0.0-preview.11",
"@azure/core-util": "^1.0.0-beta.1",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is appconfiguration in beta? if not, it should not take a dep on a beta.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

App-config is beta, but we may soon GA in the next couple of months.
Why is core-util in beta anyways? Can't we make that GA instead?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because we did not settle on the public surface for it yet.

appconfig is in beta but core-http and core-amqp are not IIRC and you added core-util as a dep there. My recommendation is to keep the local implementation of delay for now and have it moved to core-util in another PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really don't want to keep 3 copies of delay (core-http, core-amqp and app-config).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really don't want to keep 3 copies of delay (core-http, core-amqp and app-config).

Why cannot App Config use the one in core-http?

Since this PR's main objective is around fixing an issue with the retry policy, can we stick to just that in this PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ramya-rao-a So, you're suggesting core-http and core-amqp would have delay? Can do that if weare fine with 2 copies.

(What's the point of core-util then?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the point of core-util then

It is pending the on the conclusion on the discussion we had today on the subject of code sharing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We now have two copies of (advanced) delay.. core-amqp and core-http.

Logged #15930 for core-util discussion.

"tslib": "^2.2.0"
},
"devDependencies": {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.

import { AbortError, AbortSignalLike } from "@azure/abort-controller";
import { AbortError } from "@azure/abort-controller";
import {
BaseRequestPolicy,
RequestPolicy,
Expand All @@ -12,7 +12,7 @@ import {
Constants,
RestError
} from "@azure/core-http";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now we just need to move to corev2 so we can fix #6484

import { isDefined } from "../internal/typeguards";
import { delay } from "@azure/core-util";

/**
* @internal
Expand All @@ -27,55 +27,6 @@ export function throttlingRetryPolicy(): RequestPolicyFactory {

const StandardAbortMessage = "The operation was aborted.";

/**
* A wrapper for setTimeout that resolves a promise after t milliseconds.
* @param delayInMs - The number of milliseconds to be delayed.
* @param abortSignal - The abortSignal associated with containing operation.
* @param abortErrorMsg - The abort error message associated with containing operation.
* @returns - Resolved promise
*/
export function delay(
delayInMs: number,
abortSignal?: AbortSignalLike,
abortErrorMsg?: string
): Promise<void> {
return new Promise((resolve, reject) => {
let timer: ReturnType<typeof setTimeout> | undefined = undefined;
let onAborted: (() => void) | undefined = undefined;

const rejectOnAbort = (): void => {
return reject(new AbortError(abortErrorMsg ? abortErrorMsg : StandardAbortMessage));
};

const removeListeners = (): void => {
if (abortSignal && onAborted) {
abortSignal.removeEventListener("abort", onAborted);
}
};

onAborted = (): void => {
if (isDefined(timer)) {
clearTimeout(timer);
}
removeListeners();
return rejectOnAbort();
};

if (abortSignal && abortSignal.aborted) {
return rejectOnAbort();
}

timer = setTimeout(() => {
removeListeners();
resolve();
}, delayInMs);

if (abortSignal) {
abortSignal.addEventListener("abort", onAborted);
}
});
}

/**
* This policy is a close copy of the ThrottlingRetryPolicy class from
* core-http with modifications to work with how AppConfig is currently
Expand Down
1 change: 1 addition & 0 deletions sdk/core/core-amqp/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@
},
"dependencies": {
"@azure/abort-controller": "^1.0.0",
"@azure/core-util": "^1.0.0-beta.1",
"@azure/core-auth": "^1.3.0",
"@azure/logger": "^1.0.0",
"buffer": "^5.2.1",
Expand Down
2 changes: 1 addition & 1 deletion sdk/core/core-amqp/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,12 @@ export {
StandardAbortMessage
} from "./errors";
export {
delay,
parseConnectionString,
defaultCancellableLock,
ParsedOutput,
WebSocketOptions
} from "./util/utils";
export { delay } from "@azure/core-util";
export { AmqpAnnotatedMessage } from "./amqpAnnotatedMessage";
export { logger } from "./log";
export * from "./internals";
Expand Down
54 changes: 0 additions & 54 deletions sdk/core/core-amqp/src/util/utils.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,7 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.

import { AbortError, AbortSignalLike } from "@azure/abort-controller";
import { WebSocketImpl } from "rhea-promise";
import { isDefined } from "./typeGuards";
import { StandardAbortMessage } from "../errors";
import { CancellableAsyncLock, CancellableAsyncLockImpl } from "./lock";

/**
Expand Down Expand Up @@ -168,57 +165,6 @@ export class Timeout {
}
}

/**
* A wrapper for setTimeout that resolves a promise after t milliseconds.
* @param delayInMs - The number of milliseconds to be delayed.
* @param abortSignal - The abortSignal associated with containing operation.
* @param abortErrorMsg - The abort error message associated with containing operation.
* @param value - The value to be resolved with after a timeout of t milliseconds.
* @returns - Resolved promise
*/
export function delay<T>(
delayInMs: number,
abortSignal?: AbortSignalLike,
abortErrorMsg?: string,
value?: T
): Promise<T | void> {
return new Promise((resolve, reject) => {
let timer: ReturnType<typeof setTimeout> | undefined = undefined;
let onAborted: (() => void) | undefined = undefined;

const rejectOnAbort = (): void => {
return reject(new AbortError(abortErrorMsg ? abortErrorMsg : StandardAbortMessage));
};

const removeListeners = (): void => {
if (abortSignal && onAborted) {
abortSignal.removeEventListener("abort", onAborted);
}
};

onAborted = (): void => {
if (isDefined(timer)) {
clearTimeout(timer);
}
removeListeners();
return rejectOnAbort();
};

if (abortSignal && abortSignal.aborted) {
return rejectOnAbort();
}

timer = setTimeout(() => {
removeListeners();
resolve(value);
}, delayInMs);

if (abortSignal) {
abortSignal.addEventListener("abort", onAborted);
}
});
}

/**
* @internal
*
Expand Down
1 change: 1 addition & 0 deletions sdk/core/core-http/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
### Fixed

- Fixed an issue where `proxySettings` does not work when there is username but no password [Issue 15720](https://github.com/Azure/azure-sdk-for-js/issues/15720)
- Throttling retry policy respects abort signal [#15796](https://github.com/Azure/azure-sdk-for-js/issues/15796)

## 1.2.6 (2021-06-14)

Expand Down
2 changes: 2 additions & 0 deletions sdk/core/core-http/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@
"@azure/core-asynciterator-polyfill": "^1.0.0",
"@azure/core-auth": "^1.3.0",
"@azure/core-tracing": "1.0.0-preview.11",
"@azure/core-util": "^1.0.0-beta.1",
"@azure/logger": "^1.0.0",
"@types/node-fetch": "^2.5.0",
"@types/tunnel": "^0.0.1",
Expand Down Expand Up @@ -179,6 +180,7 @@
"karma-sourcemap-loader": "^0.3.8",
"mocha": "^7.1.1",
"mocha-junit-reporter": "^1.18.0",
"nock": "^12.0.3",
"npm-run-all": "^4.1.5",
"nyc": "^14.0.0",
"prettier": "^1.16.4",
Expand Down
11 changes: 9 additions & 2 deletions sdk/core/core-http/src/policies/throttlingRetryPolicy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ import {
import { WebResourceLike } from "../webResource";
import { HttpOperationResponse } from "../httpOperationResponse";
import { Constants } from "../util/constants";
import { delay } from "../util/utils";
import { delay } from "@azure/core-util";
import { AbortError } from "@azure/abort-controller";

type ResponseHandler = (
httpRequest: WebResourceLike,
Expand All @@ -26,6 +27,8 @@ export function throttlingRetryPolicy(): RequestPolicyFactory {
};
}

const StandardAbortMessage = "The operation was aborted.";

/**
* To learn more, please refer to
* https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-request-limits,
Expand Down Expand Up @@ -67,7 +70,11 @@ export class ThrottlingRetryPolicy extends BaseRequestPolicy {
retryAfterHeader
);
if (delayInMs) {
return delay(delayInMs).then((_: any) => this._nextPolicy.sendRequest(httpRequest));
await delay(delayInMs, httpRequest.abortSignal, StandardAbortMessage);
if (httpRequest.abortSignal?.aborted) {
throw new AbortError(StandardAbortMessage);
}
return await this._nextPolicy.sendRequest(httpRequest);
}
}

Expand Down
49 changes: 49 additions & 0 deletions sdk/core/core-http/test/policies/throttlingRetryPolicy.node.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
import nock from "nock";
import { Constants, ServiceClient } from "../../src/coreHttp";
import { AbortController } from "@azure/abort-controller";
import { assert } from "chai";

describe("Throttling retry policy", () => {
let client: ServiceClient;

beforeEach(function() {
if (!nock.isActive()) {
nock.activate();
}
nock("https://fakeservice.io:443")
.persist()
.put(/.*/g)
.reply(
Constants.HttpConstants.StatusCodes.TooManyRequests,
{
type: "https://fakeservice.io/errors/too-many-requests",
title: "Resource utilization has surpassed the assigned quota",
policy: "Total Requests",
status: Constants.HttpConstants.StatusCodes.TooManyRequests
},
["Retry-After", "10000"]
);
client = new ServiceClient();
});

afterEach(async function() {
nock.restore();
nock.cleanAll();
nock.enableNetConnect();
});

it("Should not retry forever (honors the abort signal passed)", async () => {
let errorWasThrown = false;
try {
await client.sendRequest({
url: "https://fakeservice.io/ABCD",
abortSignal: AbortController.timeout(100),
method: "PUT"
});
} catch (error) {
errorWasThrown = true;
assert.equal((error as any).name, "AbortError", "Unexpected error thrown");
}
assert.equal(errorWasThrown, true, "Error was not thrown");
});
});
3 changes: 2 additions & 1 deletion sdk/core/core-util/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,8 @@
"sideEffects": false,
"prettier": "@azure/eslint-plugin-azure-sdk/prettier.json",
"dependencies": {
"tslib": "^2.2.0"
"tslib": "^2.2.0",
"@azure/abort-controller": "^1.0.0"
},
"devDependencies": {
"@azure/dev-tool": "^1.0.0",
Expand Down
4 changes: 3 additions & 1 deletion sdk/core/core-util/review/core-util.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,10 @@

```ts

import { AbortSignalLike } from '@azure/abort-controller';

// @public
export function delay(timeInMs: number): Promise<void>;
export function delay<T>(delayInMs: number, abortSignal?: AbortSignalLike, abortErrorMsg?: string, value?: T): Promise<T | void>;

// @public
export const isNode: boolean;
Expand Down
56 changes: 51 additions & 5 deletions sdk/core/core-util/src/delay.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,57 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.

import { isDefined } from "./typeguards";
import { AbortError, AbortSignalLike } from "@azure/abort-controller";
const StandardAbortMessage = "The operation was aborted.";

/**
* A wrapper for setTimeout that resolves a promise after timeInMs milliseconds.
* @param timeInMs - The number of milliseconds to be delayed.
* @returns Promise that is resolved after timeInMs
* A wrapper for setTimeout that resolves a promise after delayInMs milliseconds.
* @param delayInMs - The number of milliseconds to be delayed.
* @param abortSignal - The abortSignal associated with containing operation.
* @param abortErrorMsg - The abort error message associated with containing operation.
* @param value - The value to be resolved with after a timeout of t milliseconds.
* @returns - Resolved promise
*/
export function delay(timeInMs: number): Promise<void> {
return new Promise((resolve) => setTimeout(() => resolve(), timeInMs));
export function delay<T>(
delayInMs: number,
abortSignal?: AbortSignalLike,
abortErrorMsg?: string,
value?: T
): Promise<T | void> {
return new Promise((resolve, reject) => {
let timer: ReturnType<typeof setTimeout> | undefined = undefined;
let onAborted: (() => void) | undefined = undefined;

const rejectOnAbort = (): void => {
return reject(new AbortError(abortErrorMsg ? abortErrorMsg : StandardAbortMessage));
};

const removeListeners = (): void => {
if (abortSignal && onAborted) {
abortSignal.removeEventListener("abort", onAborted);
}
};

onAborted = (): void => {
if (isDefined(timer)) {
clearTimeout(timer);
}
removeListeners();
return rejectOnAbort();
};

if (abortSignal && abortSignal.aborted) {
return rejectOnAbort();
}

timer = setTimeout(() => {
removeListeners();
resolve(value);
}, delayInMs);

if (abortSignal) {
abortSignal.addEventListener("abort", onAborted);
}
});
}
11 changes: 11 additions & 0 deletions sdk/core/core-util/src/typeguards.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.

/**
* Helper TypeGuard that checks if something is defined or not.
* @param thing - Anything
* @internal
*/
export function isDefined<T>(thing: T | undefined | null): thing is T {
return typeof thing !== "undefined" && thing !== null;
}