Skip to content

Commit ce307fe

Browse files
scottaddiescottaddie
authored
Final consistency review of Monitor Query README (#20667)
* Final consistency review of Monitor Query README * Add rate limits section
1 parent c4d64b2 commit ce307fe

File tree

1 file changed

+113
-123
lines changed

1 file changed

+113
-123
lines changed

‎sdk/monitor/azure-monitor-query/README.md‎

Lines changed: 113 additions & 123 deletions
Original file line numberDiff line numberDiff line change
@@ -1,38 +1,43 @@
11
# Azure Monitor Query client library for Python
22

3-
Azure Monitor helps you maximize the availability and performance of your apps. It delivers a comprehensive solution for collecting, analyzing, and acting on telemetry from your cloud and on-premises environments.
3+
The Azure Monitor Query client library is used to execute read-only queries against [Azure Monitor][azure_monitor_overview]'s two data platforms:
44

5-
All data collected by Azure Monitor fits into one of two fundamental types:
5+
- [Logs](https://docs.microsoft.com/azure/azure-monitor/logs/data-platform-logs) - Collects and organizes log and performance data from monitored resources. Data from different sources such as platform logs from Azure services, log and performance data from virtual machines agents, and usage and performance data from apps can be consolidated into a single [Azure Log Analytics workspace](https://docs.microsoft.com/azure/azure-monitor/logs/data-platform-logs#log-analytics-workspaces). The various data types can be analyzed together using the [Kusto Query Language][kusto_query_language].
6+
- [Metrics](https://docs.microsoft.com/azure/azure-monitor/essentials/data-platform-metrics) - Collects numeric data from monitored resources into a time series database. Metrics are numerical values that are collected at regular intervals and describe some aspect of a system at a particular time. Metrics are lightweight and capable of supporting near real-time scenarios, making them particularly useful for alerting and fast detection of issues.
67

7-
- **Metrics** - Numerical values that describe some aspect of a system at a particular time. They're lightweight and can support near real-time scenarios.
8-
- **Logs** - Disparate types of data organized into records with different sets of properties for each type. Performance data and telemetry such as events, exceptions, and traces are stored as logs.
8+
**Resources:**
99

10-
To programmatically analyze these data sources, the Azure Monitor Query client library can be used.
11-
12-
[Source code][python-query-src] | [Package (PyPI)][python-query-pypi] | [API reference documentation][python-query-ref-docs] | [Product documentation][python-query-product-docs] | [Samples][python-query-samples] | [Changelog][python-query-changelog]
10+
- [Source code][source]
11+
- [Package (PyPI)][package]
12+
- [API reference documentation][python-query-ref-docs]
13+
- [Service documentation][azure_monitor_overview]
14+
- [Samples][samples]
15+
- [Change log][changelog]
1316

1417
## Getting started
1518

1619
### Prerequisites
1720

18-
- Python 2.7, or 3.6 or later.
19-
- An [Azure subscription][azure_subscription].
21+
- Python 2.7, or 3.6 or later
22+
- An [Azure subscription][azure_subscription]
23+
- To query Logs, you need an [Azure Log Analytics workspace][azure_monitor_create_using_portal].
24+
- To query Metrics, you need an Azure resource of any kind (Storage Account, Key Vault, Cosmos DB, etc.).
2025

2126
### Install the package
2227

2328
Install the Azure Monitor Query client library for Python with [pip][pip]:
2429

2530
```bash
26-
pip install azure-monitor-query --pre
31+
pip install azure-monitor-query
2732
```
2833

2934
### Create the client
3035

31-
To interact with the Azure Monitor service, create an instance of a token credential. Use that instance when creating a `LogsQueryClient` or `MetricsQueryClient`.
36+
An authenticated client is required to query Logs or Metrics. The library includes both synchronous and asynchronous forms of the clients. To authenticate, create an instance of a token credential. Use that instance when creating a `LogsQueryClient` or `MetricsQueryClient`. The following examples use `DefaultAzureCredential` from the [azure-identity](https://pypi.org/project/azure-identity/) package.
3237

3338
#### Synchronous clients
3439

35-
Consider the following example, which creates synchronous clients for both logs and metrics querying:
40+
Consider the following example, which creates synchronous clients for both Logs and Metrics querying:
3641

3742
```python
3843
from azure.identity import DefaultAzureCredential
@@ -56,37 +61,23 @@ async_logs_client = LogsQueryClient(credential)
5661
async_metrics_client = MetricsQueryClient(credential)
5762
```
5863

59-
## Key concepts
60-
61-
### Logs
62-
63-
Azure Monitor Logs collects and organizes log and performance data from monitored resources. Data from different sources can be consolidated into a single workspace. Examples of data sources include:
64-
65-
- Platform logs from Azure services.
66-
- Log and performance data from virtual machine agents.
67-
- Usage and performance data from apps.
64+
### Execute the query
6865

69-
#### Azure Log Analytics workspaces
66+
For examples of Logs and Metrics queries, see the [Examples](#examples) section.
7067

71-
Data collected by Azure Monitor Logs is stored in one or more [Log Analytics workspaces](https://docs.microsoft.com/azure/azure-monitor/logs/data-platform-logs#log-analytics-workspaces). The workspace defines the:
72-
73-
- Geographic location of the data.
74-
- Access rights defining which users can access data.
75-
- Configuration settings, such as the pricing tier and data retention.
76-
77-
#### Log queries
68+
## Key concepts
7869

79-
Data from the disparate sources can be analyzed together using [Kusto Query Language (KQL)](https://docs.microsoft.com/azure/data-explorer/kusto/query/)—the same query language used by [Azure Data Explorer](https://docs.microsoft.com/azure/data-explorer/data-explorer-overview). Data is retrieved from a Log Analytics workspace using a KQL query—a read-only request to process data and return results. For more information, see [Log queries in Azure Monitor](https://docs.microsoft.com/azure/azure-monitor/logs/log-query-overview).
70+
### Logs query rate limits and throttling
8071

81-
### Metrics
72+
Each Azure Active Directory user is able to make up to 200 requests per 30 seconds, with no cap on the total calls per day. If requests are made at a rate higher than this, these requests will receive HTTP status code 429 (Too Many Requests) along with the `Retry-After: <delta-seconds>` header. The header indicates the number of seconds until requests to this app are likely to be accepted.
8273

83-
Azure Monitor Metrics collects numeric data from monitored resources into a time series database. Metrics are collected at regular intervals and describe some aspect of a system at a particular time. Metrics in Azure Monitor are lightweight and can support near real-time scenarios. They're useful for alerting and fast detection of issues. Metrics can be:
74+
In addition to call rate limits and daily quota caps, there are limits on queries themselves. Queries cannot:
8475

85-
- Analyzed interactively with [Metrics Explorer](https://docs.microsoft.com/azure/azure-monitor/essentials/metrics-getting-started).
86-
- Used to receive notifications with an alert when a value crosses a threshold.
87-
- Visualized in a workbook or dashboard.
76+
- Return more than 500,000 rows.
77+
- Return more than 64,000,000 bytes (~61 MiB total data).
78+
- Run longer than 10 minutes by default. See this for details.
8879

89-
#### Metrics data structure
80+
### Metrics data structure
9081

9182
Each set of metric values is a time series with the following characteristics:
9283

@@ -99,19 +90,20 @@ Each set of metric values is a time series with the following characteristics:
9990

10091
## Examples
10192

102-
- [Single logs query](#single-logs-query)
93+
- [Logs query](#logs-query)
10394
- [Specify timespan](#specify-timespan)
104-
- [Set logs query timeout](#set-logs-query-timeout)
95+
- [Handle logs query response](#handle-logs-query-response)
10596
- [Batch logs query](#batch-logs-query)
106-
- [Query metrics](#query-metrics)
107-
- [Handle metrics response](#handle-metrics-response)
108-
- [Example of handling response](#example-of-handling-response)
109-
- [Advanced scenarios](#advanced-scenarios)
97+
- [Advanced logs query scenarios](#advanced-logs-query-scenarios)
98+
- [Set logs query timeout](#set-logs-query-timeout)
11099
- [Query multiple workspaces](#query-multiple-workspaces)
100+
- [Metrics query](#metrics-query)
101+
- [Handle metrics query response](#handle-metrics-query-response)
102+
- [Example of handling response](#example-of-handling-response)
111103

112-
### Single logs query
104+
### Logs query
113105

114-
This example shows getting a log query. To handle the response and view it in a tabular form, the [pandas](https://pypi.org/project/pandas/) library is used. See the [samples][python-query-samples] if you choose not to use pandas.
106+
This example shows getting a logs query. To handle the response and view it in a tabular form, the [pandas](https://pypi.org/project/pandas/) library is used. See the [samples][samples] if you choose not to use pandas.
115107

116108
#### Specify timespan
117109

@@ -150,29 +142,45 @@ for table in response.tables:
150142
print(df)
151143
```
152144

153-
#### Set logs query timeout
145+
#### Handle logs query response
154146

155-
The following example shows setting a server timeout in seconds. A gateway timeout is raised if the query takes more time than the mentioned timeout. The default is 180 seconds and can be set up to 10 minutes (600 seconds).
147+
The `query` API returns the `LogsQueryResult` while the `batch_query` API returns list of `LogsQueryResult`. Here's a hierarchy of the response:
148+
149+
```
150+
LogsQueryResult
151+
|---statistics
152+
|---visualization
153+
|---error
154+
|---tables (list of `LogsTable` objects)
155+
|---name
156+
|---rows
157+
|---columns (list of `LogsTableColumn` objects)
158+
|---name
159+
|---type
160+
```
161+
162+
For example, to handle a logs query response with tables and display it using pandas:
156163

157164
```python
158-
import os
159-
import pandas as pd
160-
from azure.monitor.query import LogsQueryClient
161-
from azure.identity import DefaultAzureCredential
165+
table = response.tables[0]
166+
df = pd.DataFrame(table.rows, columns=[col.name for col in table.columns])
167+
```
162168

163-
credential = DefaultAzureCredential()
164-
client = LogsQueryClient(credential)
169+
A full sample can be found [here](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/monitor/azure-monitor-query/samples/sample_log_query_client.py).
165170

166-
response = client.query(
167-
os.environ['LOG_WORKSPACE_ID'],
168-
"range x from 1 to 10000000000 step 1 | count",
169-
server_timeout=1,
170-
)
171+
In a similar fashion, to handle a batch logs query response:
172+
173+
```python
174+
for result in response:
175+
table = result.tables[0]
176+
df = pd.DataFrame(table.rows, columns=[col.name for col in table.columns])
171177
```
172178

179+
A full sample can be found [here](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/monitor/azure-monitor-query/samples/sample_batch_query.py).
180+
173181
### Batch logs query
174182

175-
The following example demonstrates sending multiple queries at the same time using batch query API. The queries can either be represented as a list of `LogQueryRequest` objects or a dictionary. This example uses the former approach.
183+
The following example demonstrates sending multiple queries at the same time using batch query API. The queries can either be represented as a list of `LogsBatchQuery` objects or a dictionary. This example uses the former approach.
176184

177185
```python
178186
import os
@@ -196,7 +204,7 @@ requests = [
196204
timespan=(datetime(2021, 6, 2), timedelta(hours=1)),
197205
workspace_id=os.environ['LOG_WORKSPACE_ID']
198206
),
199-
LogsBatchQueryRequest(
207+
LogsBatchQuery(
200208
query= "AppRequests | take 2",
201209
workspace_id=os.environ['LOG_WORKSPACE_ID']
202210
),
@@ -213,43 +221,51 @@ for rsp in response:
213221
print(df)
214222
```
215223

216-
#### Handling the response for Logs Query
224+
### Advanced logs query scenarios
217225

218-
The `query` API returns the `LogsQueryResult` while the `batch_query` API returns list of `LogsQueryResult`.
226+
#### Set logs query timeout
219227

220-
Here is a heirarchy of the response:
228+
The following example shows setting a server timeout in seconds. A gateway timeout is raised if the query takes more time than the mentioned timeout. The default is 180 seconds and can be set up to 10 minutes (600 seconds).
221229

222-
```
223-
LogsQueryResult
224-
|---statistics
225-
|---visualization
226-
|---error
227-
|---tables (list of `LogsTable` objects)
228-
|---name
229-
|---rows
230-
|---columns (list of `LogsTableColumn` objects)
231-
|---name
232-
|---type
233-
```
230+
```python
231+
import os
232+
import pandas as pd
233+
from azure.monitor.query import LogsQueryClient
234+
from azure.identity import DefaultAzureCredential
234235

235-
So, to handle a response with tables and display it using pandas,
236+
credential = DefaultAzureCredential()
237+
client = LogsQueryClient(credential)
236238

237-
```python
238-
table = response.tables[0]
239-
df = pd.DataFrame(table.rows, columns=[col.name for col in table.columns])
239+
response = client.query(
240+
os.environ['LOG_WORKSPACE_ID'],
241+
"range x from 1 to 10000000000 step 1 | count",
242+
server_timeout=1,
243+
)
240244
```
241-
A full sample can be found [here](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/monitor/azure-monitor-query/samples/sample_log_query_client.py).
242245

243-
In a very similar fashion, to handle a batch response,
246+
#### Query multiple workspaces
247+
248+
The same logs query can be executed across multiple Log Analytics workspaces. In addition to the Kusto query, the following parameters are required:
249+
250+
- `workspace_id` - The first (primary) workspace ID.
251+
- `additional_workspaces` - A list of workspaces, excluding the workspace provided in the `workspace_id` parameter. The parameter's list items may consist of the following identifier formats:
252+
- Qualified workspace names
253+
- Workspace IDs
254+
- Azure resource IDs
255+
256+
For example, the following query executes in three workspaces:
244257

245258
```python
246-
for result in response:
247-
table = result.tables[0]
248-
df = pd.DataFrame(table.rows, columns=[col.name for col in table.columns])
259+
client.query(
260+
<workspace_id>,
261+
query,
262+
additional_workspaces=['<workspace 2>', '<workspace 3>']
263+
)
249264
```
250-
A full sample can be found [here](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/monitor/azure-monitor-query/samples/sample_batch_query.py).
251265

252-
### Query metrics
266+
A full sample can be found [here](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/monitor/azure-monitor-query/samples/sample_log_query_multiple_workspaces.py).
267+
268+
### Metrics query
253269

254270
The following example gets metrics for an Event Grid subscription. The resource URI is that of an event grid topic.
255271

@@ -285,7 +301,7 @@ for metric in response.metrics:
285301
print(metric_value.time_stamp)
286302
```
287303

288-
### Handle metrics response
304+
#### Handle metrics query response
289305

290306
The metrics query API returns a `MetricsResult` object. The `MetricsResult` object contains properties such as a list of `Metric`-typed objects, `granularity`, `namespace`, and `timespan`. The `Metric` objects list can be accessed using the `metrics` param. Each `Metric` object in this list contains a list of `TimeSeriesElement` objects. Each `TimeSeriesElement` contains `data` and `metadata_values` properties. In visual form, the object hierarchy of the response resembles the following structure:
291307

@@ -337,30 +353,6 @@ for metric in response.metrics:
337353
)
338354
```
339355

340-
### Advanced scenarios
341-
342-
#### Query multiple workspaces
343-
344-
The same log query can be executed across multiple Log Analytics workspaces. In addition to the KQL query, the following parameters are required:
345-
346-
- `workspace_id` - The first (primary) workspace ID.
347-
- `additional_workspaces` - A list of workspaces, excluding the workspace provided in the `workspace_id` parameter. The parameter's list items may consist of the following identifier formats:
348-
- Qualified workspace names
349-
- Workspace IDs
350-
- Azure resource IDs
351-
352-
For example, the following query executes in three workspaces:
353-
354-
```python
355-
client.query(
356-
<workspace_id>,
357-
query,
358-
additional_workspaces=['<workspace 2>', '<workspace 3>']
359-
)
360-
```
361-
362-
A full sample can be found [here](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/monitor/azure-monitor-query/samples/sample_log_query_multiple_workspaces.py).
363-
364356
## Troubleshooting
365357

366358
Enable the `azure.monitor.query` logger to collect traces from the library.
@@ -379,9 +371,7 @@ Optional keyword arguments can be passed in at the client and per-operation leve
379371

380372
## Next steps
381373

382-
### Additional documentation
383-
384-
For more extensive documentation, see the [Azure Monitor Query documentation][python-query-product-docs].
374+
To learn more about Azure Monitor, see the [Azure Monitor service documentation][azure_monitor_overview].
385375

386376
## Contributing
387377

@@ -393,19 +383,19 @@ This project has adopted the [Microsoft Open Source Code of Conduct][code_of_con
393383

394384
<!-- LINKS -->
395385

396-
[azure_cli_link]: https://pypi.org/project/azure-cli/
397-
[python-query-src]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/monitor/azure-monitor-query/
398-
[python-query-pypi]: https://aka.ms/azsdk-python-monitor-query-pypi
399-
[python-query-product-docs]: https://docs.microsoft.com/azure/azure-monitor/
400-
[python-query-ref-docs]: https://docs.microsoft.com/python/api/overview/azure/monitor-query-readme?view=azure-python-preview
401-
[python-query-samples]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/monitor/azure-monitor-query/samples
402-
[python-query-changelog]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/monitor/azure-monitor-query/CHANGELOG.md
403-
[pip]: https://pypi.org/project/pip/
404-
405386
[azure_core_exceptions]: https://aka.ms/azsdk/python/core/docs#module-azure.core.exceptions
406-
[python_logging]: https://docs.python.org/3/library/logging.html
407387
[azure_core_ref_docs]: https://aka.ms/azsdk/python/core/docs
388+
[azure_monitor_create_using_portal]: https://docs.microsoft.com/azure/azure-monitor/logs/quick-create-workspace
389+
[azure_monitor_overview]: https://docs.microsoft.com/azure/azure-monitor/
408390
[azure_subscription]: https://azure.microsoft.com/free/python/
391+
[changelog]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/monitor/azure-monitor-query/CHANGELOG.md
392+
[kusto_query_language]: https://docs.microsoft.com/azure/data-explorer/kusto/query/
393+
[package]: https://aka.ms/azsdk-python-monitor-query-pypi
394+
[pip]: https://pypi.org/project/pip/
395+
[python_logging]: https://docs.python.org/3/library/logging.html
396+
[python-query-ref-docs]: https://docs.microsoft.com/python/api/overview/azure/monitor-query-readme?view=azure-python-preview
397+
[samples]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/monitor/azure-monitor-query/samples
398+
[source]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/monitor/azure-monitor-query/
409399

410400
[cla]: https://cla.microsoft.com
411401
[code_of_conduct]: https://opensource.microsoft.com/codeofconduct/

0 commit comments

Comments
 (0)