Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 21 additions & 23 deletions pipeline/outputs/azure_kusto.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,22 +55,21 @@ By default, Kusto will insert incoming ingestion data into a table by inferring
| `azure_kusto_buffer_key` | When buffering is `true`, set the Azure Kusto buffer key which must be specified when using multiple instances of Azure Kusto output plugin and buffering is enabled. | `key` |
| `blob_uri_length` | Set the length of generated blob URI before ingesting to Kusto. | `64` |
| `buffer_dir` | When buffering is `true`, specifies the location of directory where the buffered data will be stored. | `/tmp/fluent-bit/azure-kusto/` |
| `buffering_enabled` | Enable buffering into disk before ingesting into Azure Kusto. | `false` |
| `buffer_file_delete_early` | When buffering is `true`, whether to delete the buffered file early after successful blob creation. | `false` |
| `unify_tag` | Optional. This creates a single buffer file when the buffering mode is `On`. | `On` |
| `buffering_enabled` | Enable buffering into disk before ingesting into Azure Kusto. | `false` |
| `client_id` | Required if `managed_identity_client_id` isn't set. The client ID of the AAD registered application. | _none_ |
| `client_secret` | Set the client secret (Application Password) of the AAD application used for authentication. | _none_ |
| `compression_enabled` | If enabled, sends compressed HTTP payload (gzip) to Kusto. | `true` |
| `database_name` | The database name. | _none_ |
| `delete_on_max_upload_error` | When buffering is `true`, whether to delete the buffer file on maximum upload errors. | `false` |
| `host` | IP address or hostname of the target HTTP server. | `127.0.0.1` |
| `io_timeout` | Configure the HTTP IO timeout for uploads. | `60s` |
| `include_tag_key` | If enabled, a tag is appended to output. The key name is used `tag_key` property. | `true` |
| `include_time_key` | If enabled, a timestamp is appended to output. The key name is used `time_key` property. | `true` |
| `ingestion_endpoint` | The cluster's ingestion endpoint, usually in the form `https://ingest-cluster_name.region.kusto.windows.net`. | _none_ |
| `ingestion_endpoint_connect_timeout` | The connection timeout of various Kusto endpoints in seconds. | `60` |
| `ingestion_mapping_reference` | The name of a [JSON ingestion mapping](https://docs.microsoft.com/en-us/azure/data-explorer/kusto/management/mappings#json-mapping) that will be used to map the ingested payload into the table columns. | _none_ |
| `ingestion_resources_refresh_interval` | Set the Azure Kusto ingestion resources refresh interval. | `3600` |
| `io_timeout` | Configure the HTTP IO timeout for uploads. | `60s` |
| `log_key` | Key name of the log content. | `log` |
| `log_level` | Specifies the log level for output plugin. If not set here, plugin uses global log level in `service` section. | `info` |
| `log_supress_interval` | Suppresses log messages from output plugin that appear similar within a specified time interval. `0` no suppression. | `0` |
Expand All @@ -85,36 +84,35 @@ By default, Kusto will insert incoming ingestion data into a table by inferring
| `net.io_timeout` | Set maximum time a connection can stay idle while assigned. | `0s` |
| `net.keepalive` | Enable or disable `Keepalive` support. | `false` |
| `net.keepalive_idle_timeout` | Set maximum time allowed for an idle `Keepalive` connection.. | `false` |
| `net.keepalive_max_recycle` | Set maximum number of times a keepalive connection can be used before it's retried. | `2000` |
| `net.max_worker_connections` | Set the maximum number of active TCP connections that can be used per worker thread. | `0` |
| `net.proxy_env_ignore` | Ignore the environment variables `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` when set. | `false` |
| `net.source_address` | Specify network address to bind for data traffic. | _none_ |
| `net.tcp_keepalive` | Enable or disable Keepalive support. | `off` |
| `net.tcp_keepalive_interval` | Interval between TCP keepalive probes when no response is received on a `keepidle` probe. | `-1` |
| `net.keepalive_max_recycle` | Set maximum number of times a keepalive connection can be used before it's retried. | `2000` |
| `net.tcp_keepalive_probes` | Number of unacknowledged probes to consider a connection dead. | `-1` |
| `net.tcp_keepalive_time` | Interval between the last data packet sent and the first TCP keepalive probe.| `-1` |
| `net.max_worker_connections` | Set the maximum number of active TCP connections that can be used per worker thread. | `0` |
| `port` | TCP port of the target HTTP server. | `0` |
| `retry_limit` | Set retry limit for output plugin when delivery fails. Integer, `no_limits`, `false`, or `off` to disable, or `no_retries` to disable retries entirely. | `1` |
| `scheduler_max_retries` | Optional. When buffering is `On`, set the maximum number of retries for ingestion using the scheduler. | `3` |
| `store_dir_limit_size` | When buffering is `true`, set the max size of the buffer directory. | `8G` |
| `tag_key` | The key name of tag. If `include_tag_key` is false, This property is ignored.| `tag` |
| `table_name` | The table name. | _none_ |
| `tag_key` | The key name of tag. If `include_tag_key` is false, this property is ignored.| `tag` |
| `tenant_id` | Required if `managed_identity_client_id` isn't set. The tenant/domain ID of the AAD registered application. | _none_ |
| `time_key` | The key name of time. If `include_time_key` is false, this property is ignored. | `timestamp` |
| `tls` | Enable or disable TLS/SSL support. | `off` |
| `tls.ca_file` | Absolute path to CA certificate file. | _none_ |
| `tls.ca_path` | Absolute path to scan for certificate files. | _none_ |
| `tls.crt_file` | Absolute path to Certificate file. | _none_ |
| `tls.ciphers` | Specify TLS ciphers up to TLSv1.2. | _none_ |
| `tls.crt_file` | Absolute path to Certificate file. | _none_ |
| `tls.debug` | Set TLS debug level. Accepts `0` (No debug), `1`(Error), `2` (State change), `3` (Informational) and `4` (Verbose). | `1` |
| `tls.key_file` | Absolute path to private Key file. | _none_ |
| `tls.key_passwd` | Optional password for tls.key_file file. | _none_ |
| `tls.max_version` | Specify the maximum version of TLS. | _none_ |
| `tls.min_version` | Specify the minimum version of TLS. | _none_ |
| `tls.verify` | Force certificate validation. | `on` |
| `tls.verify_hostname` | Enable or disable to verify hostname. | `off` |
| `tls.vhost` | Hostname to be used for TLS SNI extension. | _none_ |
| `tls.verify` | Force certificate validation. | `on` |
| `tls.windows.certstore_name` | Sets the `certstore` name on an output (Windows). | _none_ |
| `tls.windows.use_enterprise_store` | Sets whether using enterprise certificate store or not on an output (Windows). | _none_ |
| `unify_tag` | This creates a single buffer file when the buffering mode is `true`. | `true` |
Expand Down Expand Up @@ -182,21 +180,21 @@ pipeline:
Database_Name <database_name>
Table_Name <table_name>
Ingestion_Mapping_Reference <mapping_name>
ingestion_endpoint_connect_timeout <ingestion_endpoint_connect_timeout>
compression_enabled <compression_enabled>
ingestion_resources_refresh_interval <ingestion_resources_refresh_interval>
buffering_enabled On
upload_timeout 2m
upload_file_size 125M
azure_kusto_buffer_key kusto1
buffer_file_delete_early Off
unify_tag On
buffer_dir /var/log/
store_dir_limit_size 16GB
blob_uri_length 128
scheduler_max_retries 3
delete_on_max_upload_error Off
io_timeout 60s
Ingestion_Endpoint_Connect_Timeout <ingestion_endpoint_connect_timeout>
Compression_Enabled <compression_enabled>
Ingestion_Resources_Refresh_Interval <ingestion_resources_refresh_interval>
Buffering_Enabled On
Upload_Timeout 2m
Upload_File_Size 125M
Azure_Kusto_Buffer_Key kusto1
Buffer_File_Delete_Early Off
Unify_Tag On
Buffer_Dir /var/log/
Store_Dir_Limit_Size 16GB
Blob_Uri_Length 128
Scheduler_Max_Retries 3
Delete_On_Max_Upload_Error Off
Io_Timeout 60s
```

{% endtab %}
Expand Down
Loading