Azure Data Explorer
Send logs to Azure Data Explorer (Kusto)
The Kusto output plugin allows to ingest your logs into an Azure Data Explorer cluster, via the Queued Ingestion mechanism.
You can create an Azure Data Explorer cluster in one of the following ways:
Fluent-Bit will use the application's credentials, to ingest data into your cluster.
Fluent-Bit ingests the event data into Kusto in a JSON format, that by default will include 3 properties:
log- the actual event payload.tag- the event tag.timestamp- the event timestamp.
A table with the expected schema must exist in order for data to be ingested properly.
.create table FluentBit (log:dynamic, tag:string, timestamp:datetime)
By default, Kusto will insert incoming ingestions into a table by inferring the mapped table columns, from the payload properties. However, this mapping can be customized by creatng a JSON ingestion mapping. The plugin can be configured to use an ingestion mapping via the
ingestion_mapping_reference configuration key.Key | Description | Default |
|---|---|---|
tenant_id | Required - The tenant/domain ID of the AAD registered application. | |
client_id | Required - The client ID of the AAD registered application. | |
client_secret | | |
ingestion_endpoint | Required - The cluster's ingestion endpoint, usually in the form `https://ingest-cluster_name.region.kusto.windows.net | |
database_name | Required - The database name. | |
table_name | Required - The table name. | |
ingestion_mapping_reference | Optional - The name of a JSON ingestion mapping that will be used to map the ingested payload into the table columns. | |
log_key | Key name of the log content. | log |
include_tag_key | If enabled, a tag is appended to output. The key name is used tag_key property. | On |
tag_key | The key name of tag. If include_tag_key is false, This property is ignored. | tag |
include_time_key | If enabled, a timestamp is appended to output. The key name is used time_key property. | On |
time_key | The key name of time. If include_time_key is false, This property is ignored. | timestamp |
Get started quickly with this configuration file:
[OUTPUT]
Match *
Name azure_kusto
Tenant_Id <app_tenant_id>
Client_Id <app_client_id>
Client_Secret <app_secret>
Ingestion_Endpoint https://ingest-<cluster>.<region>.kusto.windows.net
Database_Name <database_name>
Table_Name <table_name>
Ingestion_Mapping_Reference <mapping_name>
If you get a
403 Forbidden error response, make sure that:- You provided the correct AAD registered application credentials.
- You authorized the application to ingest into your database or table.
Last modified 1d ago