arrow-left

All pages
gitbookPowered by GitBook
1 of 1

Loading...

Azure Logs Ingestion API

Send logs to Azure Log Analytics using Logs Ingestion API with DCE and DCR

Azure Logs Ingestion plugin allows you ingest your records using Logs Ingestion API in Azure Monitorarrow-up-right to supported Azure tablesarrow-up-right or to custom tablesarrow-up-right that you create.

The Logs ingestion API requires the following components:

  • A Data Collection Endpoint (DCE)

  • A Data Collection Rule (DCR) and

  • A Log Analytics Workspace

Note: According to , all resources should be in the same region.

To visualize basic Logs Ingestion operation, see the following image:

To get more details about how to setup these components, please refer to the following documentations:

hashtag
Configuration Parameters

Key
Description
Default

hashtag
Getting Started

To send records into an Azure Log Analytics using Logs Ingestion API the following resources needs to be created:

  • A Data Collection Endpoint (DCE) for ingestion

  • A Data Collection Rule (DCR) for data transformation

  • Either an or

You can follow to setup the DCE, DCR, app registration and a custom table.

hashtag
Configuration File

Use this configuration to quickly get started:

Setup your DCR transformation accordingly based on the json output from fluent-bit's pipeline (input, parser, filter, output).

dcr_id

Required - Data Collection Rule (DCR) immutable ID (see to collect the immutable id)

table_name

Required - The name of the custom log table (include the _CL suffix as well if applicable)

time_key

Optional - Specify the key name where the timestamp will be stored.

@timestamp

time_generated

Optional - If enabled, will generate a timestamp and append it to JSON. The key name is set by the 'time_key' parameter.

true

compress

Optional - Enable HTTP payload gzip compression.

true

workers

The number of to perform flush operations for this output.

0

An app registration with client secrets (for DCR access).

tenant_id

Required - The tenant ID of the AAD application.

client_id

Required - The client ID of the AAD application.

client_secret

Required - The client secret of the AAD application (App Secretarrow-up-right).

dce_url

Required - Data Collection Endpoint(DCE) URL.

this documentarrow-up-right
Azure Logs Ingestion APIarrow-up-right
Send data to Azure Monitor Logs with Logs ingestion API (setup DCE, DCR and Log Analytics)arrow-up-right
Azure tablesarrow-up-right
custom tablesarrow-up-right
this guidelinearrow-up-right
[INPUT]
    Name    tail
    Path    /path/to/your/sample.log
    Tag     sample
    Key     RawData
# Or use other plugins Plugin
# [INPUT]
#     Name    cpu
#     Tag     sample

[FILTER]
    Name modify
    Match sample
    # Add a json key named "Application":"fb_log"
    Add Application fb_log

# Enable this section to see your json-log format
#[OUTPUT]
#    Name stdout
#    Match *
[OUTPUT]
    Name            azure_logs_ingestion
    Match           sample
    client_id       XXXXXXXX-xxxx-yyyy-zzzz-xxxxyyyyzzzzxyzz
    client_secret   some.secret.xxxzzz
    tenant_id       XXXXXXXX-xxxx-yyyy-zzzz-xxxxyyyyzzzzxyzz
    dce_url         https://log-analytics-dce-XXXX.region-code.ingest.monitor.azure.com
    dcr_id          dcr-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    table_name      ladcr_CL
    time_generated  true
    time_key        Time
    Compress        true
this documentarrow-up-right
workers