# Azure Logs Ingestion API

![](https://4005883584-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F1NBbcZmdFBruUw1adUIg%2Fuploads%2Fgit-blob-257937256c2f5950d0a098755cbc92a479827630%2Fimage%20\(7\).png?alt=media)

Azure Logs Ingestion plugin allows you ingest your records using [Logs Ingestion API in Azure Monitor](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/logs-ingestion-api-overview) to supported [Azure tables](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/logs-ingestion-api-overview#supported-tables) or to [custom tables](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/create-custom-table#create-a-custom-table) that you create.

The Logs ingestion API requires the following components:

* A Data Collection Endpoint (DCE)
* A Data Collection Rule (DCR) and
* A Log Analytics Workspace

> Note: According to [this document](https://github.com/MicrosoftDocs/azure-docs/blob/main/articles/azure-monitor/logs/logs-ingestion-api-overview.md#components), all resources should be in the same region.

To visualize basic Logs Ingestion operation, see the following image: ![](https://4005883584-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F1NBbcZmdFBruUw1adUIg%2Fuploads%2Fgit-blob-044a660ea944e10683679d7731dcd6c14c81f1ac%2Fazure-logs-ingestion-overview.png?alt=media)

To get more details about how to setup these components, please refer to the following documentations:

* [Azure Logs Ingestion API](https://docs.microsoft.com/en-us/azure/log-analytics/)
* [Send data to Azure Monitor Logs with Logs ingestion API (setup DCE, DCR and Log Analytics)](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/tutorial-logs-ingestion-portal)

## Configuration Parameters

| Key             | Description                                                                                                                                                                                                                        | Default      |
| --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ |
| tenant\_id      | *Required* - The tenant ID of the AAD application.                                                                                                                                                                                 |              |
| client\_id      | *Required* - The client ID of the AAD application.                                                                                                                                                                                 |              |
| client\_secret  | *Required* - The client secret of the AAD application ([App Secret](https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal#option-2-create-a-new-application-secret)).              |              |
| dce\_url        | *Required* - Data Collection Endpoint(DCE) URL.                                                                                                                                                                                    |              |
| dcr\_id         | *Required* - Data Collection Rule (DCR) immutable ID (see [this document](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/tutorial-logs-ingestion-portal#collect-information-from-the-dcr) to collect the immutable id) |              |
| table\_name     | *Required* - The name of the custom log table (include the `_CL` suffix as well if applicable)                                                                                                                                     |              |
| time\_key       | *Optional* - Specify the key name where the timestamp will be stored.                                                                                                                                                              | `@timestamp` |
| time\_generated | *Optional* - If enabled, will generate a timestamp and append it to JSON. The key name is set by the 'time\_key' parameter.                                                                                                        | `true`       |
| compress        | *Optional* - Enable HTTP payload gzip compression.                                                                                                                                                                                 | `true`       |

## Getting Started

To send records into an Azure Log Analytics using Logs Ingestion API the following resources needs to be created:

* A Data Collection Endpoint (DCE) for ingestion
* A Data Collection Rule (DCR) for data transformation
* Either an [Azure tables](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/logs-ingestion-api-overview#supported-tables) or [custom tables](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/create-custom-table#create-a-custom-table)
* An app registration with client secrets (for DCR access).

You can follow [this guideline](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/tutorial-logs-ingestion-portal) to setup the DCE, DCR, app registration and a custom table.

### Configuration File

Use this configuration to quickly get started:

```ini
[INPUT]
    Name    tail
    Path    /path/to/your/sample.log
    Tag     sample
    Key     RawData 
# Or use other plugins Plugin
# [INPUT]
#     Name    cpu
#     Tag     sample

[FILTER]
    Name modify
    Match sample
    # Add a json key named "Application":"fb_log"
    Add Application fb_log

# Enable this section to see your json-log format
#[OUTPUT]
#    Name stdout
#    Match *
[OUTPUT]
    Name            azure_logs_ingestion
    Match           sample
    client_id       XXXXXXXX-xxxx-yyyy-zzzz-xxxxyyyyzzzzxyzz
    client_secret   some.secret.xxxzzz
    tenant_id       XXXXXXXX-xxxx-yyyy-zzzz-xxxxyyyyzzzzxyzz
    dce_url         https://log-analytics-dce-XXXX.region-code.ingest.monitor.azure.com
    dcr_id          dcr-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    table_name      ladcr_CL
    time_generated  true
    time_key        Time
    Compress        true
```

Setup your DCR transformation accordingly based on the json output from fluent-bit's pipeline (input, parser, filter, output).