# Decoders

There are cases where the log messages being parsed contain encoded data. A typical use case can be found in containerized environments with Docker. Docker logs its data in JSON format, which uses escaped strings.

Consider the following message generated by the application:

```
{"status": "up and running"}
```

The Docker log message encapsulates something like this:

```
{"log":"{\"status\": \"up and running\"}\r\n","stream":"stdout","time":"2018-03-09T01:01:44.851160855Z"}
```

The original message is handled as an escaped string. Fluent Bit wants to use the original structured message and not a string.

## Getting Started

Decoders are a built-in feature available through the Parsers file. Each parser definition can optionally set one or more decoders. There are two types of decoders:

* `Decode_Field`: If the content can be decoded in a structured message, append the structured message (keys and values) to the original log message.
* `Decode_Field_As`: Any decoded content (unstructured or structured) will be replaced in the same key/value, and no extra keys are added.

Our pre-defined Docker parser has the following definition:

```
[PARSER]
    Name         docker
    Format       json
    Time_Key     time
    Time_Format  %Y-%m-%dT%H:%M:%S.%L
    Time_Keep    On
    # Command       |  Decoder  | Field | Optional Action   |
    # ==============|===========|=======|===================|
    Decode_Field_As    escaped     log
```

Each line in the parser with a key `Decode_Field` instructs the parser to apply a specific decoder on a given field. Optionally, it offers the option to take an extra action if the decoder doesn't succeed.

### Decoder options

| Name           | Description                                                                                                    |
| -------------- | -------------------------------------------------------------------------------------------------------------- |
| `json`         | Handle the field content as a JSON map. If it finds a JSON map, it replaces the content with a structured map. |
| `escaped`      | Decode an escaped string.                                                                                      |
| `escaped_utf8` | Decode a UTF8 escaped string.                                                                                  |

### Optional Actions

If a decoder fails to decode the field or, you want to try another decoder, you can define an optional action. Available actions are:

| Name       | Description                                                                                |
| ---------- | ------------------------------------------------------------------------------------------ |
| `try_next` | if the decoder failed, apply the next decoder in the list for the same field.              |
| `do_next`  | if the decoder succeeded or failed, apply the next decoder in the list for the same field. |

Actions are affected by some restrictions:

* `Decode_Field_As`: If successful, another decoder of the same type and the same field can be applied only if the data continues being an unstructured message (raw text).
* `Decode_Field`: If successful, can only be applied once for the same field. `Decode`\_Field\` is intended to decode a structured message.

### Examples

#### `escaped_utf8`

Example input from `/path/to/log.log`:

```
{"log":"\u0009Checking indexes...\n","stream":"stdout","time":"2018-02-19T23:25:29.1845444Z"}
{"log":"\u0009\u0009Validated: _audit _internal _introspection _telemetry _thefishbucket history main snmp_data summary\n","stream":"stdout","time":"2018-02-19T23:25:29.1845536Z"}
{"log":"\u0009Done\n","stream":"stdout","time":"2018-02-19T23:25:29.1845622Z"}
```

Example output:

```
[24] tail.0: [1519082729.184544400, {"log"=>"   Checking indexes...
", "stream"=>"stdout", "time"=>"2018-02-19T23:25:29.1845444Z"}]
[25] tail.0: [1519082729.184553600, {"log"=>"           Validated: _audit _internal _introspection _telemetry _thefishbucket history main snmp_data summary
", "stream"=>"stdout", "time"=>"2018-02-19T23:25:29.1845536Z"}]
[26] tail.0: [1519082729.184562200, {"log"=>"   Done
", "stream"=>"stdout", "time"=>"2018-02-19T23:25:29.1845622Z"}]
```

Decoder configuration file:

```
[SERVICE]
    Parsers_File fluent-bit-parsers.conf

[INPUT]
    Name        tail
    Parser      docker
    Path        /path/to/log.log

[OUTPUT]
    Name   stdout
    Match  *
```

The `fluent-bit-parsers.conf` file:

```
[PARSER]
    Name        docker
    Format      json
    Time_Key    time
    Time_Format %Y-%m-%dT%H:%M:%S %z
    Decode_Field_as escaped_utf8 log
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.fluentbit.io/manual/3.2/pipeline/parsers/decoders.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.