# Content modifier

![](https://static.scarf.sh/a.png?x-pxid=ee1ad690-a3e9-434f-9635-3e53c670e96c)

The *content modifier* processor lets you manipulate the content, metadata, and attributes of logs and traces.

Similar to how filters work, this processor uses a unified mechanism to perform operations for data manipulation. The most significant difference is that processors perform better than filters, and when chaining them, there are no encoding/decoding performance penalties.

{% hint style="info" %}
Only [YAML configuration files](https://docs.fluentbit.io/manual/4.0/administration/configuring-fluent-bit/yaml) support processors.
{% endhint %}

## Contexts

The content modifier relies on *context*, meaning the place where the content modification will happen. Fluent Bit provides different contexts to manipulate the desired information.

The following contexts are available:

| Name              | Telemetry type | Description                                          |
| ----------------- | -------------- | ---------------------------------------------------- |
| `attributes`      | Logs           | Modifies the attributes or metadata of a log record. |
| `body`            | Logs           | Modifies the content of a log record.                |
| `span_name`       | Traces         | Modifies the name of a span.                         |
| `span_kind`       | Traces         | Modifies the kind of a span.                         |
| `span_status`     | Traces         | Modifies the status of a span.                       |
| `span_attributes` | Traces         | Modifies the attributes of a span.                   |

### OpenTelemetry contexts

Additionally, Fluent Bit provides specific contexts for modifying data that follows the OpenTelemetry log schema. All of these contexts operate on shared data across a group of records.

The following contexts are available:

| Name                       | Telemetry type | Description                                  |
| -------------------------- | -------------- | -------------------------------------------- |
| `otel_resource_attributes` | Logs           | Modifies the attributes of the log resource. |
| `otel_scope_name`          | Logs           | Modifies the name of a log scope.            |
| `otel_scope_version`       | Logs           | Modifies version of a log scope.             |
| `otel_scope_attributes`    | Logs           | Modifies the attributes of a log scope.      |

{% hint style="info" %}
If your data doesn't follow the OpenTelemetry log schema, but your log destination expects to be in that format, you can use the [OpenTelemetry envelope](https://docs.fluentbit.io/manual/4.0/data-pipeline/processors/opentelemetry-envelope) processor to transform your data. You can then pass that transformed data through the content modifier filter and use OpenTelemetry contexts accordingly.
{% endhint %}

## Configuration parameters

The following configuration parameters are available:

| Key              | Description                                                                                                |
| ---------------- | ---------------------------------------------------------------------------------------------------------- |
| `context`        | Specifies the [context](#contexts) where the modifications will happen.                                    |
| `key`            | Specifies the name of the key that will be used to apply the modification.                                 |
| `value`          | The role of this parameter changes based on the [action](#actions) type.                                   |
| `pattern`        | Defines a regular expression pattern. This property is only used by the `extract` [action](#actions).      |
| `converted_type` | Defines the data type to perform the conversion. Possible values: `string`, `boolean`, `int` and `double`. |

### Actions

The actions specify the type of operation to run on top of a specific key or content from a log or a trace. The following actions are available:

| Action    | Description                                                                                                                                                                                              |
| --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `insert`  | Inserts a new key with a value into the target context. The `key` and `value` parameters are required.                                                                                                   |
| `upsert`  | Given a specific key with a value, the `upsert` operation will try to update the value of the key. If the key doesn't exist, a new key will be created. The `key` and `value` parameters are required.   |
| `delete`  | Deletes a key from the target context. The `key` parameter is required.                                                                                                                                  |
| `rename`  | Changes the name of a key. The `value` set in the configuration will represent the new name. The `key` and `value` parameters are required.                                                              |
| `hash`    | Replaces the key value with a hash generated by the SHA-256 algorithm, the binary value generated is finally set as a hex string representation. The `key` parameter is required.                        |
| `extract` | Extracts the value of a single key as a list of key/value pairs. This action needs the configuration of a regular expression in the `pattern` property. The `key` and `pattern` parameters are required. |
| `convert` | Converts the data type of a key value. The `key` and `converted_type` parameters are required.                                                                                                           |

#### Insert example

The following example appends the key `color` with the value `blue` to the log stream.

{% tabs %}
{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:
  inputs:
    - name: dummy
      dummy: '{"key1": "123.4"}'

      processors:
        logs:
          - name: content_modifier
            action: insert
            key: "color"
            value: "blue"

  outputs:
    - name : stdout
      match: '*'
      format: json_lines
```

{% endtab %}
{% endtabs %}

#### Upsert example

Update the value of `key1` and insert `key2`:

{% tabs %}
{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:
  inputs:
    - name: dummy
      dummy: '{"key1": "123.4"}'

      processors:
        logs:
          - name: content_modifier
            action: upsert
            key: "key1"
            value: "5678"

          - name: content_modifier
            action: upsert
            key: "key2"
            value: "example"

  outputs:
    - name : stdout
      match: '*'
      format: json_lines
```

{% endtab %}
{% endtabs %}

#### Delete example

Delete `key2` from the stream:

{% tabs %}
{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:
  inputs:
    - name: dummy
      dummy: '{"key1": "123.4", "key2": "example"}'

      processors:
        logs:
          - name: content_modifier
            action: delete
            key: "key2"

  outputs:
    - name : stdout
      match: '*'
      format: json_lines
```

{% endtab %}
{% endtabs %}

#### Rename example

Change the name of `key2` to `test`:

{% tabs %}
{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:
  inputs:
    - name: dummy
      dummy: '{"key1": "123.4", "key2": "example"}'

      processors:
        logs:
          - name: content_modifier
            action: rename
            key: "key2"
            value: "test"

  outputs:
    - name : stdout
      match: '*'
      format: json_lines
```

{% endtab %}
{% endtabs %}

#### Hash example

Apply the SHA-256 algorithm for the value of the key `password`:

{% tabs %}
{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:
  inputs:
    - name: dummy
      dummy: '{"username": "bob", "password": "12345"}'

      processors:
        logs:
          - name: content_modifier
            action: hash
            key: "password"

  outputs:
    - name : stdout
      match: '*'
      format: json_lines
```

{% endtab %}
{% endtabs %}

#### Extract example

By using a domain address, perform an extraction of the components of it as a list of key value pairs:

{% tabs %}
{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:
  inputs:
    - name: dummy
      dummy: '{"http.url": "https://fluentbit.io/docs?q=example"}'

      processors:
        logs:
          - name: content_modifier
            action: extract
            key: "http.url"
            pattern: ^(?<http_protocol>https?):\/\/(?<http_domain>[^\/\?]+)(?<http_path>\/[^?]*)?(?:\?(?<http_query_params>.*))?

  outputs:
    - name : stdout
      match: '*'
      format: json_lines
```

{% endtab %}
{% endtabs %}

#### Convert example

Both keys in the example are strings. Convert the `key1` to a double/float type and `key2` to a boolean:

{% tabs %}
{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:
  inputs:
    - name: dummy
      dummy: '{"key1": "123.4", "key2": "true"}'

      processors:
        logs:
          - name: content_modifier
            action: convert
            key: key1
            converted_type: int

          - name: content_modifier
            action: convert
            key: key2
            converted_type: boolean

  outputs:
    - name : stdout
      match: '*'
      format: json_lines
```

{% endtab %}
{% endtabs %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.fluentbit.io/manual/4.0/data-pipeline/processors/content-modifier.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.