> For the complete documentation index, see [llms.txt](https://docs.fluentbit.io/manual/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.fluentbit.io/manual/concepts/data-pipeline.md).

# Data pipeline

The Fluent Bit data pipeline incorporates several specific concepts. Data processing flows through the pipeline following these concepts in order.

```mermaid
graph LR
    accTitle: Fluent Bit data pipeline
    accDescr: A diagram of the Fluent Bit data pipeline, which includes input, processors, a parser, a filter, a buffer, routing, and various outputs.
    A[Input] --> B[Processors]
    B --> C[Parser]
    C --> D[Filter]
    D --> E[Buffer]
    E --> F((Routing))
    F --> G[Output 1]
    F --> H[Output 2]
    F --> I[Output 3]
```

## Inputs

[Input plugins](/manual/data-pipeline/inputs.md) gather information from different sources. Some plugins collect data from log files, and others gather metrics information from the operating system. There are many plugins to suit different needs.

## Processors

[Processors](/manual/data-pipeline/processors.md) are components that modify, transform, or enhance data as it flows through the pipeline. Processors are attached directly to individual input or output plugins rather than defined globally, and they don't use tag matching.

Because processors run in the same thread as their associated plugin, they can reduce performance overhead compared to filters—especially when [multithreading](/manual/administration/multithreading.md) is enabled.

Processors are configured in [YAML configuration files](/manual/administration/configuring-fluent-bit/yaml.md) only.

## Parser

[Parsers](/manual/data-pipeline/parsers.md) convert unstructured data to structured data. Use a parser to set a structure to the incoming data by using input plugins as data is collected.

## Filter

[Filters](/manual/data-pipeline/filters.md) let you alter the collected data before delivering it to a destination. In production environments you need full control of the data you're collecting. Using filters lets you control data before processing.

## Buffer

The [buffering](/manual/data-pipeline/buffering.md) phase in the pipeline aims to provide a unified and persistent mechanism to store your data, using the primary in-memory model or the file system-based mode.

## Routing

[Routing](/manual/data-pipeline/router.md) is a core feature that lets you route your data through filters, and then to one or multiple destinations. The router relies on the concept of [tags](/manual/concepts/key-concepts.md#tag) and [matching](/manual/concepts/key-concepts.md#match) rules.

## Output

[Output plugins](/manual/data-pipeline/outputs.md) let you define destinations for your data. Common destinations are remote services, local file systems, or other standard interfaces.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.fluentbit.io/manual/concepts/data-pipeline.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.