Histórico da Página

1. 1. 1. 1. 
            [SERVICE]
                flush        5
                daemon       Off
                log_level    info
                parsers_file parsers.conf
                plugins_file plugins.conf
                http_server  On
                http_listen  0.0.0.0
                http_port    2020
                storage.metrics on
                storage.path /tmp/storage
                storage.sync normal
                storage.checksum on
                storage.backlog.mem_limit 1G

            [INPUT]
                Name        tcp
                Listen      0.0.0.0
                Port        5140
                Tag         udp.logs
                Format      none
                Buffer_Size 256KB

            [FILTER]
                Name          lua
                Match         udp.logs
                script        

Para a utilização do Agente https://fluentbit.io/ recebendo dados da integração [LogClient] .

O fluxo abaixo exemplifica bem o processo:

Image Removed

Processo de Configuração

1. Instalar o TD-Agent( FluentBit) na maquina Linux que irá se comunicar com o servidor FluentBit;
   1. Instalação do TD-Agent →
      1. 1. # Importar a chave GPG do Fluent Bit
            sudo rpm --import https://packages.fluentbit.io/fluentbit.key
         2. # Criar o arquivo de repositório
            sudo tee /etc/yum.repos.d/fluentbit.repo <<EOF
            [fluentbit]
            name=Fluent Bit
            baseurl=https://packages.fluentbit.io/centos/8
            gpgcheck=1
            gpgkey=https://packages.fluentbit.io/fluentbit.key
            enabled=1
            EOF
         3. # Instalar o TD-Agent
            dnf install td-agent-bit
            # 

            editar o arquivo:
            /etc/td-agent-bit/td-agent-bit.conf

            E acrescentar essas regras:
            Bloco de código

            language	bash

            [SERVICE] # Flush # ===== # set an interval of seconds before to flush records to a destination flush 10 # Daemon # ====== # instruct Fluent Bit to run in foreground or background mode. daemon Off # Log_Level # ========= # Set the verbosity level of the service, values can be: # # - error # - warning # - info # - debug # - trace # # by default 'info' is set, that means it includes 'error' and 'warning'. log_level trace # Parsers File # ============ # specify an optional 'Parsers' configuration file parsers_file parsers.conf # Plugins File # ============ # specify an optional 'Plugins' configuration file to load external plugins. plugins_file plugins.conf # HTTP Server # =========== # Enable/Disable the built-in HTTP Server for metrics http_server Off http_listen 0.0.0.0 http_port 2020 # Storage # ======= # Fluent Bit can use memory and filesystem buffering based mechanisms # # - https://docs.fluentbit.io/manual/administration/buffering-and-storage # # storage metrics # --------------- # publish storage pipeline metrics in '/api/v1/storage'. The metrics are # exported only if the 'http_server' option is enabled. # storage.metrics on # storage.path # ------------ # absolute file system path to store filesystem data buffers (chunks). # storage.path /tmp/storage # storage.sync # ------------ # configure the synchronization mode used to store the data into the # filesystem. It can take the values normal or full. # storage.sync full # storage.checksum # ---------------- # enable the data integrity check when writing and reading data from the # filesystem. The storage layer uses the CRC32 algorithm. # # storage.checksum off # storage.backlog.mem_limit # ------------------------- # if storage.path is set, Fluent Bit will look for data chunks that were # not delivered and are still in the storage layer, these are called # backlog data. This option configure a hint of maximum value of memory # to use when processing these records. # storage.backlog.mem_limit 800M [INPUT] Name tcp Listen 0.0.0.0 Port 5140 Tag udp.logs Format none #[OUTPUT] # Name file # Match udp.logs # Path /var/log/fluent-bit-output.log # Format plain # Buffer_Chunk_Size 1M # Buffer_Max_Size 5M #[OUTPUT] # Name stdout # Match udp.logs [FILTER] Name lua Match udp.logs script

            /etc/td-agent-bit/processa_log.lua

            call parse_logmsg [OUTPUT] name http match udp.logs host 12.345.6.789 port 443 uri /ProtheusProduction.logs format json tls On tls.verify Off http_User userX http_Passwd passX Retry_Limit False net.connect_timeout 15s net.keepalive true

            
                call          parse_logmsg

            [OUTPUT]
                name        http
                match       udp.logs
                host        <url>
                port        443
                uri         /ProtheusProduction.logs
                format      json
                tls         On
                tls.verify  On
                http_User   <usuario> 
                http_Passwd <password>
                Retry_Limit 5
                net.connect_timeout 15s
                net.keepalive true
                net.keepalive_idle_timeout

            60s

            60s
                workers     4
                compress    gzip