Integrating a log management platform with Dokploy

I spent too much time figuring out how to integrate a log management platform with Dokploy.

I was looking at 3 different platforms: NewRelic, OpenObserve, and Papertrail. Eventually, I've chosen NewRelic due to the combination of UX (haha) and ease of integration. OpenObserve seems good but doesn't support ANSI colors. I couldn't sign up with Papertrail successfully; it just failed. I was a big fan of Papertrail, so that was disappointing.

What have I tried and failed?

I wanted to use --log-driver=fluentd. With NewRelic, we can add nrLicenseKey=YOUR_LICENSE_KEY as a fluentd tag, and it will get processed into your account.

Unfortunately, Dokploy doesn't have a good way to support --log-driver. I would have to use Docker Compose, and I don't want to do that.

There is an open issue about this a few months ago: https://github.com/Dokploy/dokploy/issues/3132

So, I hope this will get resolved some day because this would have been a better altenative.

What works!

Installing fluentbit and NewRelic's plugin on the machine that reads from /var/docker/containers/*/*.log works out pretty well.

fluentbit is chosen because it's a light-weight, less-featureful, and more performant version of fluentd. They are both fully compatible with each other in terms of format and protocol.

Here's the configuration at /etc/fluent-bit/fluent-bit.conf:

[INPUT]
        Name             tail
        Path             /var/lib/docker/containers/*/*.log
        Path_Key         filepath
        Tag              docker_tail.*
        Parser           docker
        DB               /var/log/flb_docker.db
        Mem_Buf_Limit    5MB
        Skip_Empty_Lines On
        # Enable Docker mode for recombining split lines
        Docker_Mode      On

[SERVICE]
        Parsers_File parsers.conf
        Plugins_File plugins.conf

[FILTER]
        Name   record_modifier
        Match  *
        Record hostname ${HOSTNAME}

[OUTPUT]
        Name       newrelic
        Match      *
        licenseKey <YOUR_LICENSE_KEY>

Then, here's the /etc/fluent-bit/plugins.conf:

[PLUGINS]
        Path /home/ubuntu/out_newrelic-linux-amd64-3.4.0.so

You will have to download out_newrelic-linux-amd64-3.4.0.so from https://github.com/newrelic/newrelic-fluent-bit-output/releases

Finally, here's the /etc/fluent-bit/parsers.conf to parse the JSON logs from Docker:

[PARSER]
        Name   docker
        Format json
        Time_Key time
        Time_Format %Y-%m-%dT%H:%M:%S.%L
        # Decode the 'log' field as an escaped string
        Decode_Field_As   escaped   lo

This works out pretty well.

The gap

You'll notice that, in fluent-bit.conf, we have Path_Key. This will add the attribute filepath to every log entry. The issue is that the filepath doesn't indicate what service it is and is very long (because it contains a hash).

I'm still trying to figure out how to pipe the service name through the log. But this is probably a problem to be solved on a different day.