Skip to main content

Logging

in Deskpro container image
Authors list
Zverejnené: 31. 7. 2023|Posledná Aktualizácia: 25. 8. 2023

The container ships with vector, a light-weight log collector. Vector is used to collect and normalize logs from all of the services and tasks in the container so they can be managed in a similar way.

Logging Output MethodsCopy link to Logging Output Methods to clipboard

Docker LogsCopy link to Docker Logs to clipboard

The default behaviour is to send all logs to stdout which can then be consumed like normal docker logs.

The exception is when running with exec or bash run modes (which are usually only used for debugging or one-off commands), the default is to write logs to disk inside the container at /var/log/docker-logs so as to keep your stdout clean. If you want to export logs then you can override this default behaviour by mounting a log directory as described below.

Mounted Logs DirectoryCopy link to Mounted Logs Directory to clipboard

If you mount /deskpro/logs, then logs are written to disk in this directory with files named $app/$app.log or $app/$app.$chan.log.

Warning

Logs in this directory can get very large. If you mount this directory, you will want to make sure that the files are rotated regularly. Log rotation on exported logs is NOT performed by the container; this is something you would have to configure on your host.

If you need to change the output directory for whatever reason, you can specify LOGS_EXPORT_DIR=/some/other/dir to change the output path.

File Permissions

The logs that get written to disk are the normalized logs collected by vector, not the literal log files being written by each service. That means you don't need to worry about file permissions or ownership. The logs written by vector will be written by the root user.

If you want logs to be owned by a particular group, then you can make your host logs directory owned by that group and set the sticky bit:

# e.g. on the host mkdir container-logs chown root:mygroup container-logs # set the sticky bit so any files written to this directory will be owned by mygroup chmod g+s container-logs # then run the container with that directory mounted
copy

Logging FormatCopy link to Logging Format to clipboard

Logs are normalized into the simple logfmt format with these common properties:

  • ts is a UTC timestamp in ISO 8601 format

  • app contains the application or service or source of the log line

  • chan is a particular "channel" (one app may have many different types of logs; we separate them into so-called channels)

  • lvl contains the message severity level (e.g. INFO or ERROR)

  • msg contains the log message

  • Many logs define other properties such as context information. Scalar values are printed as-is; more complex values may be encoded as JSON strings.

With logfmt, each log entry is a single line. For example:

ts=2023-06-27T16:04:12Z app=entrypoint lvl=DEBUG msg="Cleaning env vars" ts=2023-06-27T16:04:12Z app=entrypoint lvl=INFO msg="Command: web, Mode: web, Exec: supervisord, Services: nginx php_fpm" ... ts=2023-06-27T16:04:15Z app=supervisord lvl=INFO msg="success: php_fpm entered RUNNING state, process has stayed up for > than 2 seconds (startsecs)" ts=2023-06-27T16:04:16Z app=supervisord lvl=INFO msg="success: nginx entered RUNNING state, process has stayed up for > than 3 seconds (startsecs)" ... ts=2023-06-27T16:04:25Z app=nginx chan=error lvl=INFO msg="client 127.0.0.1 closed keepalive connection" cid=1 pid=192 tid=192 ts=2023-06-27T16:04:29Z app=nginx chan=access lvl=INFO method=GET path=/ status=200 host=localhost scheme=http proto=HTTP/1.1 body_bytes_sent=0 remote_addr=127.0.0.1 request_id=6ddd26905fdaa89f8cde604ba32c305c request_length=78 request_time=0.000 user_agent=curl/8.1.2 ... ts=2023-06-27T16:06:46Z app=php chan=error lvl=WARNING msg="PHP Warning: Undefined variable $foo in Command line code on line 1"
copy
Note

In some cases, log lines can get very very long (e.g. an application error with a complicated backtrace). You can use a utility like lnav to make reading and filtering output easier.

Here are the most common logs:

  • app=entrypoint -- Emitted during boot (docker entrypoint)

  • app=supervisord -- supervisord logs (logs to do with starting and stopping services) and the raw stdout of processes

  • app=nginx -- Logs for the internal HTTP server

    • chan=error -- error logs

    • chan=access -- access logs

  • app=php

    • chan=error -- PHP error logs

  • app=php_fpm

    • chan=error -- php-fpm error log

  • app=deskpro and app=deskpro-dpv5 -- Deskpro application logs. There are a number of channels but notably:

    • chan=general -- General app logs

    • chan=migrations -- Migration logs

    • chan=errors -- Deskpro error log (some overlap with the PHP error log)

  • app=deskpro-tasks -- Output from running Deskpro's various tasks. These can be very noisy and should be rotated frequently.

JSON OutputCopy link to JSON Output to clipboard

If you want to output as JSON instead of logfmt, set LOGS_OUTPUT_FORMAT=json

When outputting to JSON, all the same properties are the same, the only difference is that the line will be JSON encoded instead of logfmt.

Log LevelsCopy link to Log Levels to clipboard

You can change logging levels by setting env vars:

  • DESKPRO_LOG_LEVEL: The default log level for most application logs. Defaults to warning

  • DESKPRO_LOG_LEVEL_EMAIL_COLLECTION: Log level for email collection. Defaults to warning

  • DESKPRO_LOG_LEVEL_EMAIL_PROCESSING: Log level for email processing. Defaults to warning

  • PHP_FPM_LOG_LEVEL -- The value for php-fpm's log_level. Default: warning

  • NGINX_ERROR_LOG_LEVEL -- The logging level used by the internal web server. Default: info

Boot Log

The boot log (app=entrypoint) is slightly special because these messages may only be readable from stderr in the event of a problem (i.e. log collection via vector might not exist if vector can't even be started).

These logs are always written to /var/log/docker-boot.log inside the container. If you have enabled log exporting to a mounted volume, then a log file by the same name will be available in that mounted directory as well.

There are two variables that determine the logging level used for output to stderr:

  • BOOT_LOG_LEVEL (default: INFO) -- This applies when you are running the container in a service mode (e.g. NOT exec or bash). It determines the min log level for messages before they get sent to stderr.

  • BOOT_LOG_LEVEL_EXEC (default: WARNING) Is the same, except it applies to "exec" or "bash" run modes. In these modes, you are often running commands manually so you don't want to see noisy verbose INFO messages that happen during container start.

Advanced: Custom Log ShippingCopy link to Advanced: Custom Log Shipping to clipboard

You can provide your own vector configuration by mounting a custom config file to /deskpro/config/vector.d/99-my-output.toml

There are two useful inputs you can use:

  • all will be all log files with a normalized .logprops structure that includes standard log properties (e.g. app, chan, lvl, msg, etc.) as well as any 'extra' properties that may be defined by the log file. You can use this as the basis of any other custom transforms or filters.

  • all_formatted will define a .message property that is a pre-formatted string (either logfmt or JSON depending on the LOGS_OUTPUT_FORMAT env var). E.g. you can use this with sinks that have encoding.codec=raw_message support.

We suggest you avoid using any other inputs (such as specific log files) as they may change in future versions. Instead, if you wish to handle specific kinds of logs, use the all input and then filter it by .app, .chan, or .lvl to get just the logs you want.

PomocnéNeužitočná
next pageDebugging / Troubleshooting
previous pageRuntime Configuration

Please log in or register to submit a comment.