NGINX has wide range of debugging features, including detailed debug log.
Most debugging nits are only activated when NGINX compiled with –with-debug configure argument.
See a debugging log in documentation for details.
To activate debugging log you have to compile NGINX with –with-debug configure option and set debug level in error_log directive.
It’s possible to debug only connections from specified addresses via debug_connection directive.
In hard cases (e.g. debugging event method related problems) it’s good idea to obtain full debugging log by setting debug level in global error_log.
To obtain core dump you usually have to tune your OS. Though NGINX simplifies some typical cases and usually adding
worker_rlimit_core 500M; working_directory /path/to/cores/;
to nginx.conf is enough. Then run gdb to obtain backtrace as usual, e.g.
gdb /path/to/nginx /path/to/cores/nginx.core backtrace full
If your gdb backtrace warns that No symbol table info available. then you will need to recompile NGINX with the appropriate compiler flags for debugging symbols.
The exact flags required depend on the compiler used. If you use GCC, the flag
-g enables the inclusion of debugging symbols.
Additionally disabling compiler optimization using
-O0 will make the debugger output easier to understand.
CFLAGS="-g -O0" ./configure ....
Sometimes socket leaks happen.
This usually results in
[alert] 15248#0: open socket #123 left in connection 456 messages in error log on NGINX reload/restart/shutdown.
To debug add
nginx.conf and configure core dumps (see above).
This will result in
abort() call once NGINX detects leak and core dump.
Something like this in gdb should be usefull (assuming 456 is connection number from error message from the process which dumped core):
set $c = &ngx_cycle->connections p $c->log->connection p *$c set $r = (ngx_http_request_t *) $c->data p *$r
p $c->log->connection will print connection number as used in logs.
It will be possible to grep debug log for relevant lines, e.g.
fgrep ' *12345678 ' /path/to/error_log;
Asking for help¶
When asking for help with debugging please provide:
- full config
- debug log
- backtrace (if NGINX exits on signal)