@@ -5,143 +5,115 @@ Dynamic debug
5
5
Introduction
6
6
============
7
7
8
- This document describes how to use the dynamic debug (dyndbg) feature.
8
+ Dynamic debug allows you to dynamically enable/disable kernel
9
+ debug-print code to obtain additional kernel information.
9
10
10
- Dynamic debug is designed to allow you to dynamically enable/disable
11
- kernel code to obtain additional kernel information. Currently, if
12
- ``CONFIG_DYNAMIC_DEBUG `` is set, then all ``pr_debug() ``/``dev_dbg() `` and
13
- ``print_hex_dump_debug() ``/``print_hex_dump_bytes() `` calls can be dynamically
14
- enabled per-callsite.
11
+ If ``/proc/dynamic_debug/control `` exists, your kernel has dynamic
12
+ debug. You'll need root access (sudo su) to use this.
15
13
16
- If you do not want to enable dynamic debug globally (i.e. in some embedded
17
- system), you may set ``CONFIG_DYNAMIC_DEBUG_CORE `` as basic support of dynamic
18
- debug and add ``ccflags := -DDYNAMIC_DEBUG_MODULE `` into the Makefile of any
19
- modules which you'd like to dynamically debug later.
20
-
21
- If ``CONFIG_DYNAMIC_DEBUG `` is not set, ``print_hex_dump_debug() `` is just
22
- shortcut for ``print_hex_dump(KERN_DEBUG) ``.
23
-
24
- For ``print_hex_dump_debug() ``/``print_hex_dump_bytes() ``, format string is
25
- its ``prefix_str `` argument, if it is constant string; or ``hexdump ``
26
- in case ``prefix_str `` is built dynamically.
14
+ Dynamic debug provides:
27
15
28
- Dynamic debug has even more useful features:
16
+ * a Catalog of all *prdbgs * in your kernel.
17
+ ``cat /proc/dynamic_debug/control `` to see them.
29
18
30
- * Simple query language allows turning on and off debugging
31
- statements by matching any combination of 0 or 1 of:
19
+ * a Simple query/command language to alter * prdbgs * by selecting on
20
+ any combination of 0 or 1 of:
32
21
33
22
- source filename
34
23
- function name
35
24
- line number (including ranges of line numbers)
36
25
- module name
37
26
- format string
38
-
39
- * Provides a debugfs control file: ``<debugfs>/dynamic_debug/control ``
40
- which can be read to display the complete list of known debug
41
- statements, to help guide you
42
-
43
- Controlling dynamic debug Behaviour
44
- ===================================
45
-
46
- The behaviour of ``pr_debug() ``/``dev_dbg() `` are controlled via writing to a
47
- control file in the 'debugfs' filesystem. Thus, you must first mount
48
- the debugfs filesystem, in order to make use of this feature.
49
- Subsequently, we refer to the control file as:
50
- ``<debugfs>/dynamic_debug/control ``. For example, if you want to enable
51
- printing from source file ``svcsock.c ``, line 1603 you simply do::
52
-
53
- nullarbor:~ # echo 'file svcsock.c line 1603 +p' >
54
- <debugfs>/dynamic_debug/control
55
-
56
- If you make a mistake with the syntax, the write will fail thus::
57
-
58
- nullarbor:~ # echo 'file svcsock.c wtf 1 +p' >
59
- <debugfs>/dynamic_debug/control
60
- -bash: echo: write error: Invalid argument
61
-
62
- Note, for systems without 'debugfs' enabled, the control file can be
63
- found in ``/proc/dynamic_debug/control ``.
27
+ - class name (as known/declared by each module)
64
28
65
29
Viewing Dynamic Debug Behaviour
66
30
===============================
67
31
68
- You can view the currently configured behaviour of all the debug
69
- statements via::
32
+ You can view the currently configured behaviour in the *prdbg * catalog::
70
33
71
- nullarbor:~ # cat <debugfs> /dynamic_debug/control
34
+ :#> head -n7 /proc /dynamic_debug/control
72
35
# filename:lineno [module]function flags format
73
- net/sunrpc/svc_rdma.c:323 [svcxprt_rdma]svc_rdma_cleanup =_ "SVCRDMA Module Removed, deregister RPC RDMA transport\012"
74
- net/sunrpc/svc_rdma.c:341 [svcxprt_rdma]svc_rdma_init =_ "\011max_inline : %d\012"
75
- net/sunrpc/svc_rdma.c:340 [svcxprt_rdma]svc_rdma_init =_ "\011sq_depth : %d\012"
76
- net/sunrpc/svc_rdma.c:338 [svcxprt_rdma]svc_rdma_init =_ "\011max_requests : %d\012"
77
- ...
36
+ init/main.c:1179 [main]initcall_blacklist =_ "blacklisting initcall %s\012
37
+ init/main.c:1218 [main]initcall_blacklisted =_ "initcall %s blacklisted\012"
38
+ init/main.c:1424 [main]run_init_process =_ " with arguments:\012"
39
+ init/main.c:1426 [main]run_init_process =_ " %s\012"
40
+ init/main.c:1427 [main]run_init_process =_ " with environment:\012"
41
+ init/main.c:1429 [main]run_init_process =_ " %s\012"
78
42
43
+ The 3rd space-delimited column shows the current flags, preceded by
44
+ a ``= `` for easy use with grep/cut. ``=p `` shows enabled callsites.
79
45
80
- You can also apply standard Unix text manipulation filters to this
81
- data, e.g.::
46
+ Controlling dynamic debug Behaviour
47
+ ===================================
82
48
83
- nullarbor:~ # grep -i rdma <debugfs>/dynamic_debug/control | wc -l
84
- 62
49
+ The behaviour of * prdbg * sites are controlled by writing
50
+ query/commands to the control file. Example::
85
51
86
- nullarbor:~ # grep -i tcp <debugfs>/dynamic_debug/control | wc -l
87
- 42
52
+ # grease the interface
53
+ :#> alias ddcmd='echo $* > /proc/dynamic_debug/control'
88
54
89
- The third column shows the currently enabled flags for each debug
90
- statement callsite (see below for definitions of the flags). The
91
- default value, with no flags enabled, is ``=_ ``. So you can view all
92
- the debug statement callsites with any non-default flags::
55
+ :#> ddcmd '-p; module main func run* +p'
56
+ :#> grep =p /proc/dynamic_debug/control
57
+ init/main.c:1424 [main]run_init_process =p " with arguments:\012"
58
+ init/main.c:1426 [main]run_init_process =p " %s\012"
59
+ init/main.c:1427 [main]run_init_process =p " with environment:\012"
60
+ init/main.c:1429 [main]run_init_process =p " %s\012"
93
61
94
- nullarbor:~ # awk '$3 != "=_"' <debugfs>/dynamic_debug/control
95
- # filename:lineno [module]function flags format
96
- net/sunrpc/svcsock.c:1603 [sunrpc]svc_send p "svc_process: st_sendto returned %d\012"
62
+ Error messages go to console/syslog::
63
+
64
+ :#> ddcmd mode foo +p
65
+ dyndbg: unknown keyword "mode"
66
+ dyndbg: query parse failed
67
+ bash: echo: write error: Invalid argument
68
+
69
+ If debugfs is also enabled and mounted, ``dynamic_debug/control `` is
70
+ also under the mount-dir, typically ``/sys/kernel/debug/ ``.
97
71
98
72
Command Language Reference
99
73
==========================
100
74
101
- At the lexical level, a command comprises a sequence of words separated
75
+ At the basic lexical level, a command is a sequence of words separated
102
76
by spaces or tabs. So these are all equivalent::
103
77
104
- nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' >
105
- <debugfs>/dynamic_debug/control
106
- nullarbor:~ # echo -n ' file svcsock.c line 1603 +p ' >
107
- <debugfs>/dynamic_debug/control
108
- nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' >
109
- <debugfs>/dynamic_debug/control
78
+ :#> ddcmd file svcsock.c line 1603 +p
79
+ :#> ddcmd "file svcsock.c line 1603 +p"
80
+ :#> ddcmd ' file svcsock.c line 1603 +p '
110
81
111
82
Command submissions are bounded by a write() system call.
112
83
Multiple commands can be written together, separated by ``; `` or ``\n ``::
113
84
114
- ~# echo "func pnpacpi_get_resources +p; func pnp_assign_mem +p" \
115
- > <debugfs>/dynamic_debug/control
85
+ :#> ddcmd "func pnpacpi_get_resources +p; func pnp_assign_mem +p"
86
+ :#> ddcmd <<"EOC"
87
+ func pnpacpi_get_resources +p
88
+ func pnp_assign_mem +p
89
+ EOC
90
+ :#> cat query-batch-file > /proc/dynamic_debug/control
116
91
117
- If your query set is big, you can batch them too::
92
+ You can also use wildcards in each query term. The match rule supports
93
+ ``* `` (matches zero or more characters) and ``? `` (matches exactly one
94
+ character). For example, you can match all usb drivers::
118
95
119
- ~# cat query-batch- file > <debugfs>/dynamic_debug/control
96
+ :#> ddcmd file "drivers/usb/*" +p # "" to suppress shell expansion
120
97
121
- Another way is to use wildcards. The match rule supports ``* `` (matches
122
- zero or more characters) and ``? `` (matches exactly one character). For
123
- example, you can match all usb drivers::
124
-
125
- ~# echo "file drivers/usb/* +p" > <debugfs>/dynamic_debug/control
126
-
127
- At the syntactical level, a command comprises a sequence of match
128
- specifications, followed by a flags change specification::
98
+ Syntactically, a command is pairs of keyword values, followed by a
99
+ flags change or setting::
129
100
130
101
command ::= match-spec* flags-spec
131
102
132
- The match-spec's are used to choose a subset of the known pr_debug()
133
- callsites to which to apply the flags-spec. Think of them as a query
134
- with implicit ANDs between each pair. Note that an empty list of
135
- match-specs will select all debug statement callsites.
103
+ The match-spec's select * prdbgs * from the catalog, upon which to apply
104
+ the flags-spec, all constraints are ANDed together. An absent keyword
105
+ is the same as keyword "*".
106
+
136
107
137
- A match specification comprises a keyword, which controls the
138
- attribute of the callsite to be compared, and a value to compare
139
- against. Possible keywords are:::
108
+ A match specification is a keyword, which selects the attribute of
109
+ the callsite to be compared, and a value to compare against. Possible
110
+ keywords are:::
140
111
141
112
match-spec ::= 'func' string |
142
113
'file' string |
143
114
'module' string |
144
115
'format' string |
116
+ 'class' string |
145
117
'line' line-range
146
118
147
119
line-range ::= lineno |
@@ -203,6 +175,16 @@ format
203
175
format "nfsd: SETATTR" // a neater way to match a format with whitespace
204
176
format 'nfsd: SETATTR' // yet another way to match a format with whitespace
205
177
178
+ class
179
+ The given class_name is validated against each module, which may
180
+ have declared a list of known class_names. If the class_name is
181
+ found for a module, callsite & class matching and adjustment
182
+ proceeds. Examples::
183
+
184
+ class DRM_UT_KMS # a DRM.debug category
185
+ class JUNK # silent non-match
186
+ // class TLD_* # NOTICE: no wildcard in class names
187
+
206
188
line
207
189
The given line number or range of line numbers is compared
208
190
against the line number of each ``pr_debug() `` callsite. A single
@@ -228,17 +210,16 @@ of the characters::
228
210
The flags are::
229
211
230
212
p enables the pr_debug() callsite.
231
- f Include the function name in the printed message
232
- l Include line number in the printed message
233
- m Include module name in the printed message
234
- t Include thread ID in messages not generated from interrupt context
235
- _ No flags are set. (Or'd with others on input)
213
+ _ enables no flags.
236
214
237
- For ``print_hex_dump_debug() `` and ``print_hex_dump_bytes() ``, only ``p `` flag
238
- have meaning, other flags ignored.
215
+ Decorator flags add to the message-prefix, in order:
216
+ t Include thread ID, or <intr>
217
+ m Include module name
218
+ f Include the function name
219
+ l Include line number
239
220
240
- For display, the flags are preceded by `` = ``
241
- (mnemonic: what the flags are currently equal to) .
221
+ For `` print_hex_dump_debug() `` and `` print_hex_dump_bytes() ``, only
222
+ the `` p `` flag has meaning, other flags are ignored .
242
223
243
224
Note the regexp ``^[-+=][flmpt_]+$ `` matches a flags specification.
244
225
To clear all flags at once, use ``=_ `` or ``-flmpt ``.
@@ -313,45 +294,39 @@ For ``CONFIG_DYNAMIC_DEBUG`` kernels, any settings given at boot-time (or
313
294
enabled by ``-DDEBUG `` flag during compilation) can be disabled later via
314
295
the debugfs interface if the debug messages are no longer needed::
315
296
316
- echo "module module_name -p" > <debugfs> /dynamic_debug/control
297
+ echo "module module_name -p" > /proc /dynamic_debug/control
317
298
318
299
Examples
319
300
========
320
301
321
302
::
322
303
323
304
// enable the message at line 1603 of file svcsock.c
324
- nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' >
325
- <debugfs>/dynamic_debug/control
305
+ :#> ddcmd 'file svcsock.c line 1603 +p'
326
306
327
307
// enable all the messages in file svcsock.c
328
- nullarbor:~ # echo -n 'file svcsock.c +p' >
329
- <debugfs>/dynamic_debug/control
308
+ :#> ddcmd 'file svcsock.c +p'
330
309
331
310
// enable all the messages in the NFS server module
332
- nullarbor:~ # echo -n 'module nfsd +p' >
333
- <debugfs>/dynamic_debug/control
311
+ :#> ddcmd 'module nfsd +p'
334
312
335
313
// enable all 12 messages in the function svc_process()
336
- nullarbor:~ # echo -n 'func svc_process +p' >
337
- <debugfs>/dynamic_debug/control
314
+ :#> ddcmd 'func svc_process +p'
338
315
339
316
// disable all 12 messages in the function svc_process()
340
- nullarbor:~ # echo -n 'func svc_process -p' >
341
- <debugfs>/dynamic_debug/control
317
+ :#> ddcmd 'func svc_process -p'
342
318
343
319
// enable messages for NFS calls READ, READLINK, READDIR and READDIR+.
344
- nullarbor:~ # echo -n 'format "nfsd: READ" +p' >
345
- <debugfs>/dynamic_debug/control
320
+ :#> ddcmd 'format "nfsd: READ" +p'
346
321
347
322
// enable messages in files of which the paths include string "usb"
348
- nullarbor:~ # echo -n 'file *usb* +p' > <debugfs> /dynamic_debug/control
323
+ :#> ddcmd 'file *usb* +p' > /proc /dynamic_debug/control
349
324
350
325
// enable all messages
351
- nullarbor:~ # echo -n '+p' > <debugfs> /dynamic_debug/control
326
+ :#> ddcmd '+p' > /proc /dynamic_debug/control
352
327
353
328
// add module, function to all enabled messages
354
- nullarbor:~ # echo -n '+mf' > <debugfs> /dynamic_debug/control
329
+ :#> ddcmd '+mf' > /proc /dynamic_debug/control
355
330
356
331
// boot-args example, with newlines and comments for readability
357
332
Kernel command line: ...
@@ -364,3 +339,38 @@ Examples
364
339
dyndbg="file init/* +p #cmt ; func parse_one +p"
365
340
// enable pr_debugs in 2 functions in a module loaded later
366
341
pc87360.dyndbg="func pc87360_init_device +p; func pc87360_find +p"
342
+
343
+ Kernel Configuration
344
+ ====================
345
+
346
+ Dynamic Debug is enabled via kernel config items::
347
+
348
+ CONFIG_DYNAMIC_DEBUG=y # build catalog, enables CORE
349
+ CONFIG_DYNAMIC_DEBUG_CORE=y # enable mechanics only, skip catalog
350
+
351
+ If you do not want to enable dynamic debug globally (i.e. in some embedded
352
+ system), you may set ``CONFIG_DYNAMIC_DEBUG_CORE `` as basic support of dynamic
353
+ debug and add ``ccflags := -DDYNAMIC_DEBUG_MODULE `` into the Makefile of any
354
+ modules which you'd like to dynamically debug later.
355
+
356
+
357
+ Kernel *prdbg * API
358
+ ==================
359
+
360
+ The following functions are cataloged and controllable when dynamic
361
+ debug is enabled::
362
+
363
+ pr_debug()
364
+ dev_dbg()
365
+ print_hex_dump_debug()
366
+ print_hex_dump_bytes()
367
+
368
+ Otherwise, they are off by default; ``ccflags += -DDEBUG `` or
369
+ ``#define DEBUG `` in a source file will enable them appropriately.
370
+
371
+ If ``CONFIG_DYNAMIC_DEBUG `` is not set, ``print_hex_dump_debug() `` is
372
+ just a shortcut for ``print_hex_dump(KERN_DEBUG) ``.
373
+
374
+ For ``print_hex_dump_debug() ``/``print_hex_dump_bytes() ``, format string is
375
+ its ``prefix_str `` argument, if it is constant string; or ``hexdump ``
376
+ in case ``prefix_str `` is built dynamically.
0 commit comments