Using Pyflame¶
Pyflame has two distinct modes: you can attach to a running process, or you can trace a command from start to finish.
Attaching To A Running Python Process¶
The default behavior of Pyflame is to attach to an existing Python process. The target process is specified via its PID:
# Profile PID for 1s, sampling every 1ms.
pyflame -p PID
This will print data to stdout in a format that is suitable for usage with
Brendan Gregg’s flamegraph.pl
tool (which you can get here). A typical command pipeline
might be like this:
# Generate flame graph for pid 12345; assumes flamegraph.pl is in your $PATH.
pyflame -p 12345 | flamegraph.pl > myprofile.svg
You can also change the sample time with -s
, and the sampling frequency with
-r
. Both units are measured in seconds.
# Profile PID for 60 seconds, sampling every 10ms.
pyflame -s 60 -r 0.01 -p PID
The default behavior is to sample for 1 second (equivalent to -s 1
), taking
a snapshot every millisecond (equivalent to -r 0.001
).
Attaching To Docker/Containerized Processes¶
Pyflame knows how to do something interesting: it can attach to containerized processes from outside the container. It does this by directly using the setns(2) system call (which is how Docker works under the hood).
If you choose to profile a process from outside the container, use the true PID,
as reported by ps
on the host (i.e. outside of the container).
You can also run Pyflame from inside containers, although this is a bit more
annoying, since normally ptrace is disabled inside containers for security
reasons. If you attach to a process this way, you will need to use the
inside-the-container PID. You can find this by running ps
inside of the
container itself.
We recommend running Pyflame from outside containers, since it means you can keep ptrace disabled inside containers. If you want to run Pyflame inside containers, and have problems, please make sure to read the Docker notes in the FAQ.
Tracing Python Commands¶
Sometimes you want to trace a command from start to finish. An example would be
tracing the run of a test suite or batch job. Pass -t
as the last
Pyflame flag to run in trace mode. Anything after the -t
flag is interpreted
literally as part of the command to run:
# Trace a given command until completion.
pyflame [regular pyflame options] -t command arg1 arg2...
Often command
will be python
or python3
, but it could be something
else, like uwsgi
or py.test
. For instance, here’s how Pyflame can be
used to trace its own test suite:
# Trace the Pyflame test suite, a.k.a. pyflameception!
pyflame -t py.test tests/
As described in the docs for attach mode, you can use -r
to control the
sampling frequency.
Tracing Programs That Print To Stdout¶
By default, Pyflame will send flame graph data to stdout. If the profiled
program is also sending data to stdout, then flamegraph.pl
will see the
output from both programs, and will get confused. To solve this, use the -o
option:
# Trace a process, sending profiling information to profile.txt
pyflame -o profile.txt -t python -c 'for x in range(1000): print(x)'
# Convert profile.txt to a flame graph named profile.svg
flamegraph.pl <profile.txt >profile.svg
Timestamp (“Flame Chart”) Mode¶
Generally we recommend using regular flame graphs, generated by
flamegraph.pl
. However, Pyflame can also generate data with a special time
stamp output format, useful for generating “flame charts” (somewhat like an
inverted flame graph) that are viewable in Chrome. In some cases, the flame
chart format is easier to understand.
To generate a flame chart, use pyflame --flamechart
, and then pass the
output to utils/flame-chart-json
to convert the output into the JSON format
required by the Chrome CPU profiler:
# Generate flame chart data viewable in Chrome.
pyflame --flamechart [other pyflame options] | flame-chart-json > foo.cpuprofile
Read the following Chrome DevTools article
for instructions on loading a .cpuprofile
file in Chrome 58+.