diff options
Diffstat (limited to 'contrib/perl5/pod/perldebguts.pod')
| -rw-r--r-- | contrib/perl5/pod/perldebguts.pod | 923 |
1 files changed, 923 insertions, 0 deletions
diff --git a/contrib/perl5/pod/perldebguts.pod b/contrib/perl5/pod/perldebguts.pod new file mode 100644 index 0000000..b74f3ef --- /dev/null +++ b/contrib/perl5/pod/perldebguts.pod @@ -0,0 +1,923 @@ +=head1 NAME + +perldebguts - Guts of Perl debugging + +=head1 DESCRIPTION + +This is not the perldebug(1) manpage, which tells you how to use +the debugger. This manpage describes low-level details ranging +between difficult and impossible for anyone who isn't incredibly +intimate with Perl's guts to understand. Caveat lector. + +=head1 Debugger Internals + +Perl has special debugging hooks at compile-time and run-time used +to create debugging environments. These hooks are not to be confused +with the I<perl -Dxxx> command described in L<perlrun>, which are +usable only if a special Perl built per the instructions the +F<INSTALL> podpage in the Perl source tree. + +For example, whenever you call Perl's built-in C<caller> function +from the package DB, the arguments that the corresponding stack +frame was called with are copied to the the @DB::args array. The +general mechanisms is enabled by calling Perl with the B<-d> switch, the +following additional features are enabled (cf. L<perlvar/$^P>): + +=over + +=item * + +Perl inserts the contents of C<$ENV{PERL5DB}> (or C<BEGIN {require +'perl5db.pl'}> if not present) before the first line of your program. + +=item * + +The array C<@{"_<$filename"}> holds the lines of $filename for all +files compiled by Perl. The same for C<eval>ed strings that contain +subroutines, or which are currently being executed. The $filename +for C<eval>ed strings looks like C<(eval 34)>. Code assertions +in regexes look like C<(re_eval 19)>. + +=item * + +The hash C<%{"_<$filename"}> contains breakpoints and actions keyed +by line number. Individual entries (as opposed to the whole hash) +are settable. Perl only cares about Boolean true here, although +the values used by F<perl5db.pl> have the form +C<"$break_condition\0$action">. Values in this hash are magical +in numeric context: they are zeros if the line is not breakable. + +The same holds for evaluated strings that contain subroutines, or +which are currently being executed. The $filename for C<eval>ed strings +looks like C<(eval 34)> or C<(re_eval 19)>. + +=item * + +The scalar C<${"_<$filename"}> contains C<"_<$filename">. This is +also the case for evaluated strings that contain subroutines, or +which are currently being executed. The $filename for C<eval>ed +strings looks like C<(eval 34)> or C<(re_eval 19)>. + +=item * + +After each C<require>d file is compiled, but before it is executed, +C<DB::postponed(*{"_<$filename"})> is called if the subroutine +C<DB::postponed> exists. Here, the $filename is the expanded name of +the C<require>d file, as found in the values of %INC. + +=item * + +After each subroutine C<subname> is compiled, the existence of +C<$DB::postponed{subname}> is checked. If this key exists, +C<DB::postponed(subname)> is called if the C<DB::postponed> subroutine +also exists. + +=item * + +A hash C<%DB::sub> is maintained, whose keys are subroutine names +and whose values have the form C<filename:startline-endline>. +C<filename> has the form C<(eval 34)> for subroutines defined inside +C<eval>s, or C<(re_eval 19)> for those within regex code assertions. + +=item * + +When the execution of your program reaches a point that can hold a +breakpoint, the C<DB::DB()> subroutine is called any of the variables +$DB::trace, $DB::single, or $DB::signal is true. These variables +are not C<local>izable. This feature is disabled when executing +inside C<DB::DB()>, including functions called from it +unless C<< $^D & (1<<30) >> is true. + +=item * + +When execution of the program reaches a subroutine call, a call to +C<&DB::sub>(I<args>) is made instead, with C<$DB::sub> holding the +name of the called subroutine. This doesn't happen if the subroutine +was compiled in the C<DB> package.) + +=back + +Note that if C<&DB::sub> needs external data for it to work, no +subroutine call is possible until this is done. For the standard +debugger, the C<$DB::deep> variable (how many levels of recursion +deep into the debugger you can go before a mandatory break) gives +an example of such a dependency. + +=head2 Writing Your Own Debugger + +The minimal working debugger consists of one line + + sub DB::DB {} + +which is quite handy as contents of C<PERL5DB> environment +variable: + + $ PERL5DB="sub DB::DB {}" perl -d your-script + +Another brief debugger, slightly more useful, could be created +with only the line: + + sub DB::DB {print ++$i; scalar <STDIN>} + +This debugger would print the sequential number of encountered +statement, and would wait for you to hit a newline before continuing. + +The following debugger is quite functional: + + { + package DB; + sub DB {} + sub sub {print ++$i, " $sub\n"; &$sub} + } + +It prints the sequential number of subroutine call and the name of the +called subroutine. Note that C<&DB::sub> should be compiled into the +package C<DB>. + +At the start, the debugger reads your rc file (F<./.perldb> or +F<~/.perldb> under Unix), which can set important options. This file may +define a subroutine C<&afterinit> to be executed after the debugger is +initialized. + +After the rc file is read, the debugger reads the PERLDB_OPTS +environment variable and parses this as the remainder of a C<O ...> +line as one might enter at the debugger prompt. + +The debugger also maintains magical internal variables, such as +C<@DB::dbline>, C<%DB::dbline>, which are aliases for +C<@{"::_<current_file"}> C<%{"::_<current_file"}>. Here C<current_file> +is the currently selected file, either explicitly chosen with the +debugger's C<f> command, or implicitly by flow of execution. + +Some functions are provided to simplify customization. See +L<perldebug/"Options"> for description of options parsed by +C<DB::parse_options(string)>. The function C<DB::dump_trace(skip[, +count])> skips the specified number of frames and returns a list +containing information about the calling frames (all of them, if +C<count> is missing). Each entry is reference to a a hash with +keys C<context> (either C<.>, C<$>, or C<@>), C<sub> (subroutine +name, or info about C<eval>), C<args> (C<undef> or a reference to +an array), C<file>, and C<line>. + +The function C<DB::print_trace(FH, skip[, count[, short]])> prints +formatted info about caller frames. The last two functions may be +convenient as arguments to C<< < >>, C<< << >> commands. + +Note that any variables and functions that are not documented in +this manpages (or in L<perldebug>) are considered for internal +use only, and as such are subject to change without notice. + +=head1 Frame Listing Output Examples + +The C<frame> option can be used to control the output of frame +information. For example, contrast this expression trace: + + $ perl -de 42 + Stack dump during die enabled outside of evals. + + Loading DB routines from perl5db.pl patch level 0.94 + Emacs support available. + + Enter h or `h h' for help. + + main::(-e:1): 0 + DB<1> sub foo { 14 } + + DB<2> sub bar { 3 } + + DB<3> t print foo() * bar() + main::((eval 172):3): print foo() + bar(); + main::foo((eval 168):2): + main::bar((eval 170):2): + 42 + +with this one, once the C<O>ption C<frame=2> has been set: + + DB<4> O f=2 + frame = '2' + DB<5> t print foo() * bar() + 3: foo() * bar() + entering main::foo + 2: sub foo { 14 }; + exited main::foo + entering main::bar + 2: sub bar { 3 }; + exited main::bar + 42 + +By way of demonstration, we present below a laborious listing +resulting from setting your C<PERLDB_OPTS> environment variable to +the value C<f=n N>, and running I<perl -d -V> from the command line. +Examples use various values of C<n> are shown to give you a feel +for the difference between settings. Long those it may be, this +is not a complete listing, but only excerpts. + +=over 4 + +=item 1 + + entering main::BEGIN + entering Config::BEGIN + Package lib/Exporter.pm. + Package lib/Carp.pm. + Package lib/Config.pm. + entering Config::TIEHASH + entering Exporter::import + entering Exporter::export + entering Config::myconfig + entering Config::FETCH + entering Config::FETCH + entering Config::FETCH + entering Config::FETCH + +=item 2 + + entering main::BEGIN + entering Config::BEGIN + Package lib/Exporter.pm. + Package lib/Carp.pm. + exited Config::BEGIN + Package lib/Config.pm. + entering Config::TIEHASH + exited Config::TIEHASH + entering Exporter::import + entering Exporter::export + exited Exporter::export + exited Exporter::import + exited main::BEGIN + entering Config::myconfig + entering Config::FETCH + exited Config::FETCH + entering Config::FETCH + exited Config::FETCH + entering Config::FETCH + +=item 4 + + in $=main::BEGIN() from /dev/null:0 + in $=Config::BEGIN() from lib/Config.pm:2 + Package lib/Exporter.pm. + Package lib/Carp.pm. + Package lib/Config.pm. + in $=Config::TIEHASH('Config') from lib/Config.pm:644 + in $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0 + in $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from li + in @=Config::myconfig() from /dev/null:0 + in $=Config::FETCH(ref(Config), 'package') from lib/Config.pm:574 + in $=Config::FETCH(ref(Config), 'baserev') from lib/Config.pm:574 + in $=Config::FETCH(ref(Config), 'PERL_VERSION') from lib/Config.pm:574 + in $=Config::FETCH(ref(Config), 'PERL_SUBVERSION') from lib/Config.pm:574 + in $=Config::FETCH(ref(Config), 'osname') from lib/Config.pm:574 + in $=Config::FETCH(ref(Config), 'osvers') from lib/Config.pm:574 + +=item 6 + + in $=main::BEGIN() from /dev/null:0 + in $=Config::BEGIN() from lib/Config.pm:2 + Package lib/Exporter.pm. + Package lib/Carp.pm. + out $=Config::BEGIN() from lib/Config.pm:0 + Package lib/Config.pm. + in $=Config::TIEHASH('Config') from lib/Config.pm:644 + out $=Config::TIEHASH('Config') from lib/Config.pm:644 + in $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0 + in $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from lib/ + out $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from lib/ + out $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0 + out $=main::BEGIN() from /dev/null:0 + in @=Config::myconfig() from /dev/null:0 + in $=Config::FETCH(ref(Config), 'package') from lib/Config.pm:574 + out $=Config::FETCH(ref(Config), 'package') from lib/Config.pm:574 + in $=Config::FETCH(ref(Config), 'baserev') from lib/Config.pm:574 + out $=Config::FETCH(ref(Config), 'baserev') from lib/Config.pm:574 + in $=Config::FETCH(ref(Config), 'PERL_VERSION') from lib/Config.pm:574 + out $=Config::FETCH(ref(Config), 'PERL_VERSION') from lib/Config.pm:574 + in $=Config::FETCH(ref(Config), 'PERL_SUBVERSION') from lib/Config.pm:574 + +=item 14 + + in $=main::BEGIN() from /dev/null:0 + in $=Config::BEGIN() from lib/Config.pm:2 + Package lib/Exporter.pm. + Package lib/Carp.pm. + out $=Config::BEGIN() from lib/Config.pm:0 + Package lib/Config.pm. + in $=Config::TIEHASH('Config') from lib/Config.pm:644 + out $=Config::TIEHASH('Config') from lib/Config.pm:644 + in $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0 + in $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from lib/E + out $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from lib/E + out $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0 + out $=main::BEGIN() from /dev/null:0 + in @=Config::myconfig() from /dev/null:0 + in $=Config::FETCH('Config=HASH(0x1aa444)', 'package') from lib/Config.pm:574 + out $=Config::FETCH('Config=HASH(0x1aa444)', 'package') from lib/Config.pm:574 + in $=Config::FETCH('Config=HASH(0x1aa444)', 'baserev') from lib/Config.pm:574 + out $=Config::FETCH('Config=HASH(0x1aa444)', 'baserev') from lib/Config.pm:574 + +=item 30 + + in $=CODE(0x15eca4)() from /dev/null:0 + in $=CODE(0x182528)() from lib/Config.pm:2 + Package lib/Exporter.pm. + out $=CODE(0x182528)() from lib/Config.pm:0 + scalar context return from CODE(0x182528): undef + Package lib/Config.pm. + in $=Config::TIEHASH('Config') from lib/Config.pm:628 + out $=Config::TIEHASH('Config') from lib/Config.pm:628 + scalar context return from Config::TIEHASH: empty hash + in $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0 + in $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from lib/Exporter.pm:171 + out $=Exporter::export('Config', 'main', 'myconfig', 'config_vars') from lib/Exporter.pm:171 + scalar context return from Exporter::export: '' + out $=Exporter::import('Config', 'myconfig', 'config_vars') from /dev/null:0 + scalar context return from Exporter::import: '' + +=back + +In all cases shown above, the line indentation shows the call tree. +If bit 2 of C<frame> is set, a line is printed on exit from a +subroutine as well. If bit 4 is set, the arguments are printed +along with the caller info. If bit 8 is set, the arguments are +printed even if they are tied or references. If bit 16 is set, the +return value is printed, too. + +When a package is compiled, a line like this + + Package lib/Carp.pm. + +is printed with proper indentation. + +=head1 Debugging regular expressions + +There are two ways to enable debugging output for regular expressions. + +If your perl is compiled with C<-DDEBUGGING>, you may use the +B<-Dr> flag on the command line. + +Otherwise, one can C<use re 'debug'>, which has effects at +compile time and run time. It is not lexically scoped. + +=head2 Compile-time output + +The debugging output at compile time looks like this: + + compiling RE `[bc]d(ef*g)+h[ij]k$' + size 43 first at 1 + 1: ANYOF(11) + 11: EXACT <d>(13) + 13: CURLYX {1,32767}(27) + 15: OPEN1(17) + 17: EXACT <e>(19) + 19: STAR(22) + 20: EXACT <f>(0) + 22: EXACT <g>(24) + 24: CLOSE1(26) + 26: WHILEM(0) + 27: NOTHING(28) + 28: EXACT <h>(30) + 30: ANYOF(40) + 40: EXACT <k>(42) + 42: EOL(43) + 43: END(0) + anchored `de' at 1 floating `gh' at 3..2147483647 (checking floating) + stclass `ANYOF' minlen 7 + +The first line shows the pre-compiled form of the regex. The second +shows the size of the compiled form (in arbitrary units, usually +4-byte words) and the label I<id> of the first node that does a +match. + +The last line (split into two lines above) contains optimizer +information. In the example shown, the optimizer found that the match +should contain a substring C<de> at offset 1, plus substring C<gh> +at some offset between 3 and infinity. Moreover, when checking for +these substrings (to abandon impossible matches quickly), Perl will check +for the substring C<gh> before checking for the substring C<de>. The +optimizer may also use the knowledge that the match starts (at the +C<first> I<id>) with a character class, and the match cannot be +shorter than 7 chars. + +The fields of interest which may appear in the last line are + +=over + +=item C<anchored> I<STRING> C<at> I<POS> + +=item C<floating> I<STRING> C<at> I<POS1..POS2> + +See above. + +=item C<matching floating/anchored> + +Which substring to check first. + +=item C<minlen> + +The minimal length of the match. + +=item C<stclass> I<TYPE> + +Type of first matching node. + +=item C<noscan> + +Don't scan for the found substrings. + +=item C<isall> + +Means that the optimizer info is all that the regular +expression contains, and thus one does not need to enter the regex engine at +all. + +=item C<GPOS> + +Set if the pattern contains C<\G>. + +=item C<plus> + +Set if the pattern starts with a repeated char (as in C<x+y>). + +=item C<implicit> + +Set if the pattern starts with C<.*>. + +=item C<with eval> + +Set if the pattern contain eval-groups, such as C<(?{ code })> and +C<(??{ code })>. + +=item C<anchored(TYPE)> + +If the pattern may match only at a handful of places, (with C<TYPE> +being C<BOL>, C<MBOL>, or C<GPOS>. See the table below. + +=back + +If a substring is known to match at end-of-line only, it may be +followed by C<$>, as in C<floating `k'$>. + +The optimizer-specific info is used to avoid entering (a slow) regex +engine on strings that will not definitely match. If C<isall> flag +is set, a call to the regex engine may be avoided even when the optimizer +found an appropriate place for the match. + +The rest of the output contains the list of I<nodes> of the compiled +form of the regex. Each line has format + +C< >I<id>: I<TYPE> I<OPTIONAL-INFO> (I<next-id>) + +=head2 Types of nodes + +Here are the possible types, with short descriptions: + + # TYPE arg-description [num-args] [longjump-len] DESCRIPTION + + # Exit points + END no End of program. + SUCCEED no Return from a subroutine, basically. + + # Anchors: + BOL no Match "" at beginning of line. + MBOL no Same, assuming multiline. + SBOL no Same, assuming singleline. + EOS no Match "" at end of string. + EOL no Match "" at end of line. + MEOL no Same, assuming multiline. + SEOL no Same, assuming singleline. + BOUND no Match "" at any word boundary + BOUNDL no Match "" at any word boundary + NBOUND no Match "" at any word non-boundary + NBOUNDL no Match "" at any word non-boundary + GPOS no Matches where last m//g left off. + + # [Special] alternatives + ANY no Match any one character (except newline). + SANY no Match any one character. + ANYOF sv Match character in (or not in) this class. + ALNUM no Match any alphanumeric character + ALNUML no Match any alphanumeric char in locale + NALNUM no Match any non-alphanumeric character + NALNUML no Match any non-alphanumeric char in locale + SPACE no Match any whitespace character + SPACEL no Match any whitespace char in locale + NSPACE no Match any non-whitespace character + NSPACEL no Match any non-whitespace char in locale + DIGIT no Match any numeric character + NDIGIT no Match any non-numeric character + + # BRANCH The set of branches constituting a single choice are hooked + # together with their "next" pointers, since precedence prevents + # anything being concatenated to any individual branch. The + # "next" pointer of the last BRANCH in a choice points to the + # thing following the whole choice. This is also where the + # final "next" pointer of each individual branch points; each + # branch starts with the operand node of a BRANCH node. + # + BRANCH node Match this alternative, or the next... + + # BACK Normal "next" pointers all implicitly point forward; BACK + # exists to make loop structures possible. + # not used + BACK no Match "", "next" ptr points backward. + + # Literals + EXACT sv Match this string (preceded by length). + EXACTF sv Match this string, folded (prec. by length). + EXACTFL sv Match this string, folded in locale (w/len). + + # Do nothing + NOTHING no Match empty string. + # A variant of above which delimits a group, thus stops optimizations + TAIL no Match empty string. Can jump here from outside. + + # STAR,PLUS '?', and complex '*' and '+', are implemented as circular + # BRANCH structures using BACK. Simple cases (one character + # per match) are implemented with STAR and PLUS for speed + # and to minimize recursive plunges. + # + STAR node Match this (simple) thing 0 or more times. + PLUS node Match this (simple) thing 1 or more times. + + CURLY sv 2 Match this simple thing {n,m} times. + CURLYN no 2 Match next-after-this simple thing + # {n,m} times, set parens. + CURLYM no 2 Match this medium-complex thing {n,m} times. + CURLYX sv 2 Match this complex thing {n,m} times. + + # This terminator creates a loop structure for CURLYX + WHILEM no Do curly processing and see if rest matches. + + # OPEN,CLOSE,GROUPP ...are numbered at compile time. + OPEN num 1 Mark this point in input as start of #n. + CLOSE num 1 Analogous to OPEN. + + REF num 1 Match some already matched string + REFF num 1 Match already matched string, folded + REFFL num 1 Match already matched string, folded in loc. + + # grouping assertions + IFMATCH off 1 2 Succeeds if the following matches. + UNLESSM off 1 2 Fails if the following matches. + SUSPEND off 1 1 "Independent" sub-regex. + IFTHEN off 1 1 Switch, should be preceded by switcher . + GROUPP num 1 Whether the group matched. + + # Support for long regex + LONGJMP off 1 1 Jump far away. + BRANCHJ off 1 1 BRANCH with long offset. + + # The heavy worker + EVAL evl 1 Execute some Perl code. + + # Modifiers + MINMOD no Next operator is not greedy. + LOGICAL no Next opcode should set the flag only. + + # This is not used yet + RENUM off 1 1 Group with independently numbered parens. + + # This is not really a node, but an optimized away piece of a "long" node. + # To simplify debugging output, we mark it as if it were a node + OPTIMIZED off Placeholder for dump. + +=head2 Run-time output + +First of all, when doing a match, one may get no run-time output even +if debugging is enabled. This means that the regex engine was never +entered and that all of the job was therefore done by the optimizer. + +If the regex engine was entered, the output may look like this: + + Matching `[bc]d(ef*g)+h[ij]k$' against `abcdefg__gh__' + Setting an EVAL scope, savestack=3 + 2 <ab> <cdefg__gh_> | 1: ANYOF + 3 <abc> <defg__gh_> | 11: EXACT <d> + 4 <abcd> <efg__gh_> | 13: CURLYX {1,32767} + 4 <abcd> <efg__gh_> | 26: WHILEM + 0 out of 1..32767 cc=effff31c + 4 <abcd> <efg__gh_> | 15: OPEN1 + 4 <abcd> <efg__gh_> | 17: EXACT <e> + 5 <abcde> <fg__gh_> | 19: STAR + EXACT <f> can match 1 times out of 32767... + Setting an EVAL scope, savestack=3 + 6 <bcdef> <g__gh__> | 22: EXACT <g> + 7 <bcdefg> <__gh__> | 24: CLOSE1 + 7 <bcdefg> <__gh__> | 26: WHILEM + 1 out of 1..32767 cc=effff31c + Setting an EVAL scope, savestack=12 + 7 <bcdefg> <__gh__> | 15: OPEN1 + 7 <bcdefg> <__gh__> | 17: EXACT <e> + restoring \1 to 4(4)..7 + failed, try continuation... + 7 <bcdefg> <__gh__> | 27: NOTHING + 7 <bcdefg> <__gh__> | 28: EXACT <h> + failed... + failed... + +The most significant information in the output is about the particular I<node> +of the compiled regex that is currently being tested against the target string. +The format of these lines is + +C< >I<STRING-OFFSET> <I<PRE-STRING>> <I<POST-STRING>> |I<ID>: I<TYPE> + +The I<TYPE> info is indented with respect to the backtracking level. +Other incidental information appears interspersed within. + +=head1 Debugging Perl memory usage + +Perl is a profligate wastrel when it comes to memory use. There +is a saying that to estimate memory usage of Perl, assume a reasonable +algorithm for memory allocation, multiply that estimate by 10, and +while you still may miss the mark, at least you won't be quite so +astonished. This is not absolutely true, but may prvide a good +grasp of what happens. + +Assume that an integer cannot take less than 20 bytes of memory, a +float cannot take less than 24 bytes, a string cannot take less +than 32 bytes (all these examples assume 32-bit architectures, the +result are quite a bit worse on 64-bit architectures). If a variable +is accessed in two of three different ways (which require an integer, +a float, or a string), the memory footprint may increase yet another +20 bytes. A sloppy malloc(3) implementation can make inflate these +numbers dramatically. + +On the opposite end of the scale, a declaration like + + sub foo; + +may take up to 500 bytes of memory, depending on which release of Perl +you're running. + +Anecdotal estimates of source-to-compiled code bloat suggest an +eightfold increase. This means that the compiled form of reasonable +(normally commented, properly indented etc.) code will take +about eight times more space in memory than the code took +on disk. + +There are two Perl-specific ways to analyze memory usage: +$ENV{PERL_DEBUG_MSTATS} and B<-DL> command-line switch. The first +is available only if Perl is compiled with Perl's malloc(); the +second only if Perl was built with C<-DDEBUGGING>. See the +instructions for how to do this in the F<INSTALL> podpage at +the top level of the Perl source tree. + +=head2 Using C<$ENV{PERL_DEBUG_MSTATS}> + +If your perl is using Perl's malloc() and was compiled with the +necessary switches (this is the default), then it will print memory +usage statistics after compiling your code hwen C<< $ENV{PERL_DEBUG_MSTATS} +> 1 >>, and before termination of the program when C<< +$ENV{PERL_DEBUG_MSTATS} >= 1 >>. The report format is similar to +the following example: + + $ PERL_DEBUG_MSTATS=2 perl -e "require Carp" + Memory allocation statistics after compilation: (buckets 4(4)..8188(8192) + 14216 free: 130 117 28 7 9 0 2 2 1 0 0 + 437 61 36 0 5 + 60924 used: 125 137 161 55 7 8 6 16 2 0 1 + 74 109 304 84 20 + Total sbrk(): 77824/21:119. Odd ends: pad+heads+chain+tail: 0+636+0+2048. + Memory allocation statistics after execution: (buckets 4(4)..8188(8192) + 30888 free: 245 78 85 13 6 2 1 3 2 0 1 + 315 162 39 42 11 + 175816 used: 265 176 1112 111 26 22 11 27 2 1 1 + 196 178 1066 798 39 + Total sbrk(): 215040/47:145. Odd ends: pad+heads+chain+tail: 0+2192+0+6144. + +It is possible to ask for such a statistic at arbitrary points in +your execution using the mstats() function out of the standard +Devel::Peek module. + +Here is some explanation of that format: + +=over + +=item C<buckets SMALLEST(APPROX)..GREATEST(APPROX)> + +Perl's malloc() uses bucketed allocations. Every request is rounded +up to the closest bucket size available, and a bucket is taken from +the pool of buckets of that size. + +The line above describes the limits of buckets currently in use. +Each bucket has two sizes: memory footprint and the maximal size +of user data that can fit into this bucket. Suppose in the above +example that the smallest bucket were size 4. The biggest bucket +would have usable size 8188, and the memory footprint would be 8192. + +In a Perl built for debugging, some buckets may have negative usable +size. This means that these buckets cannot (and will not) be used. +For larger buckets, the memory footprint may be one page greater +than a power of 2. If so, case the corresponding power of two is +printed in the C<APPROX> field above. + +=item Free/Used + +The 1 or 2 rows of numbers following that correspond to the number +of buckets of each size between C<SMALLEST> and C<GREATEST>. In +the first row, the sizes (memory footprints) of buckets are powers +of two--or possibly one page greater. In the second row, if present, +the memory footprints of the buckets are between the memory footprints +of two buckets "above". + +For example, suppose under the pervious example, the memory footprints +were + + free: 8 16 32 64 128 256 512 1024 2048 4096 8192 + 4 12 24 48 80 + +With non-C<DEBUGGING> perl, the buckets starting from C<128> have +a 4-byte overhead, and thus a 8192-long bucket may take up to +8188-byte allocations. + +=item C<Total sbrk(): SBRKed/SBRKs:CONTINUOUS> + +The first two fields give the total amount of memory perl sbrk(2)ed +(ess-broken? :-) and number of sbrk(2)s used. The third number is +what perl thinks about continuity of returned chunks. So long as +this number is positive, malloc() will assume that it is probable +that sbrk(2) will provide continuous memory. + +Memory allocated by external libraries is not counted. + +=item C<pad: 0> + +The amount of sbrk(2)ed memory needed to keep buckets aligned. + +=item C<heads: 2192> + +Although memory overhead of bigger buckets is kept inside the bucket, for +smaller buckets, it is kept in separate areas. This field gives the +total size of these areas. + +=item C<chain: 0> + +malloc() may want to subdivide a bigger bucket into smaller buckets. +If only a part of the deceased bucket is left unsubdivided, the rest +is kept as an element of a linked list. This field gives the total +size of these chunks. + +=item C<tail: 6144> + +To minimize the number of sbrk(2)s, malloc() asks for more memory. This +field gives the size of the yet unused part, which is sbrk(2)ed, but +never touched. + +=back + +=head2 Example of using B<-DL> switch + +Below we show how to analyse memory usage by + + do 'lib/auto/POSIX/autosplit.ix'; + +The file in question contains a header and 146 lines similar to + + sub getcwd; + +B<WARNING>: The discussion below supposes 32-bit architecture. In +newer releases of Perl, memory usage of the constructs discussed +here is greatly improved, but the story discussed below is a real-life +story. This story is mercilessly terse, and assumes rather more than cursory +knowledge of Perl internals. Type space to continue, `q' to quit. +(Actually, you just want to skip to the next section.) + +Here is the itemized list of Perl allocations performed during parsing +of this file: + + !!! "after" at test.pl line 3. + Id subtot 4 8 12 16 20 24 28 32 36 40 48 56 64 72 80 80+ + 0 02 13752 . . . . 294 . . . . . . . . . . 4 + 0 54 5545 . . 8 124 16 . . . 1 1 . . . . . 3 + 5 05 32 . . . . . . . 1 . . . . . . . . + 6 02 7152 . . . . . . . . . . 149 . . . . . + 7 02 3600 . . . . . 150 . . . . . . . . . . + 7 03 64 . -1 . 1 . . 2 . . . . . . . . . + 7 04 7056 . . . . . . . . . . . . . . . 7 + 7 17 38404 . . . . . . . 1 . . 442 149 . . 147 . + 9 03 2078 17 249 32 . . . . 2 . . . . . . . . + + +To see this list, insert two C<warn('!...')> statements around the call: + + warn('!'); + do 'lib/auto/POSIX/autosplit.ix'; + warn('!!! "after"'); + +and run it with PErl's B<-DL> option. The first warn() will print +memory allocation info before parsing the file and will memorize +the statistics at this point (we ignore what it prints). The second +warn() prints increments with respect to these memorized data. This +is the printout shown above. + +Different I<Id>s on the left correspond to different subsystems of +the perl interpreter. They are just the first argument given to +the perl memory allocation API named New(). To find what C<9 03> +means, just B<grep> the perl source for C<903>. You'll find it in +F<util.c>, function savepvn(). (I know, you wonder why we told you +to B<grep> and then gave away the answer. That's because grepping +the source is good for the soul.) This function is used to store +a copy of an existing chunk of memory. Using a C debugger, one can +see that the function was called either directly from gv_init() or +via sv_magic(), and that gv_init() is called from gv_fetchpv()--which +was itself called from newSUB(). Please stop to catch your breath now. + +B<NOTE>: To reach this point in the debugger and skip the calls to +savepvn() during the compilation of the main program, you should +set a C breakpoint +in Perl_warn(), continue until this point is reached, and I<then> set +a C breakpoint in Perl_savepvn(). Note that you may need to skip a +handful of Perl_savepvn() calls that do not correspond to mass production +of CVs (there are more C<903> allocations than 146 similar lines of +F<lib/auto/POSIX/autosplit.ix>). Note also that C<Perl_> prefixes are +added by macroization code in perl header files to avoid conflicts +with external libraries. + +Anyway, we see that C<903> ids correspond to creation of globs, twice +per glob - for glob name, and glob stringification magic. + +Here are explanations for other I<Id>s above: + +=over + +=item C<717> + +CReates bigger C<XPV*> structures. In the case above, it +creates 3 C<AV>s per subroutine, one for a list of lexical variable +names, one for a scratchpad (which contains lexical variables and +C<targets>), and one for the array of scratchpads needed for +recursion. + +It also creates a C<GV> and a C<CV> per subroutine, all called from +start_subparse(). + +=item C<002> + +Creates a C array corresponding to the C<AV> of scratchpads and the +scratchpad itself. The first fake entry of this scratchpad is +created though the subroutine itself is not defined yet. + +It also creates C arrays to keep data for the stash. This is one HV, +but it grows; thus, there are 4 big allocations: the big chunks are not +freed, but are kept as additional arenas for C<SV> allocations. + +=item C<054> + +Creates a C<HEK> for the name of the glob for the subroutine. This +name is a key in a I<stash>. + +Big allocations with this I<Id> correspond to allocations of new +arenas to keep C<HE>. + +=item C<602> + +Creates a C<GP> for the glob for the subroutine. + +=item C<702> + +Creates the C<MAGIC> for the glob for the subroutine. + +=item C<704> + +Creates I<arenas> which keep SVs. + +=back + +=head2 B<-DL> details + +If Perl is run with B<-DL> option, then warn()s that start with `!' +behave specially. They print a list of I<categories> of memory +allocations, and statistics of allocations of different sizes for +these categories. + +If warn() string starts with + +=over + +=item C<!!!> + +print changed categories only, print the differences in counts of allocations. + +=item C<!!> + +print grown categories only; print the absolute values of counts, and totals. + +=item C<!> + +print nonempty categories, print the absolute values of counts and totals. + +=back + +=head2 Limitations of B<-DL> statistics + +If an extension or external library does not use the Perl API to +allocate memory, such allocations are not counted. + +=head1 SEE ALSO + +L<perldebug>, +L<perlguts>, +L<perlrun> +L<re>, +and +L<Devel::Dprof>. |
