diff options
Diffstat (limited to 'docs/DriverInternals.html')
| -rw-r--r-- | docs/DriverInternals.html | 523 |
1 files changed, 0 insertions, 523 deletions
diff --git a/docs/DriverInternals.html b/docs/DriverInternals.html deleted file mode 100644 index ce707b9..0000000 --- a/docs/DriverInternals.html +++ /dev/null @@ -1,523 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" - "http://www.w3.org/TR/html4/strict.dtd"> -<html> - <head> - <title>Clang Driver Manual</title> - <link type="text/css" rel="stylesheet" href="../menu.css"> - <link type="text/css" rel="stylesheet" href="../content.css"> - <style type="text/css"> - td { - vertical-align: top; - } - </style> - </head> - <body> - - <!--#include virtual="../menu.html.incl"--> - - <div id="content"> - - <h1>Driver Design & Internals</h1> - - <ul> - <li><a href="#intro">Introduction</a></li> - <li><a href="#features">Features and Goals</a> - <ul> - <li><a href="#gcccompat">GCC Compatibility</a></li> - <li><a href="#components">Flexible</a></li> - <li><a href="#performance">Low Overhead</a></li> - <li><a href="#simple">Simple</a></li> - </ul> - </li> - <li><a href="#design">Design</a> - <ul> - <li><a href="#int_intro">Internals Introduction</a></li> - <li><a href="#int_overview">Design Overview</a></li> - <li><a href="#int_notes">Additional Notes</a> - <ul> - <li><a href="#int_compilation">The Compilation Object</a></li> - <li><a href="#int_unified_parsing">Unified Parsing & Pipelining</a></li> - <li><a href="#int_toolchain_translation">ToolChain Argument Translation</a></li> - <li><a href="#int_unused_warnings">Unused Argument Warnings</a></li> - </ul> - </li> - <li><a href="#int_gcc_concepts">Relation to GCC Driver Concepts</a></li> - </ul> - </li> - </ul> - - - <!-- ======================================================================= --> - <h2 id="intro">Introduction</h2> - <!-- ======================================================================= --> - - <p>This document describes the Clang driver. The purpose of this - document is to describe both the motivation and design goals - for the driver, as well as details of the internal - implementation.</p> - - <!-- ======================================================================= --> - <h2 id="features">Features and Goals</h2> - <!-- ======================================================================= --> - - <p>The Clang driver is intended to be a production quality - compiler driver providing access to the Clang compiler and - tools, with a command line interface which is compatible with - the gcc driver.</p> - - <p>Although the driver is part of and driven by the Clang - project, it is logically a separate tool which shares many of - the same goals as Clang:</p> - - <p><b>Features</b>:</p> - <ul> - <li><a href="#gcccompat">GCC Compatibility</a></li> - <li><a href="#components">Flexible</a></li> - <li><a href="#performance">Low Overhead</a></li> - <li><a href="#simple">Simple</a></li> - </ul> - - <!--=======================================================================--> - <h3 id="gcccompat">GCC Compatibility</h3> - <!--=======================================================================--> - - <p>The number one goal of the driver is to ease the adoption of - Clang by allowing users to drop Clang into a build system - which was designed to call GCC. Although this makes the driver - much more complicated than might otherwise be necessary, we - decided that being very compatible with the gcc command line - interface was worth it in order to allow users to quickly test - clang on their projects.</p> - - <!--=======================================================================--> - <h3 id="components">Flexible</h3> - <!--=======================================================================--> - - <p>The driver was designed to be flexible and easily accommodate - new uses as we grow the clang and LLVM infrastructure. As one - example, the driver can easily support the introduction of - tools which have an integrated assembler; something we hope to - add to LLVM in the future.</p> - - <p>Similarly, most of the driver functionality is kept in a - library which can be used to build other tools which want to - implement or accept a gcc like interface. </p> - - <!--=======================================================================--> - <h3 id="performance">Low Overhead</h3> - <!--=======================================================================--> - - <p>The driver should have as little overhead as possible. In - practice, we found that the gcc driver by itself incurred a - small but meaningful overhead when compiling many small - files. The driver doesn't do much work compared to a - compilation, but we have tried to keep it as efficient as - possible by following a few simple principles:</p> - <ul> - <li>Avoid memory allocation and string copying when - possible.</li> - - <li>Don't parse arguments more than once.</li> - - <li>Provide a few simple interfaces for efficiently searching - arguments.</li> - </ul> - - <!--=======================================================================--> - <h3 id="simple">Simple</h3> - <!--=======================================================================--> - - <p>Finally, the driver was designed to be "as simple as - possible", given the other goals. Notably, trying to be - completely compatible with the gcc driver adds a significant - amount of complexity. However, the design of the driver - attempts to mitigate this complexity by dividing the process - into a number of independent stages instead of a single - monolithic task.</p> - - <!-- ======================================================================= --> - <h2 id="design">Internal Design and Implementation</h2> - <!-- ======================================================================= --> - - <ul> - <li><a href="#int_intro">Internals Introduction</a></li> - <li><a href="#int_overview">Design Overview</a></li> - <li><a href="#int_notes">Additional Notes</a></li> - <li><a href="#int_gcc_concepts">Relation to GCC Driver Concepts</a></li> - </ul> - - <!--=======================================================================--> - <h3><a name="int_intro">Internals Introduction</a></h3> - <!--=======================================================================--> - - <p>In order to satisfy the stated goals, the driver was designed - to completely subsume the functionality of the gcc executable; - that is, the driver should not need to delegate to gcc to - perform subtasks. On Darwin, this implies that the Clang - driver also subsumes the gcc driver-driver, which is used to - implement support for building universal images (binaries and - object files). This also implies that the driver should be - able to call the language specific compilers (e.g. cc1) - directly, which means that it must have enough information to - forward command line arguments to child processes - correctly.</p> - - <!--=======================================================================--> - <h3><a name="int_overview">Design Overview</a></h3> - <!--=======================================================================--> - - <p>The diagram below shows the significant components of the - driver architecture and how they relate to one another. The - orange components represent concrete data structures built by - the driver, the green components indicate conceptually - distinct stages which manipulate these data structures, and - the blue components are important helper classes. </p> - - <div style="text-align:center"> - <a href="DriverArchitecture.png"> - <img width=400 src="DriverArchitecture.png" - alt="Driver Architecture Diagram"> - </a> - </div> - - <!--=======================================================================--> - <h3><a name="int_stages">Driver Stages</a></h3> - <!--=======================================================================--> - - <p>The driver functionality is conceptually divided into five stages:</p> - - <ol> - <li> - <b>Parse: Option Parsing</b> - - <p>The command line argument strings are decomposed into - arguments (<tt>Arg</tt> instances). The driver expects to - understand all available options, although there is some - facility for just passing certain classes of options - through (like <tt>-Wl,</tt>).</p> - - <p>Each argument corresponds to exactly one - abstract <tt>Option</tt> definition, which describes how - the option is parsed along with some additional - metadata. The Arg instances themselves are lightweight and - merely contain enough information for clients to determine - which option they correspond to and their values (if they - have additional parameters).</p> - - <p>For example, a command line like "-Ifoo -I foo" would - parse to two Arg instances (a JoinedArg and a SeparateArg - instance), but each would refer to the same Option.</p> - - <p>Options are lazily created in order to avoid populating - all Option classes when the driver is loaded. Most of the - driver code only needs to deal with options by their - unique ID (e.g., <tt>options::OPT_I</tt>),</p> - - <p>Arg instances themselves do not generally store the - values of parameters. In many cases, this would - simply result in creating unnecessary string - copies. Instead, Arg instances are always embedded inside - an ArgList structure, which contains the original vector - of argument strings. Each Arg itself only needs to contain - an index into this vector instead of storing its values - directly.</p> - - <p>The clang driver can dump the results of this - stage using the <tt>-ccc-print-options</tt> flag (which - must precede any actual command line arguments). For - example:</p> - <pre> - $ <b>clang -ccc-print-options -Xarch_i386 -fomit-frame-pointer -Wa,-fast -Ifoo -I foo t.c</b> - Option 0 - Name: "-Xarch_", Values: {"i386", "-fomit-frame-pointer"} - Option 1 - Name: "-Wa,", Values: {"-fast"} - Option 2 - Name: "-I", Values: {"foo"} - Option 3 - Name: "-I", Values: {"foo"} - Option 4 - Name: "<input>", Values: {"t.c"} - </pre> - - <p>After this stage is complete the command line should be - broken down into well defined option objects with their - appropriate parameters. Subsequent stages should rarely, - if ever, need to do any string processing.</p> - </li> - - <li> - <b>Pipeline: Compilation Job Construction</b> - - <p>Once the arguments are parsed, the tree of subprocess - jobs needed for the desired compilation sequence are - constructed. This involves determining the input files and - their types, what work is to be done on them (preprocess, - compile, assemble, link, etc.), and constructing a list of - Action instances for each task. The result is a list of - one or more top-level actions, each of which generally - corresponds to a single output (for example, an object or - linked executable).</p> - - <p>The majority of Actions correspond to actual tasks, - however there are two special Actions. The first is - InputAction, which simply serves to adapt an input - argument for use as an input to other Actions. The second - is BindArchAction, which conceptually alters the - architecture to be used for all of its input Actions.</p> - - <p>The clang driver can dump the results of this - stage using the <tt>-ccc-print-phases</tt> flag. For - example:</p> - <pre> - $ <b>clang -ccc-print-phases -x c t.c -x assembler t.s</b> - 0: input, "t.c", c - 1: preprocessor, {0}, cpp-output - 2: compiler, {1}, assembler - 3: assembler, {2}, object - 4: input, "t.s", assembler - 5: assembler, {4}, object - 6: linker, {3, 5}, image - </pre> - <p>Here the driver is constructing seven distinct actions, - four to compile the "t.c" input into an object file, two to - assemble the "t.s" input, and one to link them together.</p> - - <p>A rather different compilation pipeline is shown here; in - this example there are two top level actions to compile - the input files into two separate object files, where each - object file is built using <tt>lipo</tt> to merge results - built for two separate architectures.</p> - <pre> - $ <b>clang -ccc-print-phases -c -arch i386 -arch x86_64 t0.c t1.c</b> - 0: input, "t0.c", c - 1: preprocessor, {0}, cpp-output - 2: compiler, {1}, assembler - 3: assembler, {2}, object - 4: bind-arch, "i386", {3}, object - 5: bind-arch, "x86_64", {3}, object - 6: lipo, {4, 5}, object - 7: input, "t1.c", c - 8: preprocessor, {7}, cpp-output - 9: compiler, {8}, assembler - 10: assembler, {9}, object - 11: bind-arch, "i386", {10}, object - 12: bind-arch, "x86_64", {10}, object - 13: lipo, {11, 12}, object - </pre> - - <p>After this stage is complete the compilation process is - divided into a simple set of actions which need to be - performed to produce intermediate or final outputs (in - some cases, like <tt>-fsyntax-only</tt>, there is no - "real" final output). Phases are well known compilation - steps, such as "preprocess", "compile", "assemble", - "link", etc.</p> - </li> - - <li> - <b>Bind: Tool & Filename Selection</b> - - <p>This stage (in conjunction with the Translate stage) - turns the tree of Actions into a list of actual subprocess - to run. Conceptually, the driver performs a top down - matching to assign Action(s) to Tools. The ToolChain is - responsible for selecting the tool to perform a particular - action; once selected the driver interacts with the tool - to see if it can match additional actions (for example, by - having an integrated preprocessor). - - <p>Once Tools have been selected for all actions, the driver - determines how the tools should be connected (for example, - using an inprocess module, pipes, temporary files, or user - provided filenames). If an output file is required, the - driver also computes the appropriate file name (the suffix - and file location depend on the input types and options - such as <tt>-save-temps</tt>). - - <p>The driver interacts with a ToolChain to perform the Tool - bindings. Each ToolChain contains information about all - the tools needed for compilation for a particular - architecture, platform, and operating system. A single - driver invocation may query multiple ToolChains during one - compilation in order to interact with tools for separate - architectures.</p> - - <p>The results of this stage are not computed directly, but - the driver can print the results via - the <tt>-ccc-print-bindings</tt> option. For example:</p> - <pre> - $ <b>clang -ccc-print-bindings -arch i386 -arch ppc t0.c</b> - # "i386-apple-darwin9" - "clang", inputs: ["t0.c"], output: "/tmp/cc-Sn4RKF.s" - # "i386-apple-darwin9" - "darwin::Assemble", inputs: ["/tmp/cc-Sn4RKF.s"], output: "/tmp/cc-gvSnbS.o" - # "i386-apple-darwin9" - "darwin::Link", inputs: ["/tmp/cc-gvSnbS.o"], output: "/tmp/cc-jgHQxi.out" - # "ppc-apple-darwin9" - "gcc::Compile", inputs: ["t0.c"], output: "/tmp/cc-Q0bTox.s" - # "ppc-apple-darwin9" - "gcc::Assemble", inputs: ["/tmp/cc-Q0bTox.s"], output: "/tmp/cc-WCdicw.o" - # "ppc-apple-darwin9" - "gcc::Link", inputs: ["/tmp/cc-WCdicw.o"], output: "/tmp/cc-HHBEBh.out" - # "i386-apple-darwin9" - "darwin::Lipo", inputs: ["/tmp/cc-jgHQxi.out", "/tmp/cc-HHBEBh.out"], output: "a.out" - </pre> - - <p>This shows the tool chain, tool, inputs and outputs which - have been bound for this compilation sequence. Here clang - is being used to compile t0.c on the i386 architecture and - darwin specific versions of the tools are being used to - assemble and link the result, but generic gcc versions of - the tools are being used on PowerPC.</p> - </li> - - <li> - <b>Translate: Tool Specific Argument Translation</b> - - <p>Once a Tool has been selected to perform a particular - Action, the Tool must construct concrete Jobs which will be - executed during compilation. The main work is in translating - from the gcc style command line options to whatever options - the subprocess expects.</p> - - <p>Some tools, such as the assembler, only interact with a - handful of arguments and just determine the path of the - executable to call and pass on their input and output - arguments. Others, like the compiler or the linker, may - translate a large number of arguments in addition.</p> - - <p>The ArgList class provides a number of simple helper - methods to assist with translating arguments; for example, - to pass on only the last of arguments corresponding to some - option, or all arguments for an option.</p> - - <p>The result of this stage is a list of Jobs (executable - paths and argument strings) to execute.</p> - </li> - - <li> - <b>Execute</b> - <p>Finally, the compilation pipeline is executed. This is - mostly straightforward, although there is some interaction - with options - like <tt>-pipe</tt>, <tt>-pass-exit-codes</tt> - and <tt>-time</tt>.</p> - </li> - - </ol> - - <!--=======================================================================--> - <h3><a name="int_notes">Additional Notes</a></h3> - <!--=======================================================================--> - - <h4 id="int_compilation">The Compilation Object</h4> - - <p>The driver constructs a Compilation object for each set of - command line arguments. The Driver itself is intended to be - invariant during construction of a Compilation; an IDE should be - able to construct a single long lived driver instance to use - for an entire build, for example.</p> - - <p>The Compilation object holds information that is particular - to each compilation sequence. For example, the list of used - temporary files (which must be removed once compilation is - finished) and result files (which should be removed if - compilation fails).</p> - - <h4 id="int_unified_parsing">Unified Parsing & Pipelining</h4> - - <p>Parsing and pipelining both occur without reference to a - Compilation instance. This is by design; the driver expects that - both of these phases are platform neutral, with a few very well - defined exceptions such as whether the platform uses a driver - driver.</p> - - <h4 id="int_toolchain_translation">ToolChain Argument Translation</h4> - - <p>In order to match gcc very closely, the clang driver - currently allows tool chains to perform their own translation of - the argument list (into a new ArgList data structure). Although - this allows the clang driver to match gcc easily, it also makes - the driver operation much harder to understand (since the Tools - stop seeing some arguments the user provided, and see new ones - instead).</p> - - <p>For example, on Darwin <tt>-gfull</tt> gets translated into two - separate arguments, <tt>-g</tt> - and <tt>-fno-eliminate-unused-debug-symbols</tt>. Trying to write Tool - logic to do something with <tt>-gfull</tt> will not work, because Tool - argument translation is done after the arguments have been - translated.</p> - - <p>A long term goal is to remove this tool chain specific - translation, and instead force each tool to change its own logic - to do the right thing on the untranslated original arguments.</p> - - <h4 id="int_unused_warnings">Unused Argument Warnings</h4> - <p>The driver operates by parsing all arguments but giving Tools - the opportunity to choose which arguments to pass on. One - downside of this infrastructure is that if the user misspells - some option, or is confused about which options to use, some - command line arguments the user really cared about may go - unused. This problem is particularly important when using - clang as a compiler, since the clang compiler does not support - anywhere near all the options that gcc does, and we want to make - sure users know which ones are being used.</p> - - <p>To support this, the driver maintains a bit associated with - each argument of whether it has been used (at all) during the - compilation. This bit usually doesn't need to be set by hand, - as the key ArgList accessors will set it automatically.</p> - - <p>When a compilation is successful (there are no errors), the - driver checks the bit and emits an "unused argument" warning for - any arguments which were never accessed. This is conservative - (the argument may not have been used to do what the user wanted) - but still catches the most obvious cases.</p> - - <!--=======================================================================--> - <h3><a name="int_gcc_concepts">Relation to GCC Driver Concepts</a></h3> - <!--=======================================================================--> - - <p>For those familiar with the gcc driver, this section provides - a brief overview of how things from the gcc driver map to the - clang driver.</p> - - <ul> - <li> - <b>Driver Driver</b> - <p>The driver driver is fully integrated into the clang - driver. The driver simply constructs additional Actions to - bind the architecture during the <i>Pipeline</i> - phase. The tool chain specific argument translation is - responsible for handling <tt>-Xarch_</tt>.</p> - - <p>The one caveat is that this approach - requires <tt>-Xarch_</tt> not be used to alter the - compilation itself (for example, one cannot - provide <tt>-S</tt> as an <tt>-Xarch_</tt> argument). The - driver attempts to reject such invocations, and overall - there isn't a good reason to abuse <tt>-Xarch_</tt> to - that end in practice.</p> - - <p>The upside is that the clang driver is more efficient and - does little extra work to support universal builds. It also - provides better error reporting and UI consistency.</p> - </li> - - <li> - <b>Specs</b> - <p>The clang driver has no direct correspondent for - "specs". The majority of the functionality that is - embedded in specs is in the Tool specific argument - translation routines. The parts of specs which control the - compilation pipeline are generally part of - the <i>Pipeline</i> stage.</p> - </li> - - <li> - <b>Toolchains</b> - <p>The gcc driver has no direct understanding of tool - chains. Each gcc binary roughly corresponds to the - information which is embedded inside a single - ToolChain.</p> - - <p>The clang driver is intended to be portable and support - complex compilation environments. All platform and tool - chain specific code should be protected behind either - abstract or well defined interfaces (such as whether the - platform supports use as a driver driver).</p> - </li> - </ul> - </div> - </body> -</html> |
