summaryrefslogtreecommitdiffstats
path: root/packages/Python/lldbsuite/test/README-TestSuite
diff options
context:
space:
mode:
Diffstat (limited to 'packages/Python/lldbsuite/test/README-TestSuite')
-rw-r--r--packages/Python/lldbsuite/test/README-TestSuite159
1 files changed, 159 insertions, 0 deletions
diff --git a/packages/Python/lldbsuite/test/README-TestSuite b/packages/Python/lldbsuite/test/README-TestSuite
new file mode 100644
index 0000000..6df4d7b
--- /dev/null
+++ b/packages/Python/lldbsuite/test/README-TestSuite
@@ -0,0 +1,159 @@
+This README file describes the files and directories related to the Python test
+suite under the current 'test' directory.
+
+o dotest.py
+
+ Provides the test driver for the test suite. To invoke it, cd to the 'test'
+ directory and issue the './dotest.py' command or './dotest.py -v' for more
+ verbose output. '.dotest.py -h' prints out the help messge.
+
+ A specific naming pattern is followed by the .py script under the 'test'
+ directory in order to be recognized by 'dotest.py' test driver as a module
+ which implements a test case, namely, Test*.py.
+
+ Some example usages:
+
+ 1. ./dotest.py -v . 2> ~/Developer/Log/lldbtest.log0
+ This runs the test suite and directs the run log to a file.
+
+ 2. LLDB_LOG=/tmp/lldb.log GDB_REMOTE_LOG=/tmp/gdb-remote.log ./dotest.py -v . 2> ~/Developer/Log/lldbtest.log
+ This runs the test suite, with logging turned on for the lldb as well as
+ the process.gdb-remote channels and directs the run log to a file.
+
+o lldbtest.py
+
+ Provides an abstract base class of lldb test case named 'TestBase', which in
+ turn inherits from Python's unittest.TestCase. The concrete subclass can
+ override lldbtest.TestBase in order to inherit the common behavior for
+ unittest.TestCase.setUp/tearDown implemented in this file.
+
+ To provide a test case, the concrete subclass provides methods whose names
+ start with the letters test. For more details about the Python's unittest
+ framework, go to http://docs.python.org/library/unittest.html.
+
+ ./command_source/TestCommandSource.py provides a simple example of test case
+ which overrides lldbtest.TestBase to exercise the lldb's 'command source'
+ command. The subclass should override the attribute 'mydir' in order for the
+ runtime to locate the individual test cases when running as part of a large
+ test suite or when running each test case as a separate Python invocation.
+
+ The doc string provides more details about the setup required for running a
+ test case on its own. To run the whole test suite, 'dotest.py' is all you
+ need to do.
+
+o subdirectories of 'test'
+
+ Most of them predate the introduction of the python test suite and contain
+ example C/C++/ObjC source files which get compiled into executables which are
+ to be exercised by the debugger.
+
+ For such subdirectory which has an associated Test*.py file, it was added as
+ part of the Python-based test suite to test lldb functionality.
+
+ Some of the subdirectories, for example, the 'help' subdirectory, do not have
+ C/C++/ObjC source files; they were created to house the Python test case which
+ does not involve lldb reading in an executable file at all.
+
+o make directory
+
+ Contains Makefile.rules, which can be utilized by test cases to write Makefile
+ based rules to build binaries for the inferiors.
+
+ By default, the built executable name is a.out, which can be overwritten by
+ specifying your EXE make variable, via the Makefile under the specific test
+ directory or via supplying a Python dictionary to the build method in your
+ Python test script. An example of the latter can be found in
+ test/lang/objc/radar-9691614/TestObjCMethodReturningBOOL.py, where:
+
+ def test_method_ret_BOOL_with_dsym(self):
+ """Test that objective-c method returning BOOL works correctly."""
+ d = {'EXE': self.exe_name}
+ self.buildDsym(dictionary=d)
+ self.setTearDownCleanup(dictionary=d)
+ self.objc_method_ret_BOOL(self.exe_name)
+
+ def test_method_ret_BOOL_with_dwarf(self):
+ """Test that objective-c method returning BOOL works correctly."""
+ d = {'EXE': self.exe_name}
+ self.buildDwarf(dictionary=d)
+ self.setTearDownCleanup(dictionary=d)
+ self.objc_method_ret_BOOL(self.exe_name)
+
+ def setUp(self):
+ # Call super's setUp().
+ TestBase.setUp(self)
+ # We'll use the test method name as the exe_name.
+ self.exe_name = self.testMethodName
+ # Find the line number to break inside main().
+ self.main_source = "main.m"
+ self.line = line_number(self.main_source, '// Set breakpoint here.')
+
+ The exe names for the two test methods are equal to the test method names and
+ are therefore guaranteed different.
+
+o plugins directory
+
+ Contains platform specific plugin to build binaries with dsym/dwarf debugging
+ info. Other platform specific functionalities may be added in the future.
+
+o unittest2 directory
+
+ Many new features were added to unittest in Python 2.7, including test
+ discovery. unittest2 allows you to use these features with earlier versions of
+ Python.
+
+ It currently has unittest2 0.5.1 from http://pypi.python.org/pypi/unittest2.
+ Version 0.5.1 of unittest2 has feature parity with unittest in Python 2.7
+ final. If you want to ensure that your tests run identically under unittest2
+ and unittest in Python 2.7 you should use unittest2 0.5.1.
+
+ Later versions of unittest2 include changes in unittest made in Python 3.2 and
+ onwards after the release of Python 2.7.
+
+o dotest.pl
+
+ In case you wonder, there is also a 'dotest.pl' perl script file. It was
+ created to visit each Python test case under the specified directory and
+ invoke Python's builtin unittest.main() on each test case.
+
+ It does not take advantage of the test runner and test suite functionality
+ provided by Python's unitest framework. Its existence is because we want a
+ different way of running the whole test suite. As lldb and the Python test
+ suite become more reliable, we don't expect to be using 'dotest.pl' anymore.
+
+ Note: dotest.pl has been moved to the attic directory.
+
+o Profiling dotest.py runs
+
+ I used the following command line thingy to do the profiling on a SnowLeopard
+ machine:
+
+$ DOTEST_PROFILE=YES DOTEST_SCRIPT_DIR=/Volumes/data/lldb/svn/trunk/test /System/Library/Frameworks/Python.framework/Versions/Current/lib/python2.6/cProfile.py -o my.profile ./dotest.py -v -w 2> ~/Developer/Log/lldbtest.log
+
+ After that, I used the pstats.py module to browse the statistics:
+
+$ python /System/Library/Frameworks/Python.framework/Versions/Current/lib/python2.6/pstats.py my.profile
+
+o Writing test cases:
+
+ We strongly prefer writing test cases using the SB API's rather than the runCmd & expect.
+ Unless you are actually testing some feature of the command line, please don't write
+ command based tests. For historical reasons there are plenty of examples of tests in the
+ test suite that use runCmd where they shouldn't, but don't copy them, copy the plenty that
+ do use the SB API's instead.
+
+ The reason for this is that our policy is that we will maintain compatibility with the
+ SB API's. But we don't make any similar guarantee about the details of command result format.
+ If your test is using the command line, it is going to have to check against the command result
+ text, and you either end up writing your check pattern by checking as little as possible so
+ you won't be exposed to random changes in the text; in which case you can end up missing some
+ failure, or you test too much and it means irrelevant changes break your tests.
+
+ However, if you use the Python API's it is possible to check all the results you want
+ to check in a very explicit way, which makes the tests much more robust.
+
+ Even if you are testing that a command-line command does some specific thing, it is still
+ better in general to use the SB API's to drive to the point where you want to run the test,
+ then use SBInterpreter::HandleCommand to run the command. You get the full result text
+ from the command in the command return object, and all the part where you are driving the
+ debugger to the point you want to test will be more robust.
OpenPOWER on IntegriCloud