update documenation
This commit is contained in:
parent
c025d22a14
commit
53b4869de9
181
doc/cmdtest.html
181
doc/cmdtest.html
@ -319,6 +319,7 @@ ul.auto-toc {
|
||||
<li><a class="reference internal" href="#assertions-stdout-stderr-file-content" id="id17">Assertions - stdout/stderr/file content</a></li>
|
||||
<li><a class="reference internal" href="#assertions-misc" id="id18">Assertions - misc</a></li>
|
||||
<li><a class="reference internal" href="#helper-functions" id="id19">Helper functions</a></li>
|
||||
<li><a class="reference internal" href="#deprecated-helper-functions" id="id20">Deprecated helper functions</a></li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
@ -327,7 +328,7 @@ ul.auto-toc {
|
||||
<h1><a class="toc-backref" href="#id1">Introduction</a></h1>
|
||||
<p>Cmdtest is a <a class="reference external" href="http://en.wikipedia.org/wiki/Unit_testing">unit testing</a> framework for testing commands (executable programs).
|
||||
In other test frameworks the "unit" tested is often a class (e.g. in Java's <a class="reference external" href="http://en.wikipedia.org/wiki/JUnit">JUnit</a> or
|
||||
Ruby's <a class="reference external" href="http://www.ruby-doc.org/stdlib/libdoc/test/unit/rdoc/classes/Test/Unit.html">Test::Unit</a>), but in Cmdtest the unit is an executable. Apart from this
|
||||
Ruby's <a class="reference external" href="https://github.com/test-unit/test-unit">Test::Unit</a>), but in Cmdtest the unit is an executable. Apart from this
|
||||
difference Cmdtest borrows many ideas from the other frameworks.
|
||||
The program <tt class="docutils literal">cmdtest</tt> runs the tests and reports the success or failure
|
||||
in different ways, e.g. by writing to standard output or producing an XML-file on
|
||||
@ -347,11 +348,17 @@ filesystem</li>
|
||||
$ cat CMDTEST_example.rb
|
||||
class CMDTEST_example < Cmdtest::Testcase
|
||||
|
||||
def test_misc
|
||||
cmd "echo hello world" do
|
||||
stdout_equal "hello world\n"
|
||||
def test_hello_world
|
||||
cmd "echo hello" do
|
||||
stdout_equal "hello\n"
|
||||
end
|
||||
|
||||
cmd "echo world" do
|
||||
stdout_equal "world\n"
|
||||
end
|
||||
end
|
||||
|
||||
def test_touch_and_exit
|
||||
cmd "touch foo.txt ; exit 7" do
|
||||
created_files "foo.txt"
|
||||
exit_status 7
|
||||
@ -367,31 +374,41 @@ Inside a method we can call the <tt class="docutils literal">cmd</tt> method. It
|
||||
execute the command given as argument and then check the assertions
|
||||
given in the do-block. When <tt class="docutils literal">cmdtest</tt> is run, it will find all
|
||||
<tt class="docutils literal"><span class="pre">CMDTEST_*.rb</span></tt> files in the current directory and run the tests in
|
||||
the files. The output looks like:</p>
|
||||
the files (see the <a class="reference internal" href="#invoking-cmdtest">Invoking cmdtest</a> section for more details
|
||||
on <tt class="docutils literal"><span class="pre">CMDTEST_*.rb</span></tt> file selection).
|
||||
The output looks like:</p>
|
||||
<pre class="literal-block">
|
||||
$ cmdtest
|
||||
### ======================================== CMDTEST_example.rb
|
||||
### echo hello world
|
||||
### ---------------------------------------- CMDTEST_example
|
||||
### ........................................ test_hello_world
|
||||
### echo hello
|
||||
### echo world
|
||||
### ........................................ test_touch_and_exit
|
||||
### touch foo.txt ; exit 7
|
||||
|
||||
### 1 test classes, 1 test methods, 2 commands, 0 errors.
|
||||
### 1 test classes, 2 test methods, 3 commands, 0 errors, 0 fatals.
|
||||
</pre>
|
||||
<p>If we change "7" to "8", "foo" to "bar" and "world" to "WORLD" in
|
||||
the example, we get the following errors:</p>
|
||||
<pre class="literal-block">
|
||||
$ cmdtest
|
||||
### ======================================== CMDTEST_example.rb
|
||||
### echo hello WORLD
|
||||
### ---------------------------------------- CMDTEST_example
|
||||
### ........................................ test_hello_world
|
||||
### echo hello
|
||||
### echo WORLD
|
||||
--- ERROR: wrong stdout
|
||||
--- actual: hello WORLD
|
||||
--- expect: hello world
|
||||
--- actual: WORLD
|
||||
--- expect: world
|
||||
### ........................................ test_touch_and_exit
|
||||
### touch bar.txt ; exit 8
|
||||
--- ERROR: created files
|
||||
--- actual: ["bar.txt"]
|
||||
--- expect: ["foo.txt"]
|
||||
--- ERROR: expected 7 exit status, got 8
|
||||
|
||||
--- 1 test classes, 1 test methods, 2 commands, 2 errors.
|
||||
--- 1 test classes, 2 test methods, 3 commands, 2 errors, 0 fatals.
|
||||
</pre>
|
||||
<p>The following sections will describe in more detail what can be done
|
||||
with Cmdtest. See also the <a class="reference external" href="../examples">examples directory</a> of the Cmdtest project,
|
||||
@ -491,12 +508,12 @@ will be the "work directory" (unless <tt class="docutils literal">chdi
|
||||
<p>Several of the assertions and helper functions take filename arguments
|
||||
that are evaluated relative to the "work directory" (or sometimes the
|
||||
current directory if they differ).</p>
|
||||
<p>To support running <tt class="docutils literal">cmdtest</tt> in parallel, methods such as
|
||||
<tt class="docutils literal">File.open</tt> and <tt class="docutils literal">Dir.chdir</tt> should not be used in the test
|
||||
methods. Instead methods supplied by Cmdtest, such as <tt class="docutils literal">create_file</tt>
|
||||
and <tt class="docutils literal">chdir</tt> should be used. The reason for this is that the <tt class="docutils literal">cmdtest</tt> process
|
||||
only has <strong>one</strong> current directory, so different threads can't set the
|
||||
current directory without interfering with each other.</p>
|
||||
<p>Cmdtest implements parallel execution of test methods by running several
|
||||
"slave processes", started by a tool like <a class="reference external" href="http://www.gnu.org/software/parallel/">GNU Parallel</a>.</p>
|
||||
<p>Methods such as <tt class="docutils literal">File.open</tt> and <tt class="docutils literal">Dir.chdir</tt> that depend on the
|
||||
"current directory" can be used in the test methods, since each slave
|
||||
is a process of its own (an earlier version of Cmdtest used Ruby threads
|
||||
and adviced against using such methods).</p>
|
||||
</div>
|
||||
<div class="section" id="specifying-files-directories">
|
||||
<h1><a class="toc-backref" href="#id7">Specifying files / directories</a></h1>
|
||||
@ -505,8 +522,8 @@ current directory without interfering with each other.</p>
|
||||
having two sets of methods, one for files and one for directories, an
|
||||
argument with a trailing "/" denotes a directory:</p>
|
||||
<pre class="literal-block">
|
||||
create_files "build/" # the directory "build"
|
||||
create_files "build" # the file "build"
|
||||
created_files "build/" # the directory "build"
|
||||
created_files "build" # the file "build"
|
||||
|
||||
ignore_file "build/" # the directory "build" (and everything below)
|
||||
ignore_file "build" # the file "build"
|
||||
@ -602,42 +619,30 @@ $ cmdtest examples /stdin/ /stdout/
|
||||
<p>For more examples of command line usage, see the section <a class="reference internal" href="#commandline-examples">Commandline Examples</a> below.</p>
|
||||
<div class="section" id="options">
|
||||
<h2><a class="toc-backref" href="#id11">Options</a></h2>
|
||||
<table class="docutils option-list" frame="void" rules="none">
|
||||
<col class="option" />
|
||||
<col class="description" />
|
||||
<tbody valign="top">
|
||||
<tr><td class="option-group">
|
||||
<kbd><span class="option">--help</span></kbd></td>
|
||||
<td>Show available options.</td></tr>
|
||||
<tr><td class="option-group">
|
||||
<kbd><span class="option">--quiet</span></kbd></td>
|
||||
<td>Be more quiet by skipping some non-essential output.</td></tr>
|
||||
<tr><td class="option-group">
|
||||
<kbd><span class="option">--verbose</span></kbd></td>
|
||||
<td>Be more verbose.</td></tr>
|
||||
<tr><td class="option-group">
|
||||
<kbd><span class="option">--fast</span></kbd></td>
|
||||
<td>Run fast without ensuring that timestamps of newly created/modified
|
||||
files are unique. This could make it impossible for Cmdtest to detect
|
||||
all side-effects of commands.</td></tr>
|
||||
<tr><td class="option-group">
|
||||
<kbd><span class="option">--test=<var>NAME</var></span></kbd></td>
|
||||
<td>Only run named test method.</td></tr>
|
||||
<tr><td class="option-group">
|
||||
<kbd><span class="option">--xml=<var>FILE</var></span></kbd></td>
|
||||
<td>Write summary on JUnit XML format.
|
||||
Useful when running under a continuous integration server that
|
||||
understands JUnit reports.</td></tr>
|
||||
<tr><td class="option-group">
|
||||
<kbd><span class="option">--no-exit-code</span></kbd></td>
|
||||
<td>Do not exit with a non-zero exit code if errors occurred.</td></tr>
|
||||
<tr><td class="option-group">
|
||||
<kbd><span class="option">-i</span></kbd></td>
|
||||
<td>Run in "incremental" mode. <strong>experimental</strong>
|
||||
Cmdtest will try to run only those test methods that are failed or
|
||||
have changed since last time.</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<p>The available options can be seen by using the <tt class="docutils literal"><span class="pre">-h</span></tt> option:</p>
|
||||
<pre class="literal-block">
|
||||
$ cmdtest -h
|
||||
usage: cmdtest [-h] [--version] [-q] [-v] [--fast] [-j N] [--test TEST]
|
||||
[--xml FILE] [--no-exit-code] [-i] [--slave SLAVE]
|
||||
[arg [arg ...]]
|
||||
|
||||
positional arguments:
|
||||
arg testfile or pattern
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
-h, --help show this help message and exit
|
||||
--version show version
|
||||
-q, --quiet be more quiet by skipping some non-essential output
|
||||
-v, --verbose be more verbose
|
||||
--fast run fast without waiting for unique mtime:s
|
||||
-j N, --parallel N build in parallel
|
||||
--test TEST only run named test
|
||||
--xml FILE write summary on JUnit format
|
||||
--no-exit-code exit with 0 status even after errors
|
||||
-i, --incremental incremental mode
|
||||
--slave SLAVE run in slave mode
|
||||
</pre>
|
||||
</div>
|
||||
<div class="section" id="commandline-examples">
|
||||
<h2><a class="toc-backref" href="#id12">Commandline Examples</a></h2>
|
||||
@ -701,6 +706,7 @@ end
|
||||
</div>
|
||||
<div class="section" id="assertions-exit-status">
|
||||
<h2><a class="toc-backref" href="#id15">Assertions - exit status</a></h2>
|
||||
<p>These methods should only be used inside a <tt class="docutils literal">cmd</tt> block.</p>
|
||||
<dl class="docutils">
|
||||
<dt><tt class="docutils literal">exit_nonzero</tt></dt>
|
||||
<dd>The command should have exited with a non-zero exit status (i.e. it
|
||||
@ -715,14 +721,13 @@ exit-related methods have been called.</dd>
|
||||
</div>
|
||||
<div class="section" id="assertions-files">
|
||||
<h2><a class="toc-backref" href="#id16">Assertions - files</a></h2>
|
||||
<p>These methods should only be used inside a <tt class="docutils literal">cmd</tt> block.</p>
|
||||
<dl class="docutils">
|
||||
<dt><tt class="docutils literal"><span class="pre">affected_files(file1,...,fileN)</span></tt></dt>
|
||||
<dd>The specified files should have been created, removed or modified by the
|
||||
command. This assertion can be used when it doesn't matter which
|
||||
of <tt class="docutils literal">created_files</tt>, <tt class="docutils literal">removed_files</tt> or <tt class="docutils literal">changed_files</tt> that apply
|
||||
(cf. <tt class="docutils literal">written_files</tt>).</dd>
|
||||
<dt><tt class="docutils literal"><span class="pre">changed_files(file1,...,fileN)</span></tt></dt>
|
||||
<dd>Deprecated. Use <tt class="docutils literal">modified_files</tt> instead.</dd>
|
||||
<dt><tt class="docutils literal"><span class="pre">created_files(file1,...,fileN)</span></tt></dt>
|
||||
<dd>The specified files should have been created by the command.</dd>
|
||||
<dt><tt class="docutils literal"><span class="pre">modified_files(file1,...,fileN)</span></tt></dt>
|
||||
@ -743,6 +748,7 @@ case special (when the file is created).</dd>
|
||||
</div>
|
||||
<div class="section" id="assertions-stdout-stderr-file-content">
|
||||
<h2><a class="toc-backref" href="#id17">Assertions - stdout/stderr/file content</a></h2>
|
||||
<p>These methods should only be used inside a <tt class="docutils literal">cmd</tt> block.</p>
|
||||
<dl class="docutils">
|
||||
<dt><tt class="docutils literal">file_equal(file, content)</tt></dt>
|
||||
<dd>Assert that the specified file matches the given content.
|
||||
@ -760,13 +766,14 @@ The content can be given in several different forms: 1) as a
|
||||
string that should be equal to the entire file, 2) as an array of
|
||||
lines that should be equal to the entire file, 3) as a regexp that
|
||||
should match the entire file (given as one string).
|
||||
For more details and examples see the section "Matching standard output content".</dd>
|
||||
For more details and examples see the section <a class="reference internal" href="#matching-standard-output-content">Matching standard output content</a>.</dd>
|
||||
<dt><tt class="docutils literal">stdout_not_equal(content)</tt></dt>
|
||||
<dd>Like <tt class="docutils literal">stdout_equal</tt> but with inverted test.</dd>
|
||||
</dl>
|
||||
</div>
|
||||
<div class="section" id="assertions-misc">
|
||||
<h2><a class="toc-backref" href="#id18">Assertions - misc</a></h2>
|
||||
<p>These methods should only be used inside a <tt class="docutils literal">cmd</tt> block.</p>
|
||||
<dl class="docutils">
|
||||
<dt><tt class="docutils literal">assert(flag, msg=nil)</tt></dt>
|
||||
<dd>Assert that <tt class="docutils literal">flag</tt> is true. This assertion is a last resort, when no other
|
||||
@ -778,10 +785,9 @@ interval given as argument.</dd>
|
||||
</div>
|
||||
<div class="section" id="helper-functions">
|
||||
<h2><a class="toc-backref" href="#id19">Helper functions</a></h2>
|
||||
<p>These methods should only be used outside a <tt class="docutils literal">cmd</tt> block in a test method,
|
||||
or in the <tt class="docutils literal">setup</tt> method.</p>
|
||||
<dl class="docutils">
|
||||
<dt><tt class="docutils literal">chdir(dir, &block)</tt></dt>
|
||||
<dd>Works like <tt class="docutils literal">Dir.chdir</tt>, but handles current directory correctly
|
||||
under <tt class="docutils literal">cmdtest</tt>.</dd>
|
||||
<dt><tt class="docutils literal">create_file(filename, content)</tt></dt>
|
||||
<dd>Create a file inside the "work directory".
|
||||
If the filename contains a directory part, intermediate directories are
|
||||
@ -790,24 +796,6 @@ The content can be specified either as an array of lines or as
|
||||
a string with the content of the whole file.
|
||||
The filename is evaluated relative to the current directory at the
|
||||
time of the call.</dd>
|
||||
<dt><tt class="docutils literal">dir_mkdir(file)</tt></dt>
|
||||
<dd>Works like <tt class="docutils literal">Dir.mkdir</tt>, but handles current directory correctly
|
||||
under <tt class="docutils literal">cmdtest</tt>.</dd>
|
||||
<dt><tt class="docutils literal">file_chmod(arg, file)</tt></dt>
|
||||
<dd>Works like <tt class="docutils literal">File.chmod</tt>, but handles current directory correctly
|
||||
under <tt class="docutils literal">cmdtest</tt>.</dd>
|
||||
<dt><tt class="docutils literal">file_open(file, *args, &block)</tt></dt>
|
||||
<dd>Works like <tt class="docutils literal">File.open</tt>, but handles current directory correctly
|
||||
under <tt class="docutils literal">cmdtest</tt>.</dd>
|
||||
<dt><tt class="docutils literal">file_read(file)</tt></dt>
|
||||
<dd>Works like <tt class="docutils literal">File.read</tt>, but handles current directory correctly
|
||||
under <tt class="docutils literal">cmdtest</tt>.</dd>
|
||||
<dt><tt class="docutils literal">file_symlink(file1, file2)</tt></dt>
|
||||
<dd>Works like <tt class="docutils literal">File.symlink</tt>, but handles current directory correctly
|
||||
under <tt class="docutils literal">cmdtest</tt>.</dd>
|
||||
<dt><tt class="docutils literal">file_utime(arg1, arg2, file)</tt></dt>
|
||||
<dd>Works like <tt class="docutils literal">File.utime</tt>, but handles current directory correctly
|
||||
under <tt class="docutils literal">cmdtest</tt>.</dd>
|
||||
<dt><tt class="docutils literal">ignore_file(file)</tt></dt>
|
||||
<dd>Ignore the specified file when looking for differences in the filesystem.
|
||||
A subdirectory can be ignored by giving a trailing "/" to the name.</dd>
|
||||
@ -829,12 +817,12 @@ relative to the current directory in effect at the time of the call
|
||||
are looked up using the modified <tt class="docutils literal">PATH</tt>. A typical use is to add the directory
|
||||
where the executable tested is located. The argument <tt class="docutils literal">dir</tt> is evaluated
|
||||
relative to the current directory in effect when <tt class="docutils literal">cmdtest</tt> was invoked.</dd>
|
||||
<dt><tt class="docutils literal">remove_file(file)</tt></dt>
|
||||
<dd>Works like <tt class="docutils literal">FileUtils.rm_f</tt>, but handles current directory correctly
|
||||
under <tt class="docutils literal">cmdtest</tt>.</dd>
|
||||
<dt><tt class="docutils literal">remove_file_tree(file)</tt></dt>
|
||||
<dd>Works like <tt class="docutils literal">FileUtils.rm_rf</tt>, but handles current directory correctly
|
||||
under <tt class="docutils literal">cmdtest</tt>.</dd>
|
||||
<dt><tt class="docutils literal">setenv(name, value)</tt></dt>
|
||||
<dd>Set an environment variable that should be in effect when commands are executed
|
||||
by later calls to <tt class="docutils literal">cmd</tt>.</dd>
|
||||
<dt><tt class="docutils literal">unsetenv(name)</tt></dt>
|
||||
<dd>Unset an environment variable that should not be in effect when commands are executed
|
||||
by later calls to <tt class="docutils literal">cmd</tt>.</dd>
|
||||
<dt><tt class="docutils literal">set_path(dir1, <span class="pre">...,</span> dirN)</tt></dt>
|
||||
<dd>Set <tt class="docutils literal">PATH</tt> to the given directories, so commands executed via <tt class="docutils literal">cmd</tt>
|
||||
are looked up using the modified <tt class="docutils literal">PATH</tt>. This method sets the whole <tt class="docutils literal">PATH</tt>
|
||||
@ -845,12 +833,33 @@ The filename is evaluated relative to the current directory at the
|
||||
time of the call.</dd>
|
||||
</dl>
|
||||
</div>
|
||||
<div class="section" id="deprecated-helper-functions">
|
||||
<h2><a class="toc-backref" href="#id20">Deprecated helper functions</a></h2>
|
||||
<dl class="docutils">
|
||||
<dt><tt class="docutils literal">dir_mkdir(file)</tt></dt>
|
||||
<dd>Deprecated, use <tt class="docutils literal">Dir.mkdir</tt> instead.</dd>
|
||||
<dt><tt class="docutils literal">file_chmod(arg, file)</tt></dt>
|
||||
<dd>Deprecated, use <tt class="docutils literal">File.chmod</tt> instead.</dd>
|
||||
<dt><tt class="docutils literal">file_open(file, *args, &block)</tt></dt>
|
||||
<dd>Deprecated, use <tt class="docutils literal">File.open</tt> instead.</dd>
|
||||
<dt><tt class="docutils literal">file_read(file)</tt></dt>
|
||||
<dd>Deprecated, use <tt class="docutils literal">File.read</tt> instead.</dd>
|
||||
<dt><tt class="docutils literal">file_symlink(file1, file2)</tt></dt>
|
||||
<dd>Deprecated, use <tt class="docutils literal">File.symlink</tt> instead.</dd>
|
||||
<dt><tt class="docutils literal">file_utime(arg1, arg2, file)</tt></dt>
|
||||
<dd>Deprecated, use <tt class="docutils literal">File.utime</tt> instead.</dd>
|
||||
<dt><tt class="docutils literal">remove_file(file)</tt></dt>
|
||||
<dd>Deprecated, use <tt class="docutils literal">FileUtils.rm_f</tt> instead.</dd>
|
||||
<dt><tt class="docutils literal">remove_file_tree(file)</tt></dt>
|
||||
<dd>Deprecated, use <tt class="docutils literal">FileUtils.rm_rf</tt> instead.</dd>
|
||||
</dl>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="footer">
|
||||
<hr class="footer" />
|
||||
<a class="reference external" href="cmdtest.txt">View document source</a>.
|
||||
Generated on: 2014-02-16.
|
||||
Generated on: 2015-08-26.
|
||||
Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source.
|
||||
|
||||
</div>
|
||||
|
188
doc/cmdtest.txt
188
doc/cmdtest.txt
@ -33,11 +33,17 @@ A simple example
|
||||
$ cat CMDTEST_example.rb
|
||||
class CMDTEST_example < Cmdtest::Testcase
|
||||
|
||||
def test_misc
|
||||
cmd "echo hello world" do
|
||||
stdout_equal "hello world\n"
|
||||
def test_hello_world
|
||||
cmd "echo hello" do
|
||||
stdout_equal "hello\n"
|
||||
end
|
||||
|
||||
cmd "echo world" do
|
||||
stdout_equal "world\n"
|
||||
end
|
||||
end
|
||||
|
||||
def test_touch_and_exit
|
||||
cmd "touch foo.txt ; exit 7" do
|
||||
created_files "foo.txt"
|
||||
exit_status 7
|
||||
@ -53,31 +59,41 @@ Inside a method we can call the ``cmd`` method. It will
|
||||
execute the command given as argument and then check the assertions
|
||||
given in the do-block. When ``cmdtest`` is run, it will find all
|
||||
``CMDTEST_*.rb`` files in the current directory and run the tests in
|
||||
the files. The output looks like::
|
||||
the files (see the `Invoking cmdtest`_ section for more details
|
||||
on ``CMDTEST_*.rb`` file selection).
|
||||
The output looks like::
|
||||
|
||||
$ cmdtest
|
||||
### ======================================== CMDTEST_example.rb
|
||||
### echo hello world
|
||||
### ---------------------------------------- CMDTEST_example
|
||||
### ........................................ test_hello_world
|
||||
### echo hello
|
||||
### echo world
|
||||
### ........................................ test_touch_and_exit
|
||||
### touch foo.txt ; exit 7
|
||||
|
||||
### 1 test classes, 1 test methods, 2 commands, 0 errors.
|
||||
### 1 test classes, 2 test methods, 3 commands, 0 errors, 0 fatals.
|
||||
|
||||
If we change "7" to "8", "foo" to "bar" and "world" to "WORLD" in
|
||||
the example, we get the following errors::
|
||||
|
||||
$ cmdtest
|
||||
### ======================================== CMDTEST_example.rb
|
||||
### echo hello WORLD
|
||||
### ---------------------------------------- CMDTEST_example
|
||||
### ........................................ test_hello_world
|
||||
### echo hello
|
||||
### echo WORLD
|
||||
--- ERROR: wrong stdout
|
||||
--- actual: hello WORLD
|
||||
--- expect: hello world
|
||||
--- actual: WORLD
|
||||
--- expect: world
|
||||
### ........................................ test_touch_and_exit
|
||||
### touch bar.txt ; exit 8
|
||||
--- ERROR: created files
|
||||
--- actual: ["bar.txt"]
|
||||
--- expect: ["foo.txt"]
|
||||
--- ERROR: expected 7 exit status, got 8
|
||||
|
||||
--- 1 test classes, 1 test methods, 2 commands, 2 errors.
|
||||
--- 1 test classes, 2 test methods, 3 commands, 2 errors, 0 fatals.
|
||||
|
||||
The following sections will describe in more detail what can be done
|
||||
with Cmdtest. See also the `examples directory <../examples>`_ of the Cmdtest project,
|
||||
@ -189,12 +205,13 @@ Several of the assertions and helper functions take filename arguments
|
||||
that are evaluated relative to the "work directory" (or sometimes the
|
||||
current directory if they differ).
|
||||
|
||||
To support running ``cmdtest`` in parallel, methods such as
|
||||
``File.open`` and ``Dir.chdir`` should not be used in the test
|
||||
methods. Instead methods supplied by Cmdtest, such as ``create_file``
|
||||
and ``chdir`` should be used. The reason for this is that the ``cmdtest`` process
|
||||
only has **one** current directory, so different threads can't set the
|
||||
current directory without interfering with each other.
|
||||
Cmdtest implements parallel execution of test methods by running several
|
||||
"slave processes", started by a tool like `GNU Parallel`_.
|
||||
|
||||
Methods such as ``File.open`` and ``Dir.chdir`` that depend on the
|
||||
"current directory" can be used in the test methods, since each slave
|
||||
is a process of its own (an earlier version of Cmdtest used Ruby threads
|
||||
and adviced against using such methods).
|
||||
|
||||
Specifying files / directories
|
||||
------------------------------
|
||||
@ -204,8 +221,8 @@ Several methods take files or directories as argument (e.g.
|
||||
having two sets of methods, one for files and one for directories, an
|
||||
argument with a trailing "/" denotes a directory::
|
||||
|
||||
create_files "build/" # the directory "build"
|
||||
create_files "build" # the file "build"
|
||||
created_files "build/" # the directory "build"
|
||||
created_files "build" # the file "build"
|
||||
|
||||
ignore_file "build/" # the directory "build" (and everything below)
|
||||
ignore_file "build" # the file "build"
|
||||
@ -219,7 +236,7 @@ argument can be a Regexp::
|
||||
|
||||
This is quite natural, since the "job" of ``ignore_file`` is to single
|
||||
out a subset of all files.
|
||||
|
||||
|
||||
PATH handling
|
||||
-------------
|
||||
|
||||
@ -311,36 +328,29 @@ For more examples of command line usage, see the section `Commandline Examples`_
|
||||
Options
|
||||
+++++++
|
||||
|
||||
--help
|
||||
Show available options.
|
||||
The available options can be seen by using the ``-h`` option::
|
||||
|
||||
--quiet
|
||||
Be more quiet by skipping some non-essential output.
|
||||
$ cmdtest -h
|
||||
usage: cmdtest [-h] [--version] [-q] [-v] [--fast] [-j N] [--test TEST]
|
||||
[--xml FILE] [--no-exit-code] [-i] [--slave SLAVE]
|
||||
[arg [arg ...]]
|
||||
|
||||
--verbose
|
||||
Be more verbose.
|
||||
|
||||
--fast
|
||||
Run fast without ensuring that timestamps of newly created/modified
|
||||
files are unique. This could make it impossible for Cmdtest to detect
|
||||
all side-effects of commands.
|
||||
|
||||
--test=NAME
|
||||
Only run named test method.
|
||||
|
||||
--xml=FILE
|
||||
Write summary on JUnit XML format.
|
||||
Useful when running under a continuous integration server that
|
||||
understands JUnit reports.
|
||||
|
||||
--no-exit-code
|
||||
Do not exit with a non-zero exit code if errors occurred.
|
||||
|
||||
-i
|
||||
Run in "incremental" mode. **experimental**
|
||||
Cmdtest will try to run only those test methods that are failed or
|
||||
have changed since last time.
|
||||
positional arguments:
|
||||
arg testfile or pattern
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
-h, --help show this help message and exit
|
||||
--version show version
|
||||
-q, --quiet be more quiet by skipping some non-essential output
|
||||
-v, --verbose be more verbose
|
||||
--fast run fast without waiting for unique mtime:s
|
||||
-j N, --parallel N build in parallel
|
||||
--test TEST only run named test
|
||||
--xml FILE write summary on JUnit format
|
||||
--no-exit-code exit with 0 status even after errors
|
||||
-i, --incremental incremental mode
|
||||
--slave SLAVE run in slave mode
|
||||
|
||||
Commandline Examples
|
||||
++++++++++++++++++++
|
||||
@ -408,6 +418,9 @@ See also the example in the `Structure of a test-method`_ section above.
|
||||
Assertions - exit status
|
||||
++++++++++++++++++++++++
|
||||
|
||||
These methods should only be used inside a ``cmd`` block.
|
||||
|
||||
|
||||
``exit_nonzero``
|
||||
The command should have exited with a non-zero exit status (i.e. it
|
||||
should have failed).
|
||||
@ -423,15 +436,14 @@ Assertions - exit status
|
||||
Assertions - files
|
||||
++++++++++++++++++
|
||||
|
||||
These methods should only be used inside a ``cmd`` block.
|
||||
|
||||
``affected_files(file1,...,fileN)``
|
||||
The specified files should have been created, removed or modified by the
|
||||
command. This assertion can be used when it doesn't matter which
|
||||
of ``created_files``, ``removed_files`` or ``changed_files`` that apply
|
||||
(cf. ``written_files``).
|
||||
|
||||
``changed_files(file1,...,fileN)``
|
||||
Deprecated. Use ``modified_files`` instead.
|
||||
|
||||
``created_files(file1,...,fileN)``
|
||||
The specified files should have been created by the command.
|
||||
|
||||
@ -455,6 +467,8 @@ Assertions - files
|
||||
Assertions - stdout/stderr/file content
|
||||
+++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
These methods should only be used inside a ``cmd`` block.
|
||||
|
||||
``file_equal(file, content)``
|
||||
Assert that the specified file matches the given content.
|
||||
See "stdout_equal" for how "content" can be specified.
|
||||
@ -475,7 +489,7 @@ Assertions - stdout/stderr/file content
|
||||
string that should be equal to the entire file, 2) as an array of
|
||||
lines that should be equal to the entire file, 3) as a regexp that
|
||||
should match the entire file (given as one string).
|
||||
For more details and examples see the section "Matching standard output content".
|
||||
For more details and examples see the section `Matching standard output content`_.
|
||||
|
||||
``stdout_not_equal(content)``
|
||||
Like ``stdout_equal`` but with inverted test.
|
||||
@ -484,6 +498,8 @@ Assertions - stdout/stderr/file content
|
||||
Assertions - misc
|
||||
+++++++++++++++++
|
||||
|
||||
These methods should only be used inside a ``cmd`` block.
|
||||
|
||||
``assert(flag, msg=nil)``
|
||||
Assert that ``flag`` is true. This assertion is a last resort, when no other
|
||||
assertion fits. Should normally not be used.
|
||||
@ -495,9 +511,8 @@ Assertions - misc
|
||||
Helper functions
|
||||
++++++++++++++++
|
||||
|
||||
``chdir(dir, &block)``
|
||||
Works like ``Dir.chdir``, but handles current directory correctly
|
||||
under ``cmdtest``.
|
||||
These methods should only be used outside a ``cmd`` block in a test method,
|
||||
or in the ``setup`` method.
|
||||
|
||||
``create_file(filename, content)``
|
||||
Create a file inside the "work directory".
|
||||
@ -508,30 +523,6 @@ Helper functions
|
||||
The filename is evaluated relative to the current directory at the
|
||||
time of the call.
|
||||
|
||||
``dir_mkdir(file)``
|
||||
Works like ``Dir.mkdir``, but handles current directory correctly
|
||||
under ``cmdtest``.
|
||||
|
||||
``file_chmod(arg, file)``
|
||||
Works like ``File.chmod``, but handles current directory correctly
|
||||
under ``cmdtest``.
|
||||
|
||||
``file_open(file, *args, &block)``
|
||||
Works like ``File.open``, but handles current directory correctly
|
||||
under ``cmdtest``.
|
||||
|
||||
``file_read(file)``
|
||||
Works like ``File.read``, but handles current directory correctly
|
||||
under ``cmdtest``.
|
||||
|
||||
``file_symlink(file1, file2)``
|
||||
Works like ``File.symlink``, but handles current directory correctly
|
||||
under ``cmdtest``.
|
||||
|
||||
``file_utime(arg1, arg2, file)``
|
||||
Works like ``File.utime``, but handles current directory correctly
|
||||
under ``cmdtest``.
|
||||
|
||||
``ignore_file(file)``
|
||||
Ignore the specified file when looking for differences in the filesystem.
|
||||
A subdirectory can be ignored by giving a trailing "/" to the name.
|
||||
@ -558,13 +549,13 @@ Helper functions
|
||||
where the executable tested is located. The argument ``dir`` is evaluated
|
||||
relative to the current directory in effect when ``cmdtest`` was invoked.
|
||||
|
||||
``remove_file(file)``
|
||||
Works like ``FileUtils.rm_f``, but handles current directory correctly
|
||||
under ``cmdtest``.
|
||||
``setenv(name, value)``
|
||||
Set an environment variable that should be in effect when commands are executed
|
||||
by later calls to ``cmd``.
|
||||
|
||||
``remove_file_tree(file)``
|
||||
Works like ``FileUtils.rm_rf``, but handles current directory correctly
|
||||
under ``cmdtest``.
|
||||
``unsetenv(name)``
|
||||
Unset an environment variable that should not be in effect when commands are executed
|
||||
by later calls to ``cmd``.
|
||||
|
||||
``set_path(dir1, ..., dirN)``
|
||||
Set ``PATH`` to the given directories, so commands executed via ``cmd``
|
||||
@ -576,9 +567,38 @@ Helper functions
|
||||
The filename is evaluated relative to the current directory at the
|
||||
time of the call.
|
||||
|
||||
Deprecated helper functions
|
||||
+++++++++++++++++++++++++++
|
||||
|
||||
``dir_mkdir(file)``
|
||||
Deprecated, use ``Dir.mkdir`` instead.
|
||||
|
||||
``file_chmod(arg, file)``
|
||||
Deprecated, use ``File.chmod`` instead.
|
||||
|
||||
``file_open(file, *args, &block)``
|
||||
Deprecated, use ``File.open`` instead.
|
||||
|
||||
``file_read(file)``
|
||||
Deprecated, use ``File.read`` instead.
|
||||
|
||||
``file_symlink(file1, file2)``
|
||||
Deprecated, use ``File.symlink`` instead.
|
||||
|
||||
``file_utime(arg1, arg2, file)``
|
||||
Deprecated, use ``File.utime`` instead.
|
||||
|
||||
``remove_file(file)``
|
||||
Deprecated, use ``FileUtils.rm_f`` instead.
|
||||
|
||||
``remove_file_tree(file)``
|
||||
Deprecated, use ``FileUtils.rm_rf`` instead.
|
||||
|
||||
|
||||
.. _`unit testing`: http://en.wikipedia.org/wiki/Unit_testing
|
||||
.. _`junit`: http://en.wikipedia.org/wiki/JUnit
|
||||
.. _`Test::Unit`: http://www.ruby-doc.org/stdlib/libdoc/test/unit/rdoc/classes/Test/Unit.html
|
||||
.. _`Test::Unit`: https://github.com/test-unit/test-unit
|
||||
|
||||
.. _`continuous integration`: http://en.wikipedia.org/wiki/Continuous_integration
|
||||
.. _Jenkins: http://jenkins-ci.org
|
||||
.. _`GNU Parallel`: http://www.gnu.org/software/parallel/
|
||||
|
Loading…
x
Reference in New Issue
Block a user