|Return to UNITTEST.txt CVS log||Up to [Zope] / Zope3 / doc|
File: [Zope] / Zope3 / doc / UNITTEST.txt
Revision: 1.5, Tue Dec 31 21:49:21 2002 UTC (10 years, 5 months ago) by k_vertigo
CVS Tags: use-config-branch, uk-april-2003-usability-branch, template_usage-branch-merge1, template_usage-branch, stevea-decorators-merge-1, stevea-decorators-branch, stevea-contextaware_in_c-merge-1, stevea-contextaware_in_c-branch, schema-vocabulary-branch, pluggable_authentication_service-branch, pluggable_authentication_service-base, paris-copypasterename-branch-merge-1, paris-copypasterename-branch, ozzope-menu, merge-new-pickle-branch, local-utility-branch, interfacegeddon2-branch, interfacegeddon-branch, integration-2003-02-merge-point, fssync-fork, fssync-branch, fdrake-zconfig-in-zope-3-branch, fdrake-local-modules-branch, faassen-roleservice_confusion-branch, cw-mail-merge3, cw-mail-merge2, cw-mail-merge1, cw-mail-branch, cw-mail-base, ajung-keywordindex-branch, ZopeX3-0_0-m3, ZopeX3-0_0-m2, Zope3x-0_0-a1
Changes since 1.4: +4 -0 lines
Zope 3 Unit Testing
How to run Zope 3 unit tests
In the toplevel directory, run "python test.py". This runs all unit
tests silently (unless errors occur). It is possible to run the
tests more verbosely or to be more selective about which tests to
run. There are also other options. For information about all this,
run "python test.py -h" which prints an extensive usage message.
If you encounter a directory named "tests" in a package within
within the Zope source code, it most likely indicates that the
directory contains test code used to ensure that the code owned by
the package works as it was designed. Many of the test scripts
contained within Zope "tests" directories will be scripts which
contain "unit tests".
What Unit Tests Are
A "unit" may be defined as a piece of code with a single intended
purpose. A "unit test" is defined as a piece of code which exists
to codify the intended behavior of a unit and to compare its
intended behavior against its actual behavior.
Unit tests are a way for developers and quality assurance engineers
to quickly ascertain whether independent units of code are working as
expected. Unit tests are generally written at the same time as the
code they are intended to test. A unit testing framework allows a
collection of unit tests to be run without human intervention,
producing a minimum of output if all the tests in the collection are
What Unit Tests Are Not
It's very useful to define unit tests in terms of what they are
not. From the "Extreme Programming Enthusiast" website
Unit tests are not:
- Manually operated.
- Automated screen-driver tests that simulate user input (these
are "functional tests").
- Interactive. They run "no questions asked."
- Coupled. They run without dependencies except those native to
the thing being tested.
- Complicated. Unit test code is typically straightforward
procedural code that simulates an event.
Unit Testing Frameworks
A unit testing framework is generally employed to collect related
unit tests together in order to make it easier to run them as a
group. When used with a unit testing framework, unit tests live
outside of the modules of code they're intended to test.
How Unit Tests Help In The Development Process
Unit tests should be written at the same time the code they test is
written. A short, healthy cycle of "code/write test/run
test/repeat" can help a developer code more quickly by reducing
"backtracking" effort and by helping the developer focus on the
actual problem at hand. Additonally, the unit tests generated at
initial development time can serve as later assurance that
maintenance and refactoring performed on code does not break any of
its intended functionality or behavior. The results of unit tests
may additionally be used as a metric by quality assurance personnel
along with the results of other tests to gauge code quality before
before it is "shipped."
Basic Unit Testing Philosophies
Write unit tests at the same time that you write the code.
Nothing's worse than being faced with the prospect of writing tests
against a huge chunk of existing code because you "have to."
Writing unit tests post-facto can be boring and also robs you of the
main benefits that unit testing can provide. Writing unit tests at
the same time you write the code helps you focus on the task at
hand. Writing unit tests in conjuction with code can be fun and
satisfying, and can help you improve the quality of your code while
its goals are fresh in your mind. Used properly, unit testing may
also help you write code faster, because you will need to
"backtrack" less. Some "Extreme Programming" enthusiasts posit that
you should write a test before you write its associated code, and
then develop the code until it passes the test.
Unit tests should be developed against as small and specific a
subset of a system's or subsystem's functionality as possible. For
instance, a one unit test may test that a unique id generator
produces ids of a specific length, while another unit test in the
same group may ensure that a thousand ids from the same unique id
generator do not contain the same value. Writing a single unit test
which tests a broad swath of a system's capabilities is
counterproductive, because it does not allow for a fine enough
granularity when attempting to figure out "what went wrong,"
requiring the developer to "backtrack". Unit testing is capable of
helping to help reduce backtracking, but only if used properly.
A unit test does not produce any output unless it fails. If a unit
test fails, it should print something useful, but short. A unit
test should never fill the screen with output or otherwise produce
output that needs to be manually examined for "clues". This is the
realm of other testing methodologies. If unit tests are written at
sufficiently granular level, it is often enough just to know the
name of the unit test that failed.
"It is better to have tried to test and failed than to not have
tried to test at all" aka "test the riskist things first." If the
prospect of writing a series of unit tests for an existing system
seems daunting, it's important to remember that no matter how many
unit tests you write, you cannot prove that your software does not
have bugs. Therefore, you cannot possibly test every case while
developing. You should plan to write tests against code based on
the risks involved in not testing that code. Don't feel that you
need to write a test case for every "corner case" (although do try
to hit the riskiest ones). In the worst case, it's better to have a
test module with one lonely unit test in it than none at all.
"Test fixtures" are employed by unit tests. Test fixtures are bits
of state and environment that allow the unit test to perform its job
properly. An example of a test fixture might be a file, an instance
of a class, or a row in a database table. Any part of the
environment needed by a unit test besides the unit testing framework
itself is considered a test fixture.
In general, the common fixtures required by individual tests in a
testing group should be more or less identical. If the fixtures
needed by a single test are radically different than the fixtures
required by the rest of the tests, or if the test does not require
the fixtures provided to the other tests, it probably belongs in
another (or its own) group of tests.
When a unit test in a group modifies the state of a test fixture,
the test fixture should be restored to a known state before the next
unit test in the group is run. Often, this means "rolling back"
changes to a transactional database or restoring the state of a
string so the next unit test can inherit a known state of the same
fixtures. The unit testing framework has capabilities which allow
you to automate most of this work by providing a "set up" method
which gets called before each test is run and a "tear down" method
that gets called after a test is finished.
Unit tests should play nicely with the unit testing framework they
employ. Unit tests should not call "sys.exit()" or do similar
things which effect their ability to be run as part of a group of
tests. The testing framework attempts to deal with misbehaved unit
tests, but it's better just to make them behave nicely in the first
Unit tests should "grow" with the code that they're intended to
test. For example, if a group of unit tests is intended to verify
the behavior of all of the routines within a module, additional unit
tests should be added to the test group when new functionality is
added to that module.
Writing Unit Tests For The Zope Core
If you're writing core code, you probably don't need to listen to
any more of this. :-) The rules for writing tests for Zope core
code are simple:
- The testing code should make use of the standard Python unittest
module. See the Python docs for usage information.
- Tests must be placed in a "tests" subdirectory of the package or
directory in which the core code you're testing lives.
- Test modules should be named something which represents the
functionality they test, and should begin with the prefix "test."
E.g., a test module for BTree should be named testBTree.py.
- An individual test module should take no longer than 60 seconds
more information available at http://dev.zope.org/Zope3/WritingUnitTests
Writing Unit Tests For Applications Based On Zope
Zope uses the standard Python unittest module. See the Python docs
for usage information. You should establish your own conventions
for naming and placement of test modules.
Writing unit tests against applications based on Zope can be
difficult. Zope is a collection of related modules, some with
non-trivial interdependencies. Running its code successfully also
in some cases depends on state provided only in the context of a web
request, so calling Zope methods directly may not work as you
expect. If you're not intimately familiar with Zope, implementing
unit tests can be fustrating. For example, for the common case,
before you are able to write code which tests a Zope SQL Method, you
must establish a test fixture which represents your entire Zope
Luckily, some tools are at your disposal to make writing unit tests
against Zope components and applications easier by making the
creation of these fixtures easier.
Surprisingly, one of the most effective tools for facilitating unit
testing is ZEO (http://www.zope.org/Products/ZEO). ZEO is an
open-source clustering solution for Zope which makes it possible to
front-end a single "storage server" which manages a Zope object
database with multiple Zope clients that run a "client storage".
The reason ZEO is interesting for unit testing is mostly an
unintended side-effect of how it works as compared to Zope without
ZEO. Zope without ZEO commonly uses a "FileStorage" to hold its
object database. When Zope is started with a FileStorage, the
FileStorage code processes an "index" file. This takes time. Zope
using a ClientStorage as with ZEO does not process an index file,
making startup faster. Fast startup of Zope is critical to
effective unit testing. It is recommended that you implement ZEO if
you're heavy in to unit testing, as it really speeds things up.
It's not strictly required, however.
Unit test scripts found in the Zope source code is based on
the PyUnit unit testing framework, available from
http://pyunit.sourceforge.net, written by Stephen Purcell (thanks
Stephen!). PyUnit is based on the JUnit testing framework for Java
(written by Kent Beck and Erich Gamma), which in turn was based on a
testing framework designed for Smalltalk (also written by Kent
Unit testing is a primary tenet of "Extreme Programming", a software
development methodology designed to faciliate the rapid production
of high quality code with a minimum of developmental ceremony. For
more information on unit tests as they relate to Extreme
Programming, see http://c2.com/cgi/wiki?UnitTestsDefined. Although
Digital Creations has not embraced the entire spectrum of Extreme
Programming methodologies in its software development process, we've
found unit tests a way to speed development and produce