diff options
Diffstat (limited to 'pactest/README')
| -rw-r--r-- | pactest/README | 304 |
1 files changed, 304 insertions, 0 deletions
diff --git a/pactest/README b/pactest/README new file mode 100644 index 00000000..7ef24b1d --- /dev/null +++ b/pactest/README @@ -0,0 +1,304 @@ +README +====== + +pactest is a test suite for the ArchLinux package manager: pacman. + +It has a rather high level view of operations performed by pacman: it +automatically creates a test environment based on a test case file +description, the run pacman, and finally check the results of test according +to a set of rules defined in the test case. + +It is written in Python and makes available most of what can be found in +pacman's code to create ArchLinux packages or read and write databases entries. + +Each test case is defined in a separate file that is sourced in order to set +the environment. + +pactest creates the environment in the subdirectory "root" created in the +current directory. +The following directory structure is used: + - var/lib/pacman: databases path (local and sync ones) + - etc/pacman.conf for pacman configuration file + - var/cache/pkg: sync packages cache + - var/log/pactest.log: log file + - var/pub: location for pseudo sync repositories + - tmp: hold all local package archives (to be used with pacman -A or -U) + +Note: the logfile is used to capture all pacman outputs. + +Test case example: + self.description = "Install a package" + + p = pmpkg("dummy", "1.0-3") + p.files = ["bin/dummy", + "usr/man/man1/dummy.1"] + self.addpkg(p) + + self.args = "-A dummy-1.0-1.pkg.tar.gz" + + self.addrule("PACMAN_RETCODE=0") + self.addrule("PKG_EXIST=dummy") + for f in p.files: + self.addrule("FILE_EXIST=%s" % f) + +Basically, the above test case will try to install a package (dummy-1.0-3), +including two files, from a local archive, by calling "pacman -A" +Upon completion, it checks that: + - pacman returned no error code, + - a "dummy" entry exists in the "local" database + - all files from the package exist in the filesystem. + + +Installation +============ + +Simply extract the pactest tarball, jump into the newly created directory and +run pactest.py. See the usage section below. + +Remark: pacman 3.x restrictions regarding fakeroot must be disabled. +It can be done by configuring pacman with the --disable-fakeroot flag: + ./configure --disable-fakeroot + +For pacman 2.9.x releases, apply the patch found in the patches directory, +then export CFLAGS as following before rebuilding pacman: + export CFLAGS=-DNOFAKEROOT + + +Usage +===== + +pactest will run the suite of tests defined by the "--test" parameter. + +Example: + ./pactest.py --test=test/* + +This example will run tests from the "test" directory. +Note: several "--test" options can be passed to pactest. + +Use the ""help" option to get the full list of parameters: + ./pactest.py --help + + +Parameters +========== + +The test environment is described by the following basic parameters: + + description + ----------- + +A short string describing the aim of the test case. It is displayed on the +standard output during test execution. + + args + ---- + +A string of arguments that are passed to the pacman binary when the test is +run. + +Example: + self.args = "-S dummy" + + option + ------ + +A dictionnary that holds the data used in the pacman configuration file. +It has 3 keys, each one of them pointing at a list of strings: + - noupgrade + - noextract + - ignorepkg + +Examples: + self.option["noupgrade"] = ["etc/X11/xorg.conf", + "etc/pacman.conf"] + self.option["noextract"] = ["etc/lilo.conf"] + + filesystem + ---------- + +A list of strings describing a set of files supposed to exist in the filesystem +when the test case is run. +Upon test startup, pactest will automatically populate the test environment +filesystem with this list of files. + +Example: + self.filesystem = ["bin/dummy", + "etc/X11/xorg.conf.pacsave"] + +Note that all paths are relative ones, and thus file names should not start +with a "/". + + +Packages +======== + +The test case file description shall define a number of packages that can be +used to either populate a database, or to feed pacman with data needed during +its execution. + +This can be achieved by creating pmpkg objects, with the following constructor: + pmpkg(name, version) + +Both "name" and "version" are strings. Also, note that if not provided, the +version defaults to "1.0-1". + +Example: + pkg1 = pmpkg("dummy", "2.1-1") + pkg2 = pmpkg("foobar") + +All fields from a ArchLinux package can be set and modified directly with no +methods to access them. +Note: some fields are automatically set by pactest and should preferably not +be modified by hand (i.e. "md5sum", "size", or "csize"). + +Examples: + pkg.depends = ["pkg2", "pkg3>=2.0"] + pkg.files = ["bin/dummy", "etc/dummy.conf", "usr/man/man1/dummy.1"] + + +Databases +========= + +The test environment provides a way to create and fill databases (local or +sync ones). + +The following methods shall be used: + + * addpkg2db(database, package) + +Notes: "database" is a string, and "package" shall be a previously created +pmpkg object. + +Examples: + self.addpkg2db("local", lpkg) + self.addpkg2db("sync1", spkg11) + self.addpkg2db("sync1", spkg12) + self.addpkg2db("sync2", spkg21) + +Note: there is no need to explicitly create a database. The "local" one +already exists (even if empty), and sync databases are created on the fly when +a new database new is given. + + * addpkg(package) + +package is an existing pmpkg object. +It creates a package archive based on the given object. The resulting archive +is located in the temporary directory of the test environment, ready to be +supplied to pacman for test purposes. + + +Files +===== + +All files created by pactest are filled with a content defaulting to the file +name, with an additional line feed. +For instance, the content of a file "bin/dummy" created in the test environment +file system is: "bin/dummy\n". + +It is possible to create directories by appending a slash "/" to the name and +to create symlinks by appending an arrow followed by a filename " -> target". + +Note: only relative symlinks are supported. + +Example: + pkg = pmpkg("dummy") + pkg.files = ["bin/dummy", + "usr/local/", + "lib/libfoo.so.O", + "lib/libfoo.so -> ./libfoo.so.0"] + +In this example, "usr/local/" is a directory, and "libfoo.so" will be a +symlink pointing at "libfoo.so.0". It is usually a good idea to also define +the target of the symlink! + +It can be interesting for some tests to create altered files. This can be +done by appending one or more asterisks "*" to the file name. + +Example: + lpkg = pmpkg("dummy") + lpkg.files = ["bin/dummy"] + self.addpkg2db("local", lpkg) + + newpkg = pmpkg("dummy", "1.0-2") + newpkg.files = ["bin/dummy*"] + self.addpkg(newpkg) + + self.args = "-U dummy-1.0-2.pkg.tar.gz" + +In this case, package "lpkg" will install a file "bin/dummy" with "bin/dummy\n" +as its content. Upon package upgrade, newpkg will provide a file named +"bin/dummy" with "bin/dummy*\n" as its content. +This is useful to simulate that a file has been modified between two different +releases of a same package. + +The same also applies to files from the "filesystem" parameter of the test +environment, and to the "backup" attribute of a package object. + + +Rules +===== + +Finally, to check test success or failure, one shall define a set of rules. + + addrule(rule) + ------------- + +A rule is a string composed by a key and an item, joined with a "=" symbol. + +Examples: + self.addrule("PACMAN_RETCODE=0") + self.addrule("PKG_EXIST=dummy") + self.addrule("FILE_MODIFIED=bin/dummy") + self.addrule("PKG_DEPENDS=xorg|fontconfig") + +Note: an item can be divided into two arguments, as shown in the latter +example. + +All rules can be prepended with a bang "!" in order to tell pactest to expect +the exact opposite result. + +Example: + self.addrule("!FILE_MODIFIED=bin/dummy") + +Finally, the following rules are supported: + + . PACMAN rules + +Possible rules are: + + PACMAN_RETCODE=value + PACMAN_OUTPUT=value + +For the RETCODE one, pactest will compare pacman return code with the value +provided as an item. +For the OUTPUT one, pactest will grep pacman outputs for the given value. + +Note: PACMAN_OUTPUT should not be used. Pacman outputs are likely to change +from one release to another, so that it's reliability is quite low. + + . PKG rules + +For each rule, pactest will read the entry "name" from the local database and +challenge the requested data with it. + +Possible rules are: + + PKG_EXIST=name + PKG_MODIFIED=name + PKG_VERSION=name|version + PKG_DEPENDS=name|depname + PKG_REQUIREDBY=name|reqbyname + +Example: + PKG_DEPENDS=ncurses|glibc + +pactest will test the local database entry "ncurses" has "glibc" in its +DEPENDS field. + + . FILE rules + + FILE_EXIST=path/to/file + FILE_MODIFIED=path/to/file + FILE_PACNEW=path/to/file + FILE_PACSAVE=path/to/file + FILE_PACORIG=path/to/file |