Commit Graph

29 Commits

Author SHA1 Message Date
NIIBE Yutaka e89d57a2cb
tests: Fix tests/gpgme for in-source-tree builds.
* tests/gpgme/Makefile.am: Don't use setup.scm/ dir.
* tests/gpgme/all-tests.scm: Fix the name of the environment.

--

GnuPG-bug-id: 6313
Fixes-commit: c19ea75f10
Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
2022-12-21 10:52:24 +09:00
NIIBE Yutaka c19ea75f10
tests: Fix make check with GPGME.
* tests/gpgme/Makefile.am: Create directories for logs.

--

Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
2022-12-16 13:11:14 +09:00
NIIBE Yutaka 1246e16432
tests: Fix to support --enable-all-tests and variants.
* tests/gpgscm/tests.scm (test::scm): Add VARIANT argument.
(tests::new): Likewise.
(open-log-file, report): Support VARIANT.
* tests/gpgme/all-tests.scm (setup-c, setup-py): Follow the change.
* tests/cms/all-tests.scm: Likewise.
* tests/cms/run-tests.scm: Likewise.
* tests/migrations/all-tests.scm: Likewise.
* tests/migrations/run-tests.scm: Likewise.
* tests/openpgp/all-tests.scm: Likewise.
* tests/openpgp/run-tests.scm: Likewise.

--

Forward port from 2.2 branch of:
	0fd7a90207

Fixes-commit: 1c88104a3f
Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
2022-11-24 15:57:25 +09:00
Werner Koch 50d12860ef
tests: Restrict the use of gpgconf.ctl to make check.
* common/homedir.c (unix_rootdir): Add ".enable" statement to
gpgconf.ctl
* tests/gpgconf.ctl.in: Use this statement.
* tests/cms/Makefile.am (OLD_TESTS_ENVIRONMENT): Enable .enable.
(TESTS_ENVIRONMENT): Ditto.
* tests/gpgme/Makefile.am (TESTS_ENVIRONMENT): Ditto.
* tests/migrations/Makefile.am (TESTS_ENVIRONMENT): Ditto.
* tests/openpgp/Makefile.am (TESTS_ENVIRONMENT): Ditto.
* tests/pkits/Makefile.am (TESTS_ENVIRONMENT): Ditto.
* tests/tpm2dtests/Makefile.am (TESTS_ENVIRONMENT): Ditto.
--

Without that it was required to delete the gpgconf.ctl to run manual
tests.

What a hack.
2022-09-14 12:05:21 +02:00
NIIBE Yutaka 39d478f5ba w32: Fix for make check.
* common/Makefile.am (module_tests): Exclude t-exechelp and
t-exectool.
* common/t-stringhelp.c (mygetcwd): Convert '\' to '/'.
* tests/cms/Makefile.am: Add $(EXEEXT).
* tests/gpgme/Makefile.am: Likewise.
* tests/migrations/Makefile.am: Likewise.
* tests/openpgp/Makefile.am: Likewise.

--

Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
2022-04-06 11:28:00 +09:00
Werner Koch bcd5feec0e
tests: New way to make use of gpgconf.ctl in tests.
* Makefile.am (all-local): New to setup symlinks.
(distclean-local): New.
* tests/Makefile.am: Remove the gpgconf related targets.  Just keep
gpgconf.ctl.in in EXTRA_DIST
* tests/cms/Makefile.am (GNUPG_BUILD_ROOT):
* tests/gpgme/Makefile.am (GPGSCM_PATH):
* tests/openpgp/Makefile.am (GNUPG_BUILD_ROOT):
* tests/pkits/Makefile.am (GNUPG_BUILD_ROOT):
* tests/tpm2dtests/defs.scm (tools): Revert to the former values.
* tests/openpgp/defs.scm (tools): Ditto.
--

This
Fixes-commit: 399ebf6d87
Fixes-commit: 84fcd8e6eb
because we ran into problems with the idea of first doing a
test-install for the checks.  "make distcheck" turned out to
be too problematic.  Symlinks are a better way of doing this.
Also fixes
GnuPG-bug-id: 5634
2021-10-12 14:30:13 +02:00
Werner Koch 399ebf6d87
build: Prepare for using installed versions for tests.
* tests/gpgconf.ctl.in: New.
* tests/Makefile.am (EXTRA_DIST): Add new file.
(TESTINST_DIRS): New.
(clean-local): New.
(clean-local-testinst): New.
(check-recursive): New hook.
(bin/gpgconf.ctl): Run a test install.
* tests/cms/Makefile.am (TESTS_ENVIRONMENT): Set new envvar
GNUPG_BUILD_ROOT.
* tests/gpgme/Makefile.am (TESTS_ENVIRONMENT): Ditto.
* tests/openpgp/Makefile.am (TESTS_ENVIRONMENT): Ditto.
* tests/pkits/Makefile.am (TESTS_ENVIRONMENT): Ditto.
* tests/tpm2dtests/Makefile.am (TESTS_ENVIRONMENT): Ditto.
--

Right now this helps only with the global configuraion dir which is
now not set and thus the build host'ss own /etc/gnupg does not anymore
affect the build.

Signed-off-by: Werner Koch <wk@gnupg.org>
2021-10-01 14:46:16 +02:00
Werner Koch 4747b9e868
tests: Improve handling of spaces in $PATH
--

Signed-off-by: Werner Koch <wk@gnupg.org>
(cherry picked from commit b2590f2e47)
2020-10-04 20:45:51 +02:00
NIIBE Yutaka 64b7c6fd19 tests: Add "disable-scdaemon" in gpg-agent.conf.
* tests/openpgp/defs.scm: Add "disable-scdaemon".  Remove
  "scdaemon-program".
* tests/gpgme/gpgme-defs.scm, tests/gpgsm/gpgsm-defs.scm: Likewise.
* tests/inittests, tests/pkits/inittests: Add "disable-scdaemon"

--

Before this change, running "make check" accesses USB device by
scdaemon on host computer.  If there is any smartcard/token available,
it may affect test results.  Because default key choice depends on
smartcard/token availability now and existing tests have nothing about
testing smartcard/token, disabling scdaemon is good.

Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
2019-02-21 12:26:09 +09:00
Justus Winter 386a7bbb24
tests: Configure the environments to use scdaemon from build tree.
* tests/gpgme/gpgme-defs.scm: Use the scdaemon from the build tree
when writing a 'gpg-agent.conf'.
* tests/gpgsm/gpgsm-defs.scm: Likewise.
* tests/openpgp/defs.scm: Likewise.
--

As of 97a2394eca gpg may query the
scdaemon for a signing key to use.  To make sure that the agent calls
the right scdaemon, we provide the path explicitly in the
'gpg-agent.conf' that is used in the tests, similar to what we do for
the agent itself and the pinentry.

GnuPG-bug-id: 3165
Signed-off-by: Justus Winter <justus@g10code.com>
2017-05-16 16:07:25 +02:00
Justus Winter f4365790da
tests: Make it possible to run all tests using our infrastructure.
* Makefile.am (TESTS_ENVIRONMENT): New variable.
(check-all): New phony target to run all tests.
* tests/gpgme/gpgme-defs.scm (have-gpgme?): New function that tests
whether the GPGME test suite is available instead of exiting the
process.
* tests/gpgscm/init.scm (export): New macro.
* tests/gpgscm/tests.scm (run-tests): New function.
(load-tests): Likewise.
* tests/gpgme/run-tests.scm: Simplify and move the parsing of the list
of tests to 'all-tests.scm'.
* tests/gpgsm/run-tests.scm: Likewise.
* tests/migrations/run-tests.scm: Likewise.
* tests/openpgp/run-tests.scm: Likewise.
* tests/gpgme/Makefile.am: To select the tests to run, use the
variable 'TESTS'.  This harmonizes the interface with the automake
test suite.
* tests/gpgsm/Makefile.am: Likewise.
* tests/migrations/Makefile.am: Likewise.
* tests/openpgp/Makefile.am: Likewise.
* tests/openpgp/README: Likewise.
* agent/all-tests.scm: New file.
* common/all-tests.scm: Likewise.
* g10/all-tests.scm: Likewise.
* g13/all-tests.scm: Likewise.
* tests/gpgme/all-tests.scm: Likewise.
* tests/gpgsm/all-tests.scm: Likewise.
* tests/migrations/all-tests.scm: Likewise.
* tests/openpgp/all-tests.scm: Likewise.
* tests/run-tests.scm: Likewise.
--

This change allows us to run all tests in parallel and write one XML
report capturing the results of every test.  It also lays the
foundation to parametrize test suites.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-05-11 18:12:37 +02:00
Justus Winter 78d6a25a2d
tests: Move the makefile parser.
* tests/gpgme/gpgme-defs.scm (parse-makefile, parse-makefile-expand):
Move...
* tests/gpgscm/makefile.scm: ... here.
* tests/gpgscm/Makefile.am (EXTRA_DIST): Add new file.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-05-11 17:41:03 +02:00
Justus Winter ef1922b3b1
tests: Fix Python detection.
* tests/gpgme/gpgme-defs.scm (python): Fix Python detection.
--
In 25e6444b3f4601c7821beab06bc4520deacb007b we changed the way GPGME
creates the build directory for the Python bindings.  We now use the
basename of the interpreter as an identifier, that means we have to
get the base name right, a link to the same binary is not sufficient.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-04-24 17:32:41 +02:00
Justus Winter 06a177ceea
tests: Deduplicate and simplify code.
* tests/gpgme/gpgme-defs.scm (create-file): Move...
* tests/gpgsm/gpgsm-defs.scm (create-file): ... likewise...
* tests/openpgp/defs.scm (create-file): Here.
(create-gpghome): Use 'create-file'.
* tests/openpgp/gpg-agent.conf.tmpl: Delete file.
* tests/openpgp/gpg.conf.tmpl: Likewise.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-04-24 15:59:49 +02:00
Justus Winter ee715201ae
gpgscm: Emit JUnit-style XML reports.
* tests/gpgscm/Makefile.am (EXTRA_DIST): Add new file.
* tests/gpgscm/lib.scm (string-translate): New function.
* tests/gpgscm/main.c (main): Load new file.
* tests/gpgscm/tests.scm (dirname): New function.
(test-pool): Record execution times, emit XML report.
(test): Record execution times, record log file name, emit XML report.
(run-tests-parallel): Write XML report.
(run-tests-sequential): Likewise.
* tests/gpgscm/xml.scm: New file.
* tests/gpgme/Makefile.am (CLEANFILES): Add 'report.xml'.
* tests/gpgsm/Makefile.am: Likewise.
* tests/migrations/Makefile.am: Likewise.
* tests/openpgp/Makefile.am: Likewise.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-04-24 14:28:57 +02:00
Justus Winter ed4d23d75e
tests: Locate resources and scripts relative to top source dir.
--

Locate every resource and every script used in the tests using a path
relative to the top of the source tree.

This is a purely mechanical change, mostly done using regular
expressions, with a few manual fixups here and there.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-04-24 14:14:05 +02:00
Justus Winter b20780658e
tests,w32: Fix importing the extra key for GPGME's keylist test.
* tests/gpgme/wrap.scm: Qualify the tests name with the extension for
executables (if any).

Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-28 16:51:18 +02:00
Justus Winter ceb4b24575
tests: Remove debugging remnants.
* tests/gpgme/gpgme-defs.scm (run-python-tests?): Remove 'trace's.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-20 14:36:36 +01:00
NIIBE Yutaka a672ddec03 tests: Fix running python condition.
* tests/gpgme/gpgme-defs.scm (run-python-tests?): We need Python.

Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
2017-03-15 16:45:18 +09:00
Justus Winter d82abbb1b6
tests: Skip Python tests if the bindings are not built.
* tests/gpgme/wrap.scm (python): Move variable...
* tests/gpgme/gpgme-defs.scm (python): ... here.
(run-python-tests?): New function.
* tests/gpgme/run-tests.scm: Only run Python tests if the bindings can
be located in GPGME's build directory.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-14 12:45:29 +01:00
Justus Winter 046a15a88c
tests: Run the tests for the Python bindings of GPGME.
* tests/gpgme/gpgme-defs.scm (create-file): Write lines.
(create-gpgmehome): Extend function to create the right environment
for the Python tests.
* tests/gpgme/run-tests.scm: Make an environment cache for the Python
tests and enable them.
* tests/gpgme/wrap.scm: Do not hardcode the path of the Python
interpreter.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-09 14:33:02 +01:00
Justus Winter cca91a3f8f
tests: Rework environment setup.
* tests/gpgscm/tests.scm (test::scm): Add a setup argument.
(test::binary): Likewise.
(run-tests-parallel): Remove setup parameter.
(run-tests-sequential): Likewise.
(make-environment-cache): New function that handles the cache
protocol.
* tests/gpgme/run-tests.scm: Adapt accordingly.
* tests/gpgsm/run-tests.scm: Likewise.
* tests/migrations/run-tests.scm: Likewise.
* tests/openpgp/run-tests.scm: Likewise.
--
This change allows us to have different environments for tests.  This
is needed to run more GPGME tests, and to increase concurrency while
running all tests.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-09 13:26:06 +01:00
Justus Winter 7e19786a5d
tests: Harmonize temporary and socket directory handling.
* tests/gpgscm/tests.scm (mkdtemp): Do not magically obey the
environment variable 'TMP', make sure to always return an absolute
path.
* tests/gpgme/Makefile.am (TMP): Drop variable.
(TESTS_ENVIRONMENT): Drop 'TMP'.
* tests/gpgme/gpgme-defs.scm (create-gpgmehome): Start the agent.  Do
not create private key store, the agent does that for us.
* tests/gpgsm/Makefile.am (TMP): Drop variable.
(TESTS_ENVIRONMENT): Drop 'TMP'.
* tests/gpgme/gpgme-defs.scm (create-gpgsmhome): Start the agent.  Do
not create private key store, the agent does that for us.
* tests/migrations/Makefile.am (TMP): Drop variable.
(TESTS_ENVIRONMENT): Drop 'TMP'.
* tests/migrations/common.scm (gpgconf): New variable.
(run-test): Create and remove socket directory.
* tests/migrations/extended-pkf.scm (src-tarball): Remove variable.
(setup): Remove function.
(trigger-migration): Likewise.
Use 'run-test' to execute the test.
* tests/migrations/from-classic.scm (src-tarball): Remove variable.
(setup): Remove function.
Use 'run-test' to execute the tests.
* tests/openpgp/Makefile.am (TMP): Drop variable.
(TESTS_ENVIRONMENT): Drop 'TMP'.
* tests/openpgp/README: Do not mention 'TMP'.
* tests/openpgp/defs.scm (with-home-directory): New macro.
(create-legacy-gpghome): Do not create private key store, the agent
does that for us.
(start-agent): Make sure to terminate the right agent with 'atexit'.
--

Previously, the test suite relied upon creating home directories in
'/tmp'.  This has been problematic in some build environments,
although POSIX mandates that '/tmp' must be available.

We now rely on 'gpgconf --create-socketdir' to create a suitable
socket directory for us.  This allows us to get rid of some cruft.  It
also aligns the environment the tests are run in closer with the
environment that we intend that GnuPG runs in.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-06 18:10:10 +01:00
Justus Winter 02a39f0d1e tests,w32: Fix GPGME tests requiring a pinentry.
* tests/gpgme/gpgme-defs.scm: Use our fake pinentry, and configure it
to supply the correct passphrase.
--
Previously, the GPGME tests used the fake pinentry distributed with
GPGME which is a Bourne shell script.  Use the fake pinentry from
GnuPG instead.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-01-25 16:58:46 +01:00
Justus Winter 5f2da5d439 tests: Skip GPGME tests that are not built.
* tests/gpgme/wrap.scm: Skip tests that are not built.
--
Previously, all tests from GPGME's test suite that were not built
(e.g. because GPGME was not configured with QT) were marked as
failures.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-01-25 11:18:23 +01:00
Justus Winter 6ecd8b3e71 tests,w32: Fix locating GPGME's tests on Windows.
* tests/gpgme/run-tests.scm: Qualify the test with the executable
extension.
--
Previously, the test suite could not locate the test executables on
Windows because the Windows-specific file extension was missing.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-01-25 11:16:23 +01:00
Justus Winter a30c0a6972 tests: Move argument parser.
* tests/gpgme/gpgme-defs.scm (flag): Move...
* tests/gpgscm/tests.scm: ... over here.

Signed-off-by: Justus Winter <justus@g10code.com>
2016-12-20 16:25:35 +01:00
Justus Winter 0e2055c7d3 build: Fix distcheck.
* tests/gpgme/Makefile.am (CLEANFILES): New variable, clean test logs.

Signed-off-by: Justus Winter <justus@g10code.com>
2016-12-15 11:02:24 +01:00
Justus Winter 948cca9c99 tests: Reuse GPGME's tests.
* configure.ac (AC_CONFIG_FILES): Add new Makefile.
* tests/Makefile.am (SUBDIRS): Add new directory.
* tests/gpgme/Makefile.am: New file.
* tests/gpgme/gpgme-defs.scm: Likewise.
* tests/gpgme/run-tests.scm: Likewise.
* tests/gpgme/setup.scm: Likewise.
* tests/gpgme/wrap.scm: Likewise.

Signed-off-by: Justus Winter <justus@g10code.com>
2016-12-14 16:50:44 +01:00