Commit Graph

7322 Commits

Author SHA1 Message Date
Werner Koch 3a10de3bfd
gpg: Print more stats for the keydb and the signature cache.
* g10/sig-check.c (sig_check_dump_stats): New.
(cache_stats): New struct.
(check_key_signature2): Update stats.
* g10/gpg.c (g10_exit): Call new function.
* g10/keydb.c (kid_not_found_cache_count): Replace by ...
(kid_not_found_stats): ... new struct.  Change users.
(keydb_stats): New struct.  Update the counters.
(keydb_dump_stats): Print all stats.

Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-31 20:08:23 +02:00
Werner Koch 52ba5e67ca
gpg: Assert that an opaque parameter is really what we expect.
* g10/gpg.h (SERVER_CONTROL_MAGIC): New const.
(server_control_s): Add field 'magic'.
* g10/gpg.c (gpg_init_default_ctrl): Init MAGIC.
* g10/import.c (impex_filter_getval): Assert MAGIC.

Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-31 20:07:33 +02:00
Werner Koch 8f2671d2cc
gpg: Pass CTRL to many more functions.
--

For proper operations as a server we need to avoid global variables.
Thus we need to pass the session state CTRL to most functions.  Quite
a lot of changes but fortunately straightforward to do.

Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-31 20:07:20 +02:00
Justus Winter 5e89144cbc
gpg: Consistent use of preprocessor conditionals.
* g10/parse-packet.c: Use '#if' instead of '#ifdef' when checking
DEBUG_PARSE_PACKET.  This fixes the build with '#define
DEBUG_PARSE_PACKET 0'.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-30 16:13:45 +02:00
Justus Winter 214fa90122
common: Avoid undefined behavior.
* common/iobuf.c (iobuf_read_line): Do not consider 'length' if
'buffer' is NULL.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-30 16:13:43 +02:00
Werner Koch a6142dbdbc
gpg: Remove the use of the signature information from a KBX.
* g10/keydb.c (keyblock_cache): Remove field SIGSTATUS.
(keyblock_cache_clear): Adjust for that removal.
(parse_keyblock_image): Remove arg SIGSTATUS.  Remove the signature
cache setting; this is now done in the parser.
(keydb_get_keyblock): Do not set SIGSTATUS.
(build_keyblock_image): Remove arg SIGSTATUS and simplify.  Change
caller.
* kbx/keybox-blob.c: Explain that the signature information is not
anymore used.
(_keybox_create_openpgp_blob): Remove arg SIGSTATUS and change
callers.
* kbx/keybox-search.c (keybox_get_keyblock): Remove arg R_SIGSTATUS
and change callers.
* kbx/keybox-update.c (keybox_insert_keyblock): Likewise.
--

This thing was too complicated and has been replaced by the new ring
trust packet code.

Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-30 16:01:55 +02:00
Werner Koch 7bf24e8146
gpg: Fix actual leak and possible leaks in the packet parser.
* g10/packet.h (struct parse_packet_ctx_s): Change LAST_PKT deom a
pointer to its struct.
(init_parse_packet): Adjust for LAST_PKT not being a pointer.
* g10/parse-packet.c (parse): Ditto. Free the last packet before
storing a new one in case of a deep link.
(parse_ring_trust): Adjust for LAST_PKT not being a pointer.
* g10/free-packet.c (free_packet): Ditto.
* g10/t-keydb-get-keyblock.c (do_test): Release keyblock.
--

Fixes-commit: afa8680908
Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-30 16:01:52 +02:00
Werner Koch 64665404e4
gpg: Fix export porting of zero length user ID packets.
* g10/build-packet.c (do_user_id): Avoid indeterminate length header.
--

We are able to import such user ids but when exporting them the
exported data could not be imported again because the parser bails out
on invalid keyrings.  This is now fixed and should be backported.

Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-30 10:35:20 +02:00
Werner Koch a8895c99a7
gpg: Revamp reading and writing of ring trust packets.
* g10/parse-packet.c (parse_trust): Rename to ...
(parse_ring_trust): this.  Change args and implement new ring trust
packet format.
(parse): Add special ring trust packet handling.
* g10/packet.h (PKT_user_id): New fields KEYUPDATE, UPDATEURL, and
KEYSRC.
(PKT_public_key): Ditto.
(RING_TRUST_SIG, RING_TRUST_KEY, RING_TRUST_UID): New consts.
(PKT_ring_trust): New.
(struct packet_struct): Remove member RING_TRUST.
(strcu parse_packet_ctx_s): Add field SKIP_META.
(init_parse_packet): Init SKIPT_META.
* g10/free-packet.c (release_public_key_parts): Free UDPATEURL.
(free_user_id): Ditto.
* g10/mainproc.c (list_node): Remove printing of non-documented "rtv"
lines.
* g10/build-packet.c (build_packet_and_meta): New.
(do_ring_trust): New.
* g10/export.c (write_keyblock_to_output): Use build_packet_and_meta
in backup mode.
(do_export_one_keyblock): Ditto.
* g10/import.c (read_block): Add arg WITH_META.  Skip ring trust
packets if that ism not set.
(import): Call read_block WITH_META in restore mode.
* g10/keydb.h (KEYSRC_UNKNOWN, KEYSRC_FILE, KEYSRC_KS, KEYSRC_PREF_KS)
(KEYSRC_WKD, KEYSRC_WKD_SD, KEYSRC_DANE): New constants.  They are not
yet used, though.
* g10/keydb.c (parse_keyblock_image): Allow ring trust packets.
(build_keyblock_image): Ditto.  Use build_packet_and_meta.
* g10/keyring.c (keyring_get_keyblock): Remove specila treatment of
ring trust packets.
(write_keyblock): Use build_packet_and_meta.  Remove special treatment
of ring trust packets and initialization of the signature caches.
--

This patch introduced the framework to store meta data for keys and
user ids in the keyrings/keyboxes.  Ring trust packets are
implementation defined and have always been used in gpg to cache the
signature verification status.

Ring trust packets are only exported with the export option "backup"
and only imported with the import option "restore".

The new code uses a cleaner way to handle the ring trust packets: When
the parser reads a ring trust packet and the previously read packet
matches the type of that ring trust packet, the information is stored
in that previously read packet (signature, user id, or primary key)
and the next packet is read immediately.  Thus only the parser sees
the ring trust packets.  Ring trust packets are written by using the
new function build_packet_and_meta instead of build_packet.  That
function writes a ring trust packet when the needed information is
available.

As a side-effect of this patch the signature status cache works again
and "gpg --check-sigs" is thus much faster.

Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-30 09:07:02 +02:00
Werner Koch afa8680908
gpg: Extend free_packet to handle a packet parser context.
* g10/packet.h (struct parse_packet_ctx_s): Add fields LAST_PKT and
FREE_LAST_PKT.
(init_parse_packet): Clear them.
(deinit_parse_packet): New macro.  Change all users if
init_parse_packet to also call this macro.
* g10/free-packet.c (free_packet): Add arg PARSECTX and handle shallow
packet copies in the context.  Change all callers.
* g10/parse-packet.c (parse): Store certain packets in the parse
context.
--

Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-29 12:08:31 +02:00
Werner Koch ba57f8302a
indent: Re-indent function free-packet.
--

Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-29 11:28:30 +02:00
Werner Koch 0526c99164
gpg: Change parse_packet to take a context.
* g10/packet.h (struct parse_packet_ctx_s): New.
(parse_packet_ctx_t): New type.
(init_parse_packet): New macro.
* g10/parse-packet.c (parse_packet, dbg_parse_packet): Change to take
a parse context.  Change all callers to provide a context instead of
directly supplying the input stream.
(search_packet, dbg_search_packet): Ditto.
(copy_all_packets, dbg_copy_all_packets): Init an use a parse context.
(copy_some_packets, dbg_copy_some_packets): Ditto.
(skip_some_packets, dbg_skip_some_packets): Ditto.
--

We will need this change to handle ring packets inside the parser.

Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-29 10:06:29 +02:00
Werner Koch f5b565a5b8
gpg: Export ring trust packets in backup mode.
* g10/export.c (write_keyblock_to_output): Export ring trust packets.

Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-29 10:06:29 +02:00
Werner Koch 5b3523d3e0
indent: Re-indent parts of build-packet.c
--
2017-03-29 10:06:29 +02:00
Justus Winter b20780658e
tests,w32: Fix importing the extra key for GPGME's keylist test.
* tests/gpgme/wrap.scm: Qualify the tests name with the extension for
executables (if any).

Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-28 16:51:18 +02:00
Werner Koch 4af389c972
gpg: Prepare for listing last_update and key origin data.
* g10/keylist.c (list_keyblock_colon): Add empty fields 19 and 20.

--

We add them now to early catch error in parsers which arbitrary limit
the number of fields in --with-colon listings.

Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-28 12:32:05 +02:00
Justus Winter 5128cd74c0
tests: Fix distcheck.
* tests/openpgp/Makefile.am (sample_msgs): Add all missing sample
messages.

Fixes-commit: 211d71f19c
Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-28 12:22:18 +02:00
Justus Winter 211d71f19c
tests: Add test for '--decrypt --unwrap'.
* tests/openpgp/Makefile.am (XTESTS): Add new test.
* tests/openpgp/decrypt-unwrap-verify.scm: New file.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-28 12:11:03 +02:00
Justus Winter 6d3edfd972
g10: Fix memory leak.
* g10/decrypt-data.c (decrypt_data): Free 'filename'.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-28 12:11:02 +02:00
Justus Winter caf0091553
common: Fix connecting to the agent.
* common/homedir.c (_gnupg_socketdir_internal): Fix error handling.
--

Prior to 26086b36 the non-existance of the socket directory was
considered an error if a non-default home directory is used.  Since
26086b36 we now create the directory on demand, but the function still
returned the fallback path.  This made the agent bind the socket in
the socket directory, and the client trying to connect to the socket
in the home directory.

Fixes-commit: 26086b362f
Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-27 16:21:02 +02:00
NIIBE Yutaka c1e6302b34 g10: Support specifying SERIALNO for --card-status.
* g10/gpg.c (main): Allow an argument for --card-status.
* g10/card-util.c (current_card_status): Rename from card_status.
(card_status): New, which supports multiple cards.
(get_one_name): Use current_card_status.

Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
2017-03-27 14:02:01 +09:00
NIIBE Yutaka d58275703f scd: Change the order of applications when accessed.
* scd/app.c (select_application): Move the app to top.

Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
2017-03-27 13:56:02 +09:00
NIIBE Yutaka 0848cfcce7 scd: Fix timeout handling for key generation.
* scd/ccid-driver.c (CCID_CMD_TIMEOUT): Back to original value.
(CCID_CMD_TIMEOUT_LONGER): New.
(ccid_transceive): Add kludge for key generation.

--

At key generation of longer key size, OpenPGP card sends back "time
extension" with BWI=100, which is unusual value in the protocol and it
actually requires host having longer timeout like 43 seconds.

Reported-by: Szczepan Zalega <szczepan@nitrokey.com>
Fixes-commit: 6510df3a7c
Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
2017-03-27 11:25:00 +09:00
Werner Koch 0b3770c421
gpg: Improve check for already compressed packets.
* common/miscellaneous.c (is_openpgp_compressed_packet): New.
(is_file_compressed): Rerad 2 more bytes and call new function.

--

Note that this does not yet allow to detect compressed data piped to
gpg.  This requires a proper read-ahead in iobuf.c which is
complicated due to the auto-removal of filter functions.  Thus such an
read-ahead needs to be done in the I/O backend of iobuf.

Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-24 11:51:44 +01:00
Werner Koch 2c237c1362
agent: New option --enable-extended-key-format.
* agent/gpg-agent.c (oEnableExtendedKeyFormat): New const.
(opts): New option --enable-extended-key-format.
(parse_rereadable_options): Set option
* agent/findkey.c (write_extended_private_key): Add arg 'update'.
(agent_write_private_key): Implement new option.

Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-24 10:35:36 +01:00
Werner Koch 6fab7bba87
agent: New option --stub-only for DELETE_KEY
* agent/findkey.c (agent_delete_key): Add arg 'only_stubs'.
* agent/command.c (cmd_delete_key): Add option --stub-only.
--

This option can be used to savely remove stub keys.
2017-03-24 09:02:30 +01:00
Werner Koch 26086b362f
common: Implicitly do a gpgconf --create-socketdir.
* common/homedir.c (_gnupg_socketdir_internal): Create the
sub-directory.
--

Although there is no auto cleanup (yet) this should be helpful.  Let's
see whether possibly leaving stale directories around is better than
running into trouble when --create-socketdir was not used.

Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-23 17:57:31 +01:00
Werner Koch 2c9d9ac55e
tests: Use gpgconf to stop the agent.
* tests/openpgp/defs.scm (stop-agent): Swap order of actions.  Kill
all daemons using gpgconf.
* tools/gpgconf.c (main) <aRemoveSocketDir>: Try to remove known
socketfails on rmdir failure.  Do no fail for ENONET.
--

Killing all daemons is not really required but it does not harm to be
prepared for the future.

Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-23 17:55:21 +01:00
Justus Winter 178b6314ab
gpgscm: Make test cleanup more robust.
* tests/gpgscm/tests.scm (mkdtemp-autoremove): New function that
cleans up at interpreter shutdown.
(run-tests-parallel): Use the new function.
(run-tests-sequential): Likewise.
(make-environment-cache): Execute setup with an temporary working
directory.
--

Make sure to remove all resources created in the filesystem even if
the test runner is interrupted.  Make sure to remove anything that the
setup script creates.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-23 15:56:34 +01:00
Justus Winter fde885bbc4
tests: Test '--quick-set-primary-uid'.
* tests/openpgp/quick-key-manipulation.scm: Test
'--quick-set-primary-uid'.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-21 16:21:49 +01:00
Justus Winter d17840c3f4
tests,w32: Use GetTempPath to get the path for temporary files.
* tests/gpgscm/ffi.c (do_get_temp_path): New function.
(ffi_init): Make function available.
* tests/gpgscm/tests.scm (mkdtemp): Use the new function.

Fixes-commit: 06f1f163e9
Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-21 15:52:47 +01:00
Werner Koch 74c1f30ad6
gpg: New command --quick-set-primary-uid.
* g10/gpg.c (aQuickSetPrimaryUid): New const.
(opts): New command --quick-set-primary-uid.
(main): Implement it.
* g10/keyedit.c (keyedit_quick_adduid): Factor some code out to ...
(quick_find_keyblock): new func.
(keyedit_quick_revuid): Use quick_find_keyblock.
(keyedit_quick_set_primary): New.

Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-21 15:09:30 +01:00
Justus Winter 483c1288a8
dirmngr: Fix error handling.
* dirmngr/dns-stuff.c (libdns_init): Convert error before printing it.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-21 14:22:13 +01:00
Justus Winter 88f1505f06
dirmngr: Load the hosts file into libdns.
* dirmngr/dns-stuff.c (libdns_init): Actually load the hosts file into
libdns.
--

Previously, connecting to key servers specified in /etc/hosts was not
possible because libdns' hosts structure was initialized, but not
filled with the content of the hosts file.

GnuPG-bug-id: 2977
Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-21 14:18:25 +01:00
Justus Winter 06f1f163e9
tests: Create temporary directories in '/tmp'.
* tests/gpgscm/tests.scm (mkdtemp): Create temporary directories in
'/tmp' on UNIX, or in '%Temp' on Windows.
* tests/migrations/common.scm (run-test): Turn error into a warning.
* tests/openpgp/defs.scm (start-agent): Likewise.
--

This fixes the problem of GnuPG components being unable to communicate
because of too long GnuPG home directories in important build
environments like the Debian build servers despite the use of socket
directories.

This reverts d75d20909d.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-21 13:15:38 +01:00
Justus Winter ceb4b24575
tests: Remove debugging remnants.
* tests/gpgme/gpgme-defs.scm (run-python-tests?): Remove 'trace's.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-20 14:36:36 +01:00
Justus Winter d75d20909d
tests: Fail if we cannot create the socket directory.
* tests/migrations/common.scm (run-test): Turn warning into an error.
* tests/openpgp/defs.scm (start-agent): Likewise.
--

We use separate directories to create the sockets in so that the
absolute path to the every socket fits into sun_path.

Fixes-commit: 7e19786a5d
Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-20 14:09:13 +01:00
Werner Koch fe0b37e123
gpg: Add new field no 18 to the colon listing.
* g10/misc.c (gnupg_pk_is_compliant): New.
* g10/keylist.c (print_compliance_flags): New.
(list_keyblock_colon): Call it here.
* sm/keylist.c (print_compliance_flags): New.
(list_cert_colon): Call it here.
--

This patch is to convey information about DE_VS compliant keys to the
caller.  The double digit value is used so that parsers do the right
thing and don't just look for a single digit.

Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-20 10:09:40 +01:00
Werner Koch e2c63a13e2
gpg: Remove unused stuff.
* g10/OPTIONS: Remove.
* g10/options.h (struct opt): Remove 'shm_coprocess'.

Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-20 08:40:21 +01:00
Neal H. Walfield fb9d68d636
tests: Add test for issue 2959.
* tests/openpgp/tofu.scm: Add test for --tofu-default-policy=ask.

Signed-off-by: Neal H. Walfield <neal@g10code.com>
2017-03-17 19:31:09 +01:00
Neal H. Walfield b1106b4d64 gpg: Make sure the conflict set includes the current key.
* g10/tofu.c (get_trust): Sanity check CONFLICT_SET after calling
get_policy.  If POLICY is 'auto' and the default policy is 'ask', make
sure CONFLICT_SET includes the current key.

--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
GnuPG-bug-id: 2959
Debian-bug-id: 854829

Signed-off-by: Neal H. Walfield <neal@g10code.com>
2017-03-17 14:35:00 +01:00
Werner Koch 69c521df42
dirmngr: Ignore warning alerts in the GNUTLS handshake.
* dirmngr/http.c (send_request) [GNUTLS]: Don't bail out on warning
alerts.
--

GnuPG-bug-id: 2833
Signed-off-by: Werner Koch <wk@gnupg.org>
2017-03-17 12:50:35 +01:00
Justus Winter 6a3f857224
gpgscm: Simplify hash tables.
* tests/gpgscm/scheme.c (oblist_add_by_name): We now always get a
slot.  Simplify accordingly.
(oblist_find_by_name): Always return the slot.
(vector_elem_slot): New function.
(new_slot_spec_in_env): We now always get a slot.  Remove parameter
'env'.  Simplify accordingly.
(find_slot_spec_in_env): Always return a slot.
(new_slot_in_env): Adapt callsite.
(opexe_0): Likewise.
(opexe_1): Likewise.
(scheme_define): Likewise.
--

Now that the ill-devised immediate values framework is gone, there is
no need to tag the pointers in vectors anymore.  Therefore, we can
always return a pointer to the slot in the hash table lookup
functions.

Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-17 10:35:25 +01:00
Justus Winter 38c955599f
gpgscm: Remove framework for immediate values.
* tests/gpgscm/scheme.c (IMMEDIATE_TAG): Remove macro.
(is_immediate): Likewise.
(set_immediate): Likewise.
(clr_immediate): Likewise.
(enum scheme_types): Set the LSB in every value.
(fill_vector): Adapt.
(vector_elem): Likewise.
(set_vector_elem): Likewise.
(mark): Likewise.
(gc): Test for the LSB to tell typeflags apart from pointers stored in
the same memory location.
--

Supporting immediate values would require invasive changes to the
interpreter and is likely not worth the trouble.  On the other hand,
tagging pointers in vectors complicated the hash table implementation
needlessly.  Therefore, I remove this again.

This fixes a crash on big endian architectures.

GnuPG-bug-id: 2996
Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-17 10:34:15 +01:00
NIIBE Yutaka 8c8ce8711d agent,g10: Remove redundant SERIALNO request.
* agent/learncard.c (agent_handle_learn): Don't call
agent_card_serialno.  Get the serialno in status response.
* g10/call-agent.c (agent_scd_learn): Don't request "SCD SERIALNO".
(agent_scd_serialno): New.
(card_cardlist_cb, agent_scd_cardlist): New.

Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
2017-03-16 14:32:51 +09:00
Justus Winter a98459d3f4
tests: Fix using tools from the build directory.
* tests/openpgp/defs.scm (gpg-conf'): Explicitly pass the build prefix
to gpgconf here...
(gpg-components): ... instead of only here.
--

Previously, gpgconf was not invoked with '--build-prefix' when
changing the configuration.  This made tests using this facility fail
(e.g. the TOFU test).  This only affected release builds, because in
development builds gpgconf picks up the build prefix from the
environment.

GnuPG-bug-id: 2979
Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-15 14:36:27 +01:00
Justus Winter c7833eca38
tests: Dump the tools that the tests are going to use.
* tests/openpgp/setup.scm: Dump the tools that the tests are going to
use.  This will help us diagnose problems with the tests picking the
wrong paths in the future.

GnuPG-bug-id: 2979
Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-15 12:34:04 +01:00
Justus Winter 6993e42088
build: Remove '--disable-tools' configuration option.
* Makefile.am (SUBDIRS): Unconditionally include 'tools'.
* configure.ac: Remove '--disable-tools' configuration option.
--
gpgconf is a core component nowadays and is always required.

GnuPG-bug-id: 2993
Signed-off-by: Justus Winter <justus@g10code.com>
2017-03-15 10:51:03 +01:00
NIIBE Yutaka 61785b679c g10: Fix check of serialno.
* g10/card-util.c (card_status): Fix.

Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
2017-03-15 16:50:48 +09:00
NIIBE Yutaka ed3248219e g10: Remove unused function.
* g10/call-agent.c (select_openpgp): Remove.

--

By this change, the function get_serialno_cb will be also unused.  But
please don't remove the function, because it will be soon used.

Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
2017-03-15 16:48:01 +09:00