* g10/keydb.c (parse_keyblock_image): Use log_info instead of
log_error for skipped packets.
* g10/keyring.c (keyring_get_keyblock): Ditto.
--
log_info should be sufficient and makes this more robust. Some
tools (e.g. Enigmail) are too picky on return codes from gpg.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (keydb_handle): New field 'keep_lock'.
(keydb_release): Clear that flag.
(keydb_lock): New function.
(unlock_all): Skip if KEEP_LOCK is set.
* g10/getkey.c (get_keyblock_byfprint_fast): Call keep_lock if
requested.
--
That change is straightforward. It helps to avoid the race condition
that another gpg process inserts a key while the first process is
between the search and the insert.
A similar change is due for gpgsm.
Note that the key edit operations may still suffer from a race.
GnuPG-bug-id: 3446
--
For proper operations as a server we need to avoid global variables.
Thus we need to pass the session state CTRL to most functions. Quite
a lot of changes but fortunately straightforward to do.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (keyblock_cache): Remove field SIGSTATUS.
(keyblock_cache_clear): Adjust for that removal.
(parse_keyblock_image): Remove arg SIGSTATUS. Remove the signature
cache setting; this is now done in the parser.
(keydb_get_keyblock): Do not set SIGSTATUS.
(build_keyblock_image): Remove arg SIGSTATUS and simplify. Change
caller.
* kbx/keybox-blob.c: Explain that the signature information is not
anymore used.
(_keybox_create_openpgp_blob): Remove arg SIGSTATUS and change
callers.
* kbx/keybox-search.c (keybox_get_keyblock): Remove arg R_SIGSTATUS
and change callers.
* kbx/keybox-update.c (keybox_insert_keyblock): Likewise.
--
This thing was too complicated and has been replaced by the new ring
trust packet code.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/parse-packet.c (parse_trust): Rename to ...
(parse_ring_trust): this. Change args and implement new ring trust
packet format.
(parse): Add special ring trust packet handling.
* g10/packet.h (PKT_user_id): New fields KEYUPDATE, UPDATEURL, and
KEYSRC.
(PKT_public_key): Ditto.
(RING_TRUST_SIG, RING_TRUST_KEY, RING_TRUST_UID): New consts.
(PKT_ring_trust): New.
(struct packet_struct): Remove member RING_TRUST.
(strcu parse_packet_ctx_s): Add field SKIP_META.
(init_parse_packet): Init SKIPT_META.
* g10/free-packet.c (release_public_key_parts): Free UDPATEURL.
(free_user_id): Ditto.
* g10/mainproc.c (list_node): Remove printing of non-documented "rtv"
lines.
* g10/build-packet.c (build_packet_and_meta): New.
(do_ring_trust): New.
* g10/export.c (write_keyblock_to_output): Use build_packet_and_meta
in backup mode.
(do_export_one_keyblock): Ditto.
* g10/import.c (read_block): Add arg WITH_META. Skip ring trust
packets if that ism not set.
(import): Call read_block WITH_META in restore mode.
* g10/keydb.h (KEYSRC_UNKNOWN, KEYSRC_FILE, KEYSRC_KS, KEYSRC_PREF_KS)
(KEYSRC_WKD, KEYSRC_WKD_SD, KEYSRC_DANE): New constants. They are not
yet used, though.
* g10/keydb.c (parse_keyblock_image): Allow ring trust packets.
(build_keyblock_image): Ditto. Use build_packet_and_meta.
* g10/keyring.c (keyring_get_keyblock): Remove specila treatment of
ring trust packets.
(write_keyblock): Use build_packet_and_meta. Remove special treatment
of ring trust packets and initialization of the signature caches.
--
This patch introduced the framework to store meta data for keys and
user ids in the keyrings/keyboxes. Ring trust packets are
implementation defined and have always been used in gpg to cache the
signature verification status.
Ring trust packets are only exported with the export option "backup"
and only imported with the import option "restore".
The new code uses a cleaner way to handle the ring trust packets: When
the parser reads a ring trust packet and the previously read packet
matches the type of that ring trust packet, the information is stored
in that previously read packet (signature, user id, or primary key)
and the next packet is read immediately. Thus only the parser sees
the ring trust packets. Ring trust packets are written by using the
new function build_packet_and_meta instead of build_packet. That
function writes a ring trust packet when the needed information is
available.
As a side-effect of this patch the signature status cache works again
and "gpg --check-sigs" is thus much faster.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/packet.h (struct parse_packet_ctx_s): Add fields LAST_PKT and
FREE_LAST_PKT.
(init_parse_packet): Clear them.
(deinit_parse_packet): New macro. Change all users if
init_parse_packet to also call this macro.
* g10/free-packet.c (free_packet): Add arg PARSECTX and handle shallow
packet copies in the context. Change all callers.
* g10/parse-packet.c (parse): Store certain packets in the parse
context.
--
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/packet.h (struct parse_packet_ctx_s): New.
(parse_packet_ctx_t): New type.
(init_parse_packet): New macro.
* g10/parse-packet.c (parse_packet, dbg_parse_packet): Change to take
a parse context. Change all callers to provide a context instead of
directly supplying the input stream.
(search_packet, dbg_search_packet): Ditto.
(copy_all_packets, dbg_copy_all_packets): Init an use a parse context.
(copy_some_packets, dbg_copy_some_packets): Ditto.
(skip_some_packets, dbg_skip_some_packets): Ditto.
--
We will need this change to handle ring packets inside the parser.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c (initdb): Add column effective_policy to the bindings
table.
(record_binding): New parameters effective_policy and set_conflict.
Save the effective policy. If SET_CONFLICT is set, then set conflict
according to CONFLICT. Otherwise, preserve the current value of
conflict. Update callers.
(get_trust): Don't compute the effective policy here...
(get_policy): ... do it here, if it was not cached. Take new
parameters, PK, the public key, and NOW, the time that the operation
started. Update callers.
(show_statistics): New parameter PK. Pass it to get_policy. Update
callers.
(tofu_notice_key_changed): New function.
* g10/gpgv.c (tofu_notice_key_changed): New stub.
* g10/import.c (import_revoke_cert): Take additional argument CTRL.
Pass it to keydb_update_keyblock.
* g10/keydb.c (keydb_update_keyblock): Take additional argument CTRL.
Update callers.
[USE_TOFU]: Call tofu_notice_key_changed.
* g10/test-stubs.c (tofu_notice_key_changed): New stub.
* tests/openpgp/tofu.scm: Assume that manually setting a binding's
policy to auto does not cause the tofu engine to forget about any
conflict.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
We now store the computed policy in the tofu DB (in the
effective_policy column of the bindings table) to avoid computing it
every time, which is expensive. Further, policy is never overridden
in case of a conflict. Instead, we detect a conflict if CONFLICT is
not empty.
This change is backwards compatible to existing DBs. The only minor
incompatibility is that unresolved conflicts won't be automatically
resolved in case we import a direct signature, or cross signatures.
* g10/keydb.c (keydb_add_resource): Make ANY_REGISTERED
file-global. Write a STATUS_ERROR.
(maybe_create_keyring_or_box): Check for non-accessible but existant
file.
(keydb_search): Write a STATUS_ERROR if no keyring has been registered
but continue to return NOT_FOUND.
* sm/keydb.c (keydb_add_resource): Rename ANY_PUBLIC to ANY_REGISTERED
and make file-global. Write a STATUS_ERROR.
(keydb_search): Write a STATUS_ERROR if no keyring has been registered
but continue to return NOT_FOUND. Also add new arg CTRL and change
all callers to pass it down.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (maybe_create_keyring_or_box): Open in binary mode.
--
This fixes keybox corruption on windows.
Signed-off-by: Andre Heinecke <aheinecke@intevation.de>
* g10/keydb.c (keydb_release): Clear keyblock cache.
(keydb_get_keyblock): Revert previous change.
* kbx/keybox-blob.c (create_blob_finish): Free previous buffer, free
fixups after applying them.
(_keybox_release_blob): Free buffer. Currently, the buffer has been
extracted before the keybox is released, but this is the right thing
to do here.
Fixes-commit: c57501cc
Signed-off-by: Justus Winter <justus@g10code.com>
* kbx/keybox-init.c (keybox_register_file): Change interface to return
the token even if the file has already been registered.
* g10/keydb.c (primary_keyring): Rename to primary_keydb.
(maybe_create_keyring_or_box): Change return type to gpg_error_t.
(keydb_add_resource): Ditto. s/rc/err/.
(keydb_add_resource): Mark an already registered as primary.
* sm/keydb.c (maybe_create_keybox): Change return type to gpg_error_t.
(keydb_add_resource): Ditto. s/rc/err/.
(keydb_add_resource): Adjust for changed keybox_register_file.
--
This change aligns the registering of keyboxes with those of
keyrings. This fixes a potential bug:
gpg --keyring foo.kbx --keyring bar.gpg --keyring foo.kbx
would have marked bar.gpg as primary resource and thus inserting new
keys there. The correct and now fixed behavior is to insert to
foo.kbx.
Signed-off-by: Werner Koch <wk@gnupg.org>
--
When using tags (e.g. GNU global) to navigate the source code it is
way easier to have the documentation close to the function we are
looking at. Having the documentation in the header file would require
an extra manual lookup to understand the function.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (keydb_update_keyblock): Don't replace the record at the
current offset. After taking the lock, extract the fingerprint from
the keyblock, find it and then replace it.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
GnuPG-bug-id: 2193
Between locating the record to update and actually updating the
keyblock, it is possible that another process modifies the keyring,
which causes the update to corrupt the keyring. This is due to a time
of check to time of use bug. The fix is straightforward: both
operations must be done while holding the lock. This changes the
semantics of the function slightly, but no callers need to be
modified. Further, it now becomes impossible to replace key A with B;
this function will only ever update B.
* kbx/keybox-search.c (keybox_seek): New function.
* g10/keydb.c (keydb_search): When reading from the cache, seek to
just after the cached record.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* kbx/keybox-search.c (keybox_offset): New function.
* g10/keydb.c (struct keyblock_cache): Add fields resource and offset.
(keyblock_cache_clear): Reset HD->KEYBLOCK_CACHE.RESOURCE and
HD->KEYBLOCK_CACHE.OFFSET.
(keydb_search): Don't use the cached result if it comes before the
current file position. When caching an entry, also record the
position at which it was found.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
GnuPG-bug-id: 2187
* g10/keydb.c (keydb_search_desc_dump): Fix buffer overflow.
--
Signed-off-by: Neal H. Walfield <neal@walfield.org>
Regression-due-to: 11ec478
Note: this buffer overflow will only occur if --debug=lookup is passed
to gpg and a search by fingerprint is done.
* g10/keydb.c (keydb_new): Print an error message if needed. Also use
xtrycalloc because we return an error anyway.
* g10/delkey.c (do_delete_key): Handle error retruned by keydb_new.
* g10/export.c (do_export_stream): Ditto.
* g10/getkey.c (get_pubkey): Ditto.
(get_pubkey_fast): Ditto.
(get_pubkeyblock): Ditto.
(get_seckey): Ditto.
(key_byname): Ditto.
(get_pubkey_byfprint): Ditto.
(get_pubkey_byfprint_fast): Ditto.
(parse_def_secret_key): Ditto.
(have_secret_key_with_kid): Ditto.
* g10/import.c (import_one): Ditto.
(import_revoke_cert): Ditto.
* g10/keyedit.c (keyedit_quick_adduid): Ditto.
* g10/keygen.c (quick_generate_keypair): Ditto.
(do_generate_keypair): Ditto.
* g10/trustdb.c (validate_keys): Ditto.
* g10/keyserver.c (keyidlist): Ditto.
* g10/revoke.c (gen_desig_revoke): Ditto.
(gen_revoke): Ditto.
* g10/gpg.c (check_user_ids): Ditto.
(main): Do not print an error message for keydb_new error.
* g10/keylist.c (list_all): Use actual error code returned by
keydb_new.
* g10/t-keydb-get-keyblock.c (do_test): Abort on keydb_new error.
* g10/t-keydb.c (do_test): Ditto.
* g10/keyring.c (keyring_new): Actually return an error so that the
existing keydb_new error checking makes sense for a keyring resource.
(keyring_rebuild_cache): Take care of keyring_new returning an error.
--
Commit 04a6b903 changed keydb_new to return an error. However the
error was not checked at most places which we fix with this patch. To
make things easier keydb_new prints an error message itself.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (dump_search_desc): Rename from this...
(keydb_search_desc_dump): ... to this. Only process a single search
descriptor. Improve output. Don't mark as static. Update callers.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/keydb.c (keydb_rebuild_caches): Only mark the cached as prepared
if it is actually prepared, which it only is if the resource is a
keybox.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/keydb.c (keyblock_cache): Don't declare this variable. Instead...
(struct keyblock_cache): ... turn its type into this first class
object...
(struct keydb_handle): ... and instantiate it once per database
handle. Update all users.
(keydb_rebuild_caches): Don't invalidate the keyblock cache.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* g10/keydb.c (keydb_get_keyblock): If the iobuf_seek fails when
reading from the cache, then simply clear the cache and try reading
from the database.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* g10/keydb.c (keydb_new): If we fail to open a keyring or keybox
correctly release all resources.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* g10/keydb.c: Improve code comments and documentation of internal
interfaces. Improve documentation of public APIs and move that to...
* g10/keydb.h: ... this file.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* g10/keydb.c (struct keydb_handle): Add new field is_reset.
(keydb_new): Initialize hd->is_reset to 1.
(keydb_locate_writable): Set hd->is_reset to 1.
(keydb_search): Set hd->is_reset to 0. Don't cache a key not found if
the search started from the beginning of the database.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* g10/keydb.c (keydb_search_first): Reset the handle before starting
the search.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
This bug hasn't shown up yet in practice, because keydb_search_first
is always called immediately after a keydb_new. This changes cleans
up the semantics and will hopefully prevent future bugs.
* g10/keydb.c (struct kid_list_s): Rename from this...
(struct kid_not_found_cache_bucket): ... to this. Update users.
Remove field state.
(kid_list_t): Remove type.
(KID_NOT_FOUND_CACHE_BUCKETS): Define. Use this instead of a literal.
(kid_found_table): Rename from this...
(kid_not_found_cache_bucket): ... to this. Update users.
(kid_found_table_count): Rename from this...
(kid_not_found_cache_count): ... to this. Update users.
(kid_not_found_p): Only return whether a key with the specified key id
is definitely not in the database.
(kid_not_found_insert): Remove parameter found. Update callers.
(keydb_search): Only insert a key id in the not found cache if it is
not found. Rename local variable once_found to already_in_cache.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
Commit e0873a33 started tracking whether key ids where definitely in
the database. This information is, however, never used and thus just
unnecessarily inflates the cache. This patch effectively reverts that
change (however, e0873a33 contains two separate changes and this only
reverts that change).
* g10/keydb.h (KEYDB_RESOURCE_FLAG_GPGVDEF): New.
* g10/keydb.c (keydb_add_resource): Take care of new flag.
* g10/gpgv.c (main): Use new flag.
--
GnuPG-bug-id: 2025
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (keydb_search_fpr): Skip legacy keys.
--
A test case for this problem can be found at
GnuPG-bug-id: 2031
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (kid_list_s): Keep a state in the table.
(kid_not_found_table): Rename to kid_found_table.
(n_kid_not_found_table): Rename to kid_found_table_count.
(kid_not_found_p): Return found state.
(kid_not_found_insert): Add arg found.
(keydb_search): Store found state in the table.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (kid_list_t): New.
(kid_not_found_table, n_kid_not_found_table): New.
(kid_not_found_p, kid_not_found_insert, kid_not_found_flush): New.
(keydb_insert_keyblock): Flush the new cache.
(keydb_delete_keyblock): Ditto.
(keydb_update_keyblock): Ditto.
(keydb_search): Use the new cache.
(keydb_dump_stats): New.
* g10/gpg.c (g10_exit): Dump keydb stats.
--
What we do here is to keep track of key searches by long keyids (as
stored in all signatures) so that we do not need to scan the keybox
again after we already found that this keyid will result in
not-found. As soon as we change gpg to run as a co-process we should
store this table per session because other instances of gpg may have
updated the keybox without us knowing.
On a test ring with
gpg: 94721 good signatures
gpg: 6831 bad signatures
gpg: 150703 signatures not checked due to missing keys
gpg: 5 signatures not checked due to errors
gpg: keydb: kid_not_found_table: total: 14132
this new cache speeds a --check-sigs listing up from 28 minutes to
less than 3 minutes.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (keydb_handle): Add field saved_found.
(keydb_new): Init new field.
(keydb_push_found_state, keydb_pop_found_state): New.
* g10/keyring.c (kyring_handle): Add field saved_found.
(keyring_push_found_state, keyring_pop_found_state): New.
--
We have the same feature in gpgsm. It is very useful to check for an
unambiguous user id with a follow up update of the keyblock.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keyring.c (keyring_get_keyblock): Whitelist allowed packet
types.
* g10/keydb.c (parse_keyblock_image): Ditto.
--
The keyring DB code did not reject packets which don't belong into a
keyring. If for example the keyblock contains a literal data packet
it is expected that the processing code stops at the data packet and
reads from the input stream which is referenced from the data packets.
Obviously the keyring processing code does not and cannot do that.
However, when exporting this messes up the IOBUF and leads to an
invalid read of sizeof (int).
We now skip all packets which are not allowed in a keyring.
Reported-by: Hanno Böck <hanno@hboeck.de>
Test data:
gpg2 --no-default-keyring --keyring FILE --export >/dev/null
With this unpacked data for FILE:
-----BEGIN PGP ARMORED FILE-----
mI0EVNP2zQEEALvETPVDCJDBXkegF4esiV1fqlne40yJnCmJeDEJYocwFPXfFA86
sSGjInzgDbpbC9gQPwq91Qe9x3Vy81CkyVonPOejhINlzfpzqAAa3A6viJccZTwt
DJ8E/I9jg53sbYW8q+VgfLn1hlggH/XQRT0HkXMP5y9ClURYnTsNwJhXABEBAAGs
CXRlc3QgdGVzdIi5BBMBCgAjBQJU0/bNAhsDBwsJCAcDAgEGFQgCCQoLBBYCAwEC
HgECF4AACgkQlsmuCapsqYLvtQP/byY0tM0Lc3moftbHQZ2eHj9ykLjsCjeMDfPx
kZUUtUS3HQaqgZLZOeqPjM7XgGh5hJsd9pfhmRWJ0x+iGB47XQNpRTtdLBV/WMCS
l5z3uW7e9Md7QVUVuSlJnBgQHTS6EgP8JQadPkAiF+jgpJZXP+gFs2j3gobS0qUF
eyTtxs+wAgAD
=uIt9
-----END PGP ARMORED FILE-----
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (keydb_search_first, keydb_search_next): Skip legacy
keys.
* g10/keyring.c (keyring_get_keyblock): Handle GPG_ERR_LEGACY_KEY.
(prepare_search): Ditto.
(keyring_rebuild_cache): Skip legacy keys.
* g10/keyserver.c (keyidlist): Ditto.
* g10/trustdb.c (validate_key_list): Ditto.
--
This is not the most elegant way to handle it but it reduces the
chance for unwanted side effects.
GnuPG-bug-id: 1816
Signed-off-by: Werner Koch <wk@gnupg.org>
