* g10/keydb.h (KEYDB_HANDLE): Move typedef to ...
* g10/gpg.h: here.
(struct server_control_s): Add field 'cached_getkey_kdb'.
* g10/gpg.c (gpg_deinit_default_ctrl): Release that keydb handle.
* g10/getkey.c (getkey_end): Cache keydb handle.
(get_pubkey): Use cached keydb handle.
* kbx/keybox-search.c (keybox_search_reset): Use lseek instead of
closing the file.
--
Before this patch a "gpg --check-sigs" opened and closed the keybox
file for almost every signature check. By caching the keydb handle
and using lseek(2) this can be limited to just 2 times. This might
speed up things on Windows.
Signed-off-by: Werner Koch <wk@gnupg.org>
--
For proper operations as a server we need to avoid global variables.
Thus we need to pass the session state CTRL to most functions. Quite
a lot of changes but fortunately straightforward to do.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/parse-packet.c (parse_trust): Rename to ...
(parse_ring_trust): this. Change args and implement new ring trust
packet format.
(parse): Add special ring trust packet handling.
* g10/packet.h (PKT_user_id): New fields KEYUPDATE, UPDATEURL, and
KEYSRC.
(PKT_public_key): Ditto.
(RING_TRUST_SIG, RING_TRUST_KEY, RING_TRUST_UID): New consts.
(PKT_ring_trust): New.
(struct packet_struct): Remove member RING_TRUST.
(strcu parse_packet_ctx_s): Add field SKIP_META.
(init_parse_packet): Init SKIPT_META.
* g10/free-packet.c (release_public_key_parts): Free UDPATEURL.
(free_user_id): Ditto.
* g10/mainproc.c (list_node): Remove printing of non-documented "rtv"
lines.
* g10/build-packet.c (build_packet_and_meta): New.
(do_ring_trust): New.
* g10/export.c (write_keyblock_to_output): Use build_packet_and_meta
in backup mode.
(do_export_one_keyblock): Ditto.
* g10/import.c (read_block): Add arg WITH_META. Skip ring trust
packets if that ism not set.
(import): Call read_block WITH_META in restore mode.
* g10/keydb.h (KEYSRC_UNKNOWN, KEYSRC_FILE, KEYSRC_KS, KEYSRC_PREF_KS)
(KEYSRC_WKD, KEYSRC_WKD_SD, KEYSRC_DANE): New constants. They are not
yet used, though.
* g10/keydb.c (parse_keyblock_image): Allow ring trust packets.
(build_keyblock_image): Ditto. Use build_packet_and_meta.
* g10/keyring.c (keyring_get_keyblock): Remove specila treatment of
ring trust packets.
(write_keyblock): Use build_packet_and_meta. Remove special treatment
of ring trust packets and initialization of the signature caches.
--
This patch introduced the framework to store meta data for keys and
user ids in the keyrings/keyboxes. Ring trust packets are
implementation defined and have always been used in gpg to cache the
signature verification status.
Ring trust packets are only exported with the export option "backup"
and only imported with the import option "restore".
The new code uses a cleaner way to handle the ring trust packets: When
the parser reads a ring trust packet and the previously read packet
matches the type of that ring trust packet, the information is stored
in that previously read packet (signature, user id, or primary key)
and the next packet is read immediately. Thus only the parser sees
the ring trust packets. Ring trust packets are written by using the
new function build_packet_and_meta instead of build_packet. That
function writes a ring trust packet when the needed information is
available.
As a side-effect of this patch the signature status cache works again
and "gpg --check-sigs" is thus much faster.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c (initdb): Add column effective_policy to the bindings
table.
(record_binding): New parameters effective_policy and set_conflict.
Save the effective policy. If SET_CONFLICT is set, then set conflict
according to CONFLICT. Otherwise, preserve the current value of
conflict. Update callers.
(get_trust): Don't compute the effective policy here...
(get_policy): ... do it here, if it was not cached. Take new
parameters, PK, the public key, and NOW, the time that the operation
started. Update callers.
(show_statistics): New parameter PK. Pass it to get_policy. Update
callers.
(tofu_notice_key_changed): New function.
* g10/gpgv.c (tofu_notice_key_changed): New stub.
* g10/import.c (import_revoke_cert): Take additional argument CTRL.
Pass it to keydb_update_keyblock.
* g10/keydb.c (keydb_update_keyblock): Take additional argument CTRL.
Update callers.
[USE_TOFU]: Call tofu_notice_key_changed.
* g10/test-stubs.c (tofu_notice_key_changed): New stub.
* tests/openpgp/tofu.scm: Assume that manually setting a binding's
policy to auto does not cause the tofu engine to forget about any
conflict.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
We now store the computed policy in the tofu DB (in the
effective_policy column of the bindings table) to avoid computing it
every time, which is expensive. Further, policy is never overridden
in case of a conflict. Instead, we detect a conflict if CONFLICT is
not empty.
This change is backwards compatible to existing DBs. The only minor
incompatibility is that unresolved conflicts won't be automatically
resolved in case we import a direct signature, or cross signatures.
* g10/getkey.c (struct pubkey_cmp_cookie): New type.
(key_is_ok, uid_is_ok, subkey_is_ok): New functions.
(pubkey_cmp): Likewise.
(get_best_pubkey_byname): Likewise.
* g10/keydb.h (get_best_pubkey_byname): New prototype.
* g10/keylist.c (locate_one): Use the new function.
* g10/pkclist.c (find_and_check_key): Likewise.
* tests/openpgp/Makefile.am (XTESTS): Add new test.
(TEST_FILES): Add new files.
* tests/openpgp/key-selection.scm: New file.
* tests/openpgp/key-selection/0.asc: Likewise.
* tests/openpgp/key-selection/1.asc: Likewise.
* tests/openpgp/key-selection/2.asc: Likewise.
* tests/openpgp/key-selection/3.asc: Likewise.
* tests/openpgp/key-selection/4.asc: Likewise.
--
When a name resembling a mail address is given to either --locate-keys
or --recipient, rank the search results and use only the most relevant
key.
This also lets us query which key will be used for encryption using
--locate-keys. However, note that --locate-keys may also return keys
incapable of encryption, though it will prefer keys that have an
encryption subkey.
GnuPG-bug-id: 2359
Signed-off-by: Justus Winter <justus@g10code.com>
* g10/passphrase.c (passphrase_get): Remove arg KEYID. Change arg
MODE to NOCACHE.
(passphrase_to_dek): Remove args KEYID and PUBKEY_ALGO. Split arg
MODE into CREATE and NOCACHE. Change all callers and adjust stubs.
(passphrase_clear_cache): Remove args KEYID and ALGO. They are not
used. Change caller.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/gpg.c (oRecipientFile, oHiddenRecipientFile): New.
(opts): Add options --recipient-file and --hidden-recipient-file.
(main): Implement them. Also remove duplicate code from similar
options.
* g10/keydb.h (PK_LIST_FROM_FILE): New.
(PK_LIST_SHIFT): Bump up.
* g10/pkclist.c (expand_group): Take care of PK_LIST_FROM_FILE.
(find_and_check_key): Add and implement arg FROM_FILE.
(build_pk_list): Pass new value for new arg.
* g10/getkey.c (get_pubkey_fromfile): New.
* g10/gpgv.c (read_key_from_file): New stub.
* g10/test-stubs.c (read_key_from_file): New stub.
* g10/server.c (cmd_recipient): Add flag --file.
* g10/import.c (read_key_from_file): New.
* tests/openpgp/defs.scm (key-file1): New.
(key-file2): New.
* tests/openpgp/setup.scm: Add their private keys and import the
key-file1.
* tests/openpgp/encrypt.scm: Add new test.
--
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.h (keyid_cmp): New function.
* g10/keyid.c (pk_keyid): New function.
(pk_main_keyid): New function.
(keyid_copy): New function.
(pk_keyid_str): New function.
* g10/packet.h (PKT_public_key): Update comments for main_keyid and
keyid.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
Before accessing pk->keyid, it is necessary to call keyid_from_pk (pk,
NULL) to ensure that pk->keyid is valid. Because it is easy to forget
to do this, these accessor functions take care of it.
* g10/keydb.h: Remove agent_open, agent_close declarations/
* g10/migrate.c: #include <unistd.h> for access()
--
agent_open() is only defined statically in common/simple-pw-query.c,
it is neither used nor referenced anywhere else. agent_close doesn't
exist anywhere. The removal of these declarations removes an
unecessary inclusion of libassuan.h.
migrate.c was relying on keydb.h -> libassuan.h -> unistd.h for the
declaration of access(), so we now handle that explicitly instead.
* g10/keyid.c (format_keyid): Return a char *, not a const char *. If
BUFFER is NULL, then set LEN to the static buffer's size.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
--
When using tags (e.g. GNU global) to navigate the source code it is
way easier to have the documentation close to the function we are
looking at. Having the documentation in the header file would require
an extra manual lookup to understand the function.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/getkey.c: Allow arg RET_KEYBLOCK to be NULL.
--
This change adds the expected behavior for the getkey_next function
to fix this NULL de-ref.
GnuPG-bug-id: 2212
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.h (struct pubkey): Rename to pubkey_s.
(pubkey_t): New. Change all struct pubkey_s to use this type.
* g10/getkey.c (get_pubkeys): Rename arg keys to r_keys.
--
It is common in GnuPG to use a suffix of _s for struct names. There
is no technical need for this (actually this pattern comes from pre
ANSI C compilers which had no separate namespaces) but it avoid
surprises when reading the code.
Adding the pubkey_t type is mainly to improve font locking by using
the common suffix _t for a typedefed type.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/gpg.c (struct result): Move from here...
* g10/keydb.h (struct pubkey): ... to here. Update users.
* g10/gpg.c (check_user_ids): Move from here...
* g10/getkey.c (get_pubkeys): ... to here. Update users. Use
get_pubkey_byname to look up the keys (this also prunes invalid keys).
(pubkey_free): New function.
(pubkeys_free): New function.
* g10/gpg.c (main): Don't check for ambiguous key specifications.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
Regression-due-to: e8c53fc
This change not only moves the checks for ambiguous key specifications
from gpg.c to getkey.c, it also disables the checks. The old code was
too divorced from the actual key lookups and, as such, it reproduced
the logic. Unfortunately, the reproduction was a poor one: despite
fixing some inconsistencies (e.g., 10cca02), it still didn't deal with
group expansion or the auto key lookup functionality. Given the
amount of instability introduced by this change, we (Neal & Werner)
decided it is better to defer introducing this functionality until
2.3.
* g10/pkclist.c (build_pk_list): Remove parameter use, which is always
called set to PUBKEY_USAGE_ENC. Update callers.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/keydb.c (keydb_update_keyblock): Don't replace the record at the
current offset. After taking the lock, extract the fingerprint from
the keyblock, find it and then replace it.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
GnuPG-bug-id: 2193
Between locating the record to update and actually updating the
keyblock, it is possible that another process modifies the keyring,
which causes the update to corrupt the keyring. This is due to a time
of check to time of use bug. The fix is straightforward: both
operations must be done while holding the lock. This changes the
semantics of the function slightly, but no callers need to be
modified. Further, it now becomes impossible to replace key A with B;
this function will only ever update B.
* g10/keydb.h (PK_LIST_ENCRYPT_TO): Change from a macro to an enum.
(PK_LIST_HIDDEN): Likewise.
(PK_LIST_CONFIG): Likewise.
(PK_LIST_SHIFT): Likewise.n
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
Using an enum has the advantage that the symbol can be used in gdb.
* g10/keydb.h (PK_LIST_ENCRYPT_TO, PK_LIST_HIDDEN, PK_LIST_CONFIG)
(PK_LIST_SHIFT): New.
* g10/pkclist.c (build_pk_list): Use them here.
* g10/gpg.c (check_user_ids, main): Ditto.
* g10/gpg.c (main): Set PK_LIST_CONFIG for REMUSR and LOCUSR.
(check_user_ids): Skip check for command line specified options.
--
If a key has been given on the command line and it has not been
given by one of the encrypt-to options, we now skip the checks. The
reason is that the actual key selection code does its own checks and
provides proper status message to the caller to detect the wrong keys.
Without this we would break most frontends because they expect for
example STATUS_INV_RECP.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (keydb_new): Print an error message if needed. Also use
xtrycalloc because we return an error anyway.
* g10/delkey.c (do_delete_key): Handle error retruned by keydb_new.
* g10/export.c (do_export_stream): Ditto.
* g10/getkey.c (get_pubkey): Ditto.
(get_pubkey_fast): Ditto.
(get_pubkeyblock): Ditto.
(get_seckey): Ditto.
(key_byname): Ditto.
(get_pubkey_byfprint): Ditto.
(get_pubkey_byfprint_fast): Ditto.
(parse_def_secret_key): Ditto.
(have_secret_key_with_kid): Ditto.
* g10/import.c (import_one): Ditto.
(import_revoke_cert): Ditto.
* g10/keyedit.c (keyedit_quick_adduid): Ditto.
* g10/keygen.c (quick_generate_keypair): Ditto.
(do_generate_keypair): Ditto.
* g10/trustdb.c (validate_keys): Ditto.
* g10/keyserver.c (keyidlist): Ditto.
* g10/revoke.c (gen_desig_revoke): Ditto.
(gen_revoke): Ditto.
* g10/gpg.c (check_user_ids): Ditto.
(main): Do not print an error message for keydb_new error.
* g10/keylist.c (list_all): Use actual error code returned by
keydb_new.
* g10/t-keydb-get-keyblock.c (do_test): Abort on keydb_new error.
* g10/t-keydb.c (do_test): Ditto.
* g10/keyring.c (keyring_new): Actually return an error so that the
existing keydb_new error checking makes sense for a keyring resource.
(keyring_rebuild_cache): Take care of keyring_new returning an error.
--
Commit 04a6b903 changed keydb_new to return an error. However the
error was not checked at most places which we fix with this patch. To
make things easier keydb_new prints an error message itself.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (dump_search_desc): Rename from this...
(keydb_search_desc_dump): ... to this. Only process a single search
descriptor. Improve output. Don't mark as static. Update callers.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/options.h (opt.keyid_format): Add new value KF_DEFAULT.
* g10/keyid.c (format_keyid): New function.
(keystr): Use it.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/getkey.c: Improve documentation and comments for most
functions. Move documentation for public functions from here...
* g10/keydb.h: ... to here.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* g10/keydb.h (get_seckey_byname): Rename from this...
(get_seckey_default): ... to this. Drop the parameter name. Update
users.
* g10/getkey.c (get_seckey_byname): Rename from this...
(get_seckey_default): ... to this. Drop the parameter name. Drop the
code which assumed that NAME is not NULL.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* g10/keydb.h (get_keyblock_byfprint): Remove prototype. Replace use
of this function with get_pubkey_byfprint.
* g10/getkey.c (get_pubkey_byname): Remove function.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* g10/keydb.h (get_pubkey_end): Remove declaration. Replace use of
function with getkey_end.
* g10/getkey.c (get_pubkey_byname): Remove function.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* g10/keydb.c: Improve code comments and documentation of internal
interfaces. Improve documentation of public APIs and move that to...
* g10/keydb.h: ... this file.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* g10/keydb.c (struct kid_list_s): Rename from this...
(struct kid_not_found_cache_bucket): ... to this. Update users.
Remove field state.
(kid_list_t): Remove type.
(KID_NOT_FOUND_CACHE_BUCKETS): Define. Use this instead of a literal.
(kid_found_table): Rename from this...
(kid_not_found_cache_bucket): ... to this. Update users.
(kid_found_table_count): Rename from this...
(kid_not_found_cache_count): ... to this. Update users.
(kid_not_found_p): Only return whether a key with the specified key id
is definitely not in the database.
(kid_not_found_insert): Remove parameter found. Update callers.
(keydb_search): Only insert a key id in the not found cache if it is
not found. Rename local variable once_found to already_in_cache.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
Commit e0873a33 started tracking whether key ids where definitely in
the database. This information is, however, never used and thus just
unnecessarily inflates the cache. This patch effectively reverts that
change (however, e0873a33 contains two separate changes and this only
reverts that change).
* g10/keydb.h (KEYDB_RESOURCE_FLAG_GPGVDEF): New.
* g10/keydb.c (keydb_add_resource): Take care of new flag.
* g10/gpgv.c (main): Use new flag.
--
GnuPG-bug-id: 2025
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (kid_list_t): New.
(kid_not_found_table, n_kid_not_found_table): New.
(kid_not_found_p, kid_not_found_insert, kid_not_found_flush): New.
(keydb_insert_keyblock): Flush the new cache.
(keydb_delete_keyblock): Ditto.
(keydb_update_keyblock): Ditto.
(keydb_search): Use the new cache.
(keydb_dump_stats): New.
* g10/gpg.c (g10_exit): Dump keydb stats.
--
What we do here is to keep track of key searches by long keyids (as
stored in all signatures) so that we do not need to scan the keybox
again after we already found that this keyid will result in
not-found. As soon as we change gpg to run as a co-process we should
store this table per session because other instances of gpg may have
updated the keybox without us knowing.
On a test ring with
gpg: 94721 good signatures
gpg: 6831 bad signatures
gpg: 150703 signatures not checked due to missing keys
gpg: 5 signatures not checked due to errors
gpg: keydb: kid_not_found_table: total: 14132
this new cache speeds a --check-sigs listing up from 28 minutes to
less than 3 minutes.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (keydb_handle): Add field saved_found.
(keydb_new): Init new field.
(keydb_push_found_state, keydb_pop_found_state): New.
* g10/keyring.c (kyring_handle): Add field saved_found.
(keyring_push_found_state, keyring_pop_found_state): New.
--
We have the same feature in gpgsm. It is very useful to check for an
unambiguous user id with a follow up update of the keyblock.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keylist.c (print_pubkey_info): Print either "pub" or "sub".
* g10/getkey.c (get_pubkey_byfprint): Add optional arg R_KEYBLOCK.
* g10/keyid.c (keyid_from_fingerprint): Adjust for change.
* g10/revoke.c (gen_desig_revoke): Adjust for change.
* g10/card-util.c (card_status): Simplify by using new arg. Align
card-no string.
* g10/card-util.c (card_status): Remove not used GnuPG-1 code.
--
This now prints "sub" if the first used card key is actually a subkey.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/getkey.c (keyid_list): Add field "fpr".
(cache_user_id): Store fpr and check for dups only by fpr.
(get_pubkey_byfpr): New.
(get_user_id_string): Make static and use xasprintf.
(get_long_user_id_string): Use xasprintf.
(get_user_id_byfpr): New.
(get_user_id_byfpr_native): New.
* g10/keyid.c (fingerprint_from_pk): Make arg RET_LEN optional.
* g10/import.c (import_one): Use get_user_id_byfpr_native.
--
We now cache the userids using the fingerprint. This allows to print
the correct user id for keys with a duplicated key id. We should
eventually start to retire the use of all the old keyid based
functions. However, at some places we only have the keyid and thus
some of them will need to be kept (maybe changed with an indication to
show that more than several user ids are matching).
Signed-off-by: Werner Koch <wk@gnupg.org>
NIIBE Yutaka