--
We need to use es_fopen on Windows to cope with non-ascii file names.
This is quite a large but fortunately straightforward change. At a
very few places we keep using stdio (for example due to the use of
popen).
GnuPG-bug-id: 5098
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/sysutils.c (gnupg_access): New. Replace all calls to access
by this wrapper.
* common/homedir.c (w32_shgetfolderpath): Change to return UTF-8
directory name.
(standard_homedir): Adjust for change.
(w32_commondir, gnupg_cachedir): Ditto.
--
Also use SHGetFolderPathW instead of SHGetFolderPathA on Windows.
This is required to correctly handle non-ascii filenames on Windows.
GnuPG-bug-id: 5098
* kbx/kbx-client-util.c: New.
* kbx/kbx-client-util.h: New.
* kbx/Makefile.am (client_sources): New.
* g10/keydb.c (parse_keyblock_image): Rename to keydb_parse_keyblock
and make global.
* g10/call-keyboxd.c: Include kbx-client-util.h.
(struct keyboxd_local_s): Remove struct datastream. Add field kcd.
Remove per_session_init_done.
(lock_datastream, unlock_datastream): Remove.
(prepare_data_pipe, datastream_thread): Remove.
(keydb_get_keyblock_do_parse): Remove.
(gpg_keyboxd_deinit_session_data): Release the KCD object.
(open_context): Use of kbx_client_data_new.
(keydb_get_keyblock): Simplify.
(keydb_search): Use kbx_client_data_cmd and _wait.
--
The data specific part of the code has been moved from gpg to a new
module in kbx/ so that it can also be used by gpgsm. The OpenPGP
parsing while reading the data has been replaced by storing the data
in memory and parse it later. That makes a nice interface and
abstracts the fd-passing/D-lines handling away.
Signed-off-by: Werner Koch <wk@gnupg.org>
No functional changes, just fixing minor spelling issues.
---
Most of these were identified from the command line by running:
codespell \
--ignore-words-list fpr,stati,keyserver,keyservers,asign,cas,iff,ifset \
--skip '*.po,ChangeLog*,help.*.txt,*.jpg,*.eps,*.pdf,*.png,*.gpg,*.asc' \
doc g13 g10 kbx agent artwork scd tests tools am common dirmngr sm \
NEWS README README.maint TODO
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
* common/util.h (UBID_LEN): New. Use it at all places.
* kbx/keybox-blob.c (create_blob_finish): Do not write the UBID item.
* kbx/keybox-dump.c (print_ubib): Remove.
(_keybox_dump_blob): Do not print the now removed ubid flag.
* kbx/keybox-search-desc.h (struct keydb_search_desc): Use constants
for the size of the ubid and grip.
* kbx/keybox-search.c (blob_cmp_ubid): New.
(has_ubid): Make it a simple wrapper around blob_cmp_ubid.
(keybox_get_data): Add arg 'r_ubid'.
* kbx/frontend.h (enum kbxd_store_modes): New.
* kbx/kbxserver.c (cmd_store): Add new option --insert.
* kbx/backend-cache.c (be_cache_initialize): New.
(be_cache_add_resource): Call it here.
* kbx/backend-kbx.c (be_kbx_seek): Remove args 'fpr' and 'fprlen'.
(be_kbx_search): Get the UBID from keybox_get_data.
* kbx/backend-support.c (be_fingerprint_from_blob): Replace by ...
(be_ubid_from_blob): new. Change all callers.
* kbx/frontend.c (kbxd_add_resource): Temporary disable the cache but
use the new cache init function.
(kbxd_store): Replace arg 'only_update' by 'mode'. Seek using the
ubid. Take care of the mode.
--
It turned out that using the hash of the entire blob was not helpful.
Thus we redefine the Unique-Blob-ID (UBID) as the primary fingerprint
of the blob. In case this is a v5 OpenPGP key a left truncated
version of the SHA-256 hash is used; in all other cases the full SHA-1
hash. Using a SHA-256 hash does not make sense because v4 keys are
and will for some time be the majority of keys and thus padding them
with zeroes won't make any difference. Even if fingerprint collisions
can eventually be created we will assume that the keys are bogus and
that it does not make sense to store its twin also in our key storage.
We can also easily extend the update code to detect a collision and
reject the update.
Signed-off-by: Werner Koch <wk@gnupg.org>
--
Resolved Conflicts:
* common/asshelp.c: Keep the new code in master for spawing under
Windows.
* g10/Makefile.am: Keep all new file.
* g10/photoid.c: Pass CTRL to pct_expando.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/gpg.c (oUseKeyboxd,oKeyboxdProgram): New consts.
(opts): New options --use-keyboxd and --keyboxd-program.
(main): Implement them.
* g10/keydb.c: Move some defs out to ...
* g10/keydb-private.h: new file.
* g10/keydb.c: prefix function names with "internal" and move original
functions to ...
* g10/call-keyboxd.c: new file. Divert to the internal fucntion if
--use-keyboxd is used. Add a CTRL arg to most fucntions and change
all callers.
* g10/Makefile.am (common_source): Add new files.
(noinst_PROGRAMS): Do bot build gpgcompose.
--
Note that this is just the framework with only a basic implementation
of searching via keyboxd.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (keydb_add_resource): Call keybox_compress.
* sm/keydb.c (keydb_add_resource): Release the lock after a compress.
--
Note that in gpgsm we already did the compress run but we didn't
released the lock on the file. This might have been a reason for some
strange hangs.
GnuPG-bug-id: 4644
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (parse_keyblock_image): Treat invalid packet special.
--
This is in particular useful to run --list-keys on a keyring with
corrupted packets. The extra flush is to keep the diagnostic close to
the regular --list-key output.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-init.c (keybox_lock): New arg TIMEOUT. Change all
callers to pass -1 when locking.
* sm/keydb.c (struct resource_item): Remove LOCKANDLE.
(struct keydb_handle): Add KEEP_LOCK.
(keydb_add_resource): Use keybox locking instead of a separate dotlock
for testing whether we can run a compress.
(keydb_release): Reset KEEP_LOCK.
(keydb_lock): Set KEEP_LOCK.
(unlock_all): Take care of KEEP_LOCK.
(lock_all): Use keybox_lock instead of dotlock fucntions.
(keydb_delete): Remove arg UNLOCK.
* sm/delete.c (delete_one): Adjust keydb_delete. Due to the KEEP_LOCK
the keydb_release takes care of unlocking.
--
This aligns the code more with g10/keydb.c and avoids the separate
calls to dotlock_take.
GnuPG-bug-id: 4505
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-search-desc.h (KEYDB_SEARCH_MODE_FPR16)
(KEYDB_SEARCH_MODE_FPR20, KEYDB_SEARCH_MODE_FPR32): Remove. Switch
all users to KEYDB_SEARCH_MODE_FPR along with the fprlen value.
--
These search modes were added over time and there has until recently
be no incentive to remove the cruft. With the change for v5 keys I
finally went over all places and allowed the generic fingerprint mode
along with a given length of the fingerprint at all places.
Consequently the other modes can now be removed.
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/userids.c (classify_user_id): Support 32 byte fingerprints.
* kbx/keybox-search-desc.h (KEYDB_SEARCH_MODE_FPR32): New.
(struct keydb_search_desc): Add field fprlen.
* kbx/keybox-defs.h (struct _keybox_openpgp_key_info): Add field
version and increase size of fpr to 32.
* kbx/keybox-blob.c: Define new version 2 for PGP and X509 blobs.
(struct keyboxblob_key): Add field fprlen and increase size of fpr.
(pgp_create_key_part_single): Allow larger fingerprints.
(create_blob_header): Implement blob version 2 and add arg want_fpr32.
(_keybox_create_openpgp_blob): Detect the need for blob version 2.
* kbx/keybox-search.c (blob_get_first_keyid): Support 32 byte
fingerprints.
(blob_cmp_fpr): Ditto.
(blob_cmp_fpr_part): Ditto.
(has_fingerprint): Add arg fprlen and pass on.
(keybox_search): Support KEYDB_SEARCH_MODE_FPR32 and adjust for
changed has_fingerprint.
* kbx/keybox-openpgp.c (parse_key): Support version 5 keys.
* kbx/keybox-dump.c (_keybox_dump_blob): Support blob version 2.
* g10/delkey.c (do_delete_key): Support KEYDB_SEARCH_MODE_FPR32.
* g10/export.c (exact_subkey_match_p): Ditto.
* g10/gpg.c (main): Ditto.
* g10/getkey.c (get_pubkey_byfprint): Adjust for changed
KEYDB_SEARCH_MODE_FPR.
* g10/keydb.c (keydb_search_desc_dump): Support
KEYDB_SEARCH_MODE_FPR32 and adjust for changed KEYDB_SEARCH_MODE_FPR.
(keydb_search): Add new arg fprlen and change all callers.
* g10/keyedit.c (find_by_primary_fpr): Ditto.
* g10/keyid.c (keystr_from_desc): Ditto.
* g10/keyring.c (keyring_search): Ditto.
* g10/keyserver.c (print_keyrec): Ditto.
(parse_keyrec): Ditto.
(keyserver_export): Ditto.
(keyserver_retrieval_screener): Ditto.
(keyserver_import): Ditto.
(keyserver_import_fprint): Ditto.
(keyidlist): Ditto.
(keyserver_get_chunk): Ditto.
* g10/keydb.c (keydb_search): Add new arg fprlen and change all
callers.
* sm/keydb.c (keydb_search_fpr): Adjust for changed
KEYDB_SEARCH_MODE_FPR.
--
This prepares the support for OpenPGP v5 keys. The new version 2 blob
format is needed for the longer fingerprints and we also use this
opportunity to prepare for storing the keygrip in the blob for faster
lookup by keygrip. Right now this is not yet functional.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (parse_keyblock_image): Use log_info instead of
log_error for skipped packets.
* g10/keyring.c (keyring_get_keyblock): Ditto.
--
log_info should be sufficient and makes this more robust. Some
tools (e.g. Enigmail) are too picky on return codes from gpg.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (keydb_handle): New field 'keep_lock'.
(keydb_release): Clear that flag.
(keydb_lock): New function.
(unlock_all): Skip if KEEP_LOCK is set.
* g10/getkey.c (get_keyblock_byfprint_fast): Call keep_lock if
requested.
--
That change is straightforward. It helps to avoid the race condition
that another gpg process inserts a key while the first process is
between the search and the insert.
A similar change is due for gpgsm.
Note that the key edit operations may still suffer from a race.
GnuPG-bug-id: 3446
--
For proper operations as a server we need to avoid global variables.
Thus we need to pass the session state CTRL to most functions. Quite
a lot of changes but fortunately straightforward to do.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (keyblock_cache): Remove field SIGSTATUS.
(keyblock_cache_clear): Adjust for that removal.
(parse_keyblock_image): Remove arg SIGSTATUS. Remove the signature
cache setting; this is now done in the parser.
(keydb_get_keyblock): Do not set SIGSTATUS.
(build_keyblock_image): Remove arg SIGSTATUS and simplify. Change
caller.
* kbx/keybox-blob.c: Explain that the signature information is not
anymore used.
(_keybox_create_openpgp_blob): Remove arg SIGSTATUS and change
callers.
* kbx/keybox-search.c (keybox_get_keyblock): Remove arg R_SIGSTATUS
and change callers.
* kbx/keybox-update.c (keybox_insert_keyblock): Likewise.
--
This thing was too complicated and has been replaced by the new ring
trust packet code.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/parse-packet.c (parse_trust): Rename to ...
(parse_ring_trust): this. Change args and implement new ring trust
packet format.
(parse): Add special ring trust packet handling.
* g10/packet.h (PKT_user_id): New fields KEYUPDATE, UPDATEURL, and
KEYSRC.
(PKT_public_key): Ditto.
(RING_TRUST_SIG, RING_TRUST_KEY, RING_TRUST_UID): New consts.
(PKT_ring_trust): New.
(struct packet_struct): Remove member RING_TRUST.
(strcu parse_packet_ctx_s): Add field SKIP_META.
(init_parse_packet): Init SKIPT_META.
* g10/free-packet.c (release_public_key_parts): Free UDPATEURL.
(free_user_id): Ditto.
* g10/mainproc.c (list_node): Remove printing of non-documented "rtv"
lines.
* g10/build-packet.c (build_packet_and_meta): New.
(do_ring_trust): New.
* g10/export.c (write_keyblock_to_output): Use build_packet_and_meta
in backup mode.
(do_export_one_keyblock): Ditto.
* g10/import.c (read_block): Add arg WITH_META. Skip ring trust
packets if that ism not set.
(import): Call read_block WITH_META in restore mode.
* g10/keydb.h (KEYSRC_UNKNOWN, KEYSRC_FILE, KEYSRC_KS, KEYSRC_PREF_KS)
(KEYSRC_WKD, KEYSRC_WKD_SD, KEYSRC_DANE): New constants. They are not
yet used, though.
* g10/keydb.c (parse_keyblock_image): Allow ring trust packets.
(build_keyblock_image): Ditto. Use build_packet_and_meta.
* g10/keyring.c (keyring_get_keyblock): Remove specila treatment of
ring trust packets.
(write_keyblock): Use build_packet_and_meta. Remove special treatment
of ring trust packets and initialization of the signature caches.
--
This patch introduced the framework to store meta data for keys and
user ids in the keyrings/keyboxes. Ring trust packets are
implementation defined and have always been used in gpg to cache the
signature verification status.
Ring trust packets are only exported with the export option "backup"
and only imported with the import option "restore".
The new code uses a cleaner way to handle the ring trust packets: When
the parser reads a ring trust packet and the previously read packet
matches the type of that ring trust packet, the information is stored
in that previously read packet (signature, user id, or primary key)
and the next packet is read immediately. Thus only the parser sees
the ring trust packets. Ring trust packets are written by using the
new function build_packet_and_meta instead of build_packet. That
function writes a ring trust packet when the needed information is
available.
As a side-effect of this patch the signature status cache works again
and "gpg --check-sigs" is thus much faster.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/packet.h (struct parse_packet_ctx_s): Add fields LAST_PKT and
FREE_LAST_PKT.
(init_parse_packet): Clear them.
(deinit_parse_packet): New macro. Change all users if
init_parse_packet to also call this macro.
* g10/free-packet.c (free_packet): Add arg PARSECTX and handle shallow
packet copies in the context. Change all callers.
* g10/parse-packet.c (parse): Store certain packets in the parse
context.
--
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/packet.h (struct parse_packet_ctx_s): New.
(parse_packet_ctx_t): New type.
(init_parse_packet): New macro.
* g10/parse-packet.c (parse_packet, dbg_parse_packet): Change to take
a parse context. Change all callers to provide a context instead of
directly supplying the input stream.
(search_packet, dbg_search_packet): Ditto.
(copy_all_packets, dbg_copy_all_packets): Init an use a parse context.
(copy_some_packets, dbg_copy_some_packets): Ditto.
(skip_some_packets, dbg_skip_some_packets): Ditto.
--
We will need this change to handle ring packets inside the parser.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c (initdb): Add column effective_policy to the bindings
table.
(record_binding): New parameters effective_policy and set_conflict.
Save the effective policy. If SET_CONFLICT is set, then set conflict
according to CONFLICT. Otherwise, preserve the current value of
conflict. Update callers.
(get_trust): Don't compute the effective policy here...
(get_policy): ... do it here, if it was not cached. Take new
parameters, PK, the public key, and NOW, the time that the operation
started. Update callers.
(show_statistics): New parameter PK. Pass it to get_policy. Update
callers.
(tofu_notice_key_changed): New function.
* g10/gpgv.c (tofu_notice_key_changed): New stub.
* g10/import.c (import_revoke_cert): Take additional argument CTRL.
Pass it to keydb_update_keyblock.
* g10/keydb.c (keydb_update_keyblock): Take additional argument CTRL.
Update callers.
[USE_TOFU]: Call tofu_notice_key_changed.
* g10/test-stubs.c (tofu_notice_key_changed): New stub.
* tests/openpgp/tofu.scm: Assume that manually setting a binding's
policy to auto does not cause the tofu engine to forget about any
conflict.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
We now store the computed policy in the tofu DB (in the
effective_policy column of the bindings table) to avoid computing it
every time, which is expensive. Further, policy is never overridden
in case of a conflict. Instead, we detect a conflict if CONFLICT is
not empty.
This change is backwards compatible to existing DBs. The only minor
incompatibility is that unresolved conflicts won't be automatically
resolved in case we import a direct signature, or cross signatures.
* g10/keydb.c (keydb_add_resource): Make ANY_REGISTERED
file-global. Write a STATUS_ERROR.
(maybe_create_keyring_or_box): Check for non-accessible but existant
file.
(keydb_search): Write a STATUS_ERROR if no keyring has been registered
but continue to return NOT_FOUND.
* sm/keydb.c (keydb_add_resource): Rename ANY_PUBLIC to ANY_REGISTERED
and make file-global. Write a STATUS_ERROR.
(keydb_search): Write a STATUS_ERROR if no keyring has been registered
but continue to return NOT_FOUND. Also add new arg CTRL and change
all callers to pass it down.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (maybe_create_keyring_or_box): Open in binary mode.
--
This fixes keybox corruption on windows.
Signed-off-by: Andre Heinecke <aheinecke@intevation.de>
* g10/keydb.c (keydb_release): Clear keyblock cache.
(keydb_get_keyblock): Revert previous change.
* kbx/keybox-blob.c (create_blob_finish): Free previous buffer, free
fixups after applying them.
(_keybox_release_blob): Free buffer. Currently, the buffer has been
extracted before the keybox is released, but this is the right thing
to do here.
Fixes-commit: c57501cc
Signed-off-by: Justus Winter <justus@g10code.com>
* kbx/keybox-init.c (keybox_register_file): Change interface to return
the token even if the file has already been registered.
* g10/keydb.c (primary_keyring): Rename to primary_keydb.
(maybe_create_keyring_or_box): Change return type to gpg_error_t.
(keydb_add_resource): Ditto. s/rc/err/.
(keydb_add_resource): Mark an already registered as primary.
* sm/keydb.c (maybe_create_keybox): Change return type to gpg_error_t.
(keydb_add_resource): Ditto. s/rc/err/.
(keydb_add_resource): Adjust for changed keybox_register_file.
--
This change aligns the registering of keyboxes with those of
keyrings. This fixes a potential bug:
gpg --keyring foo.kbx --keyring bar.gpg --keyring foo.kbx
would have marked bar.gpg as primary resource and thus inserting new
keys there. The correct and now fixed behavior is to insert to
foo.kbx.
Signed-off-by: Werner Koch <wk@gnupg.org>
--
When using tags (e.g. GNU global) to navigate the source code it is
way easier to have the documentation close to the function we are
looking at. Having the documentation in the header file would require
an extra manual lookup to understand the function.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (keydb_update_keyblock): Don't replace the record at the
current offset. After taking the lock, extract the fingerprint from
the keyblock, find it and then replace it.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
GnuPG-bug-id: 2193
Between locating the record to update and actually updating the
keyblock, it is possible that another process modifies the keyring,
which causes the update to corrupt the keyring. This is due to a time
of check to time of use bug. The fix is straightforward: both
operations must be done while holding the lock. This changes the
semantics of the function slightly, but no callers need to be
modified. Further, it now becomes impossible to replace key A with B;
this function will only ever update B.
* kbx/keybox-search.c (keybox_seek): New function.
* g10/keydb.c (keydb_search): When reading from the cache, seek to
just after the cached record.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* kbx/keybox-search.c (keybox_offset): New function.
* g10/keydb.c (struct keyblock_cache): Add fields resource and offset.
(keyblock_cache_clear): Reset HD->KEYBLOCK_CACHE.RESOURCE and
HD->KEYBLOCK_CACHE.OFFSET.
(keydb_search): Don't use the cached result if it comes before the
current file position. When caching an entry, also record the
position at which it was found.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
GnuPG-bug-id: 2187
* g10/keydb.c (keydb_search_desc_dump): Fix buffer overflow.
--
Signed-off-by: Neal H. Walfield <neal@walfield.org>
Regression-due-to: 11ec478
Note: this buffer overflow will only occur if --debug=lookup is passed
to gpg and a search by fingerprint is done.
* g10/keydb.c (keydb_new): Print an error message if needed. Also use
xtrycalloc because we return an error anyway.
* g10/delkey.c (do_delete_key): Handle error retruned by keydb_new.
* g10/export.c (do_export_stream): Ditto.
* g10/getkey.c (get_pubkey): Ditto.
(get_pubkey_fast): Ditto.
(get_pubkeyblock): Ditto.
(get_seckey): Ditto.
(key_byname): Ditto.
(get_pubkey_byfprint): Ditto.
(get_pubkey_byfprint_fast): Ditto.
(parse_def_secret_key): Ditto.
(have_secret_key_with_kid): Ditto.
* g10/import.c (import_one): Ditto.
(import_revoke_cert): Ditto.
* g10/keyedit.c (keyedit_quick_adduid): Ditto.
* g10/keygen.c (quick_generate_keypair): Ditto.
(do_generate_keypair): Ditto.
* g10/trustdb.c (validate_keys): Ditto.
* g10/keyserver.c (keyidlist): Ditto.
* g10/revoke.c (gen_desig_revoke): Ditto.
(gen_revoke): Ditto.
* g10/gpg.c (check_user_ids): Ditto.
(main): Do not print an error message for keydb_new error.
* g10/keylist.c (list_all): Use actual error code returned by
keydb_new.
* g10/t-keydb-get-keyblock.c (do_test): Abort on keydb_new error.
* g10/t-keydb.c (do_test): Ditto.
* g10/keyring.c (keyring_new): Actually return an error so that the
existing keydb_new error checking makes sense for a keyring resource.
(keyring_rebuild_cache): Take care of keyring_new returning an error.
--
Commit 04a6b903 changed keydb_new to return an error. However the
error was not checked at most places which we fix with this patch. To
make things easier keydb_new prints an error message itself.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (dump_search_desc): Rename from this...
(keydb_search_desc_dump): ... to this. Only process a single search
descriptor. Improve output. Don't mark as static. Update callers.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
