* kbx/kbx-client-util.c: New.
* kbx/kbx-client-util.h: New.
* kbx/Makefile.am (client_sources): New.
* g10/keydb.c (parse_keyblock_image): Rename to keydb_parse_keyblock
and make global.
* g10/call-keyboxd.c: Include kbx-client-util.h.
(struct keyboxd_local_s): Remove struct datastream. Add field kcd.
Remove per_session_init_done.
(lock_datastream, unlock_datastream): Remove.
(prepare_data_pipe, datastream_thread): Remove.
(keydb_get_keyblock_do_parse): Remove.
(gpg_keyboxd_deinit_session_data): Release the KCD object.
(open_context): Use of kbx_client_data_new.
(keydb_get_keyblock): Simplify.
(keydb_search): Use kbx_client_data_cmd and _wait.
--
The data specific part of the code has been moved from gpg to a new
module in kbx/ so that it can also be used by gpgsm. The OpenPGP
parsing while reading the data has been replaced by storing the data
in memory and parse it later. That makes a nice interface and
abstracts the fd-passing/D-lines handling away.
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/gc-opt-flags.h (GC_OPT_FLAG_RUNTIME): Move to ...
* tools/gpgconf-comp.c: here.
(known_options_scdaemon): Remove "options".
(known_options_dirmngr): Remove "options".
(known_options_gpgsm): Remove "options".
(known_options_gpg): Remove "options" and "keyserver".
(struct gc_option_s): Rename active t gpgconf_list.
(gc_component_list_options): Do not act upon active.
(option_check_validity): Ditto.
(is_known_option): Make it work correctly for unknown options.
(retrieve_options_from_program): Use renamed flag gpgconf_list only to
detect duplicated items from --gpgconf-list. Do not set runtime.
Only e set the options if set by --gpgconf-list; never clear them.
* agent/gpg-agent.c: Simplify the --gpgconf-list output.
* dirmngr/dirmngr.c: Ditto.
* g10/gpg.c: Ditto.
* kbx/keyboxd.c: Ditto.
* scd/scdaemon.c: Ditto.
* sm/gpgsm.c: Ditto.
* tests/openpgp/gpgconf.scm: Use "compliance" instead of "keyserver"
for the string arg test.
--
There is no need to read the list of options from the components
unless they convey a default value. It is better to consult only the
list we have in gpgconf-comp.c to decide on whether an option should
be displayed. Right, this might mess up thing if a newer gpgconf
version is used with an older component, but we already print warnings
in this case and in general we do not want to support this anymore -
the times of gpg 1.4. and 2.0 are long over now.
GnuPG-bug-id: 4788
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/util.h: Remove argparse.h.
* common/argparse.c: Undef GPGRT_ENABLE_ARGPARSE_MACROS.
* configure.ac (GPGRT_ENABLE_ARGPARSE_MACROS): Define.
* agent/gpg-agent.c: Undef GPGRT_ENABLE_ARGPARSE_MACROS and include
argparse.h. Do this also for all main modules which use our option
parser except for gpg. Replace calls to strusage by calls to
gpgrt_strusage everywhere.
* g10/gpg.c (opts): Change type to gpgrt_opt_t. Flag oOptions and
oNoOptions with ARGPARSE_conffile and ARGPARSE_no_conffile.
(main): Change type of pargs to gpgrt_argparse_t. Rework the option
parser to make use of the new gpgrt_argparser.
--
This is not yet finished but a make check works. gpg has the most
complex and oldest option handling and thus this is the first
migration target. SE-Linux checks and version-ed config files are
missing and will be added later.
GnuPG-bug-id: 4788
Signed-off-by: Werner Koch <wk@gnupg.org>
No functional changes, just fixing minor spelling issues.
---
Most of these were identified from the command line by running:
codespell \
--ignore-words-list fpr,stati,keyserver,keyservers,asign,cas,iff,ifset \
--skip '*.po,ChangeLog*,help.*.txt,*.jpg,*.eps,*.pdf,*.png,*.gpg,*.asc' \
doc g13 g10 kbx agent artwork scd tests tools am common dirmngr sm \
NEWS README README.maint TODO
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
* configure.ac: Add option --dsiable-keyboxd
* kbx/Makefile.am: Do not build keyboxd in that case.
--
This is useful to build a minimal version of gpgv.
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/util.h (EXTERN_UNLESS_MAIN_MODULE): Add the definion only
here but now without the Norcroft-C. Change all other places where it
gets defined.
* common/iobuf.h (iobuf_debug_mode): Declare unconditionally as
extern.
* common/iobuf.c (iobuf_debug_mode): Define it here.
* agent/gpg-agent.c (INCLUDED_BY_MAIN_MODULE): Define here and also in
all main modules of all other programs.
* g10/main.h: Put util.h before the local header files.
--
This change is required for use with gcc/ld's LTO feature which does
not allow common blocks. Further gcc 10 will make -fno-common the
default and thus this chnage is always needed. What a pitty.
Co-authored-by: Tomáš Mráz
GnuPG-bug-id: 4831
Signed-off-by: Werner Koch <wk@gnupg.org>
(cherry picked from commit 21d9bd8b87a9f793a106095e3838eb71825189d7)
- Applied respective chnages also to gpg-card and keyboxd.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/backend-sqlite.c: New.
* kbx/Makefile.am (keyboxd_SOURCES): Add it.
(keyboxd_CFLAGS, keyboxd_LDADD): Add SQLite flags.
* kbx/backend.h (enum database_types): Add DB_TYPE_SQLITE.
(be_sqlite_local_t): New typedef.
(struct db_request_part_s): Add field besqlite.
* kbx/backend-support.c (strdbtype): Add string for DB_TYPE_SQLITE.
(be_generic_release_backend): Support SQLite.
(be_release_request): Ditto.
(be_find_request_part): Ditto.
(is_x509_blob): Rename to ...
(be_is_x509_blob): this and make global.
* kbx/frontend.c (kbxd_set_database): Detect ".db" suffix and use that
for SQLite.
(kbxd_search): Support SQLite
(kbxd_store): Ditto.
(kbxd_delete): Ditto.
* kbx/frontend.h (kbxd_store_modes): Move to ...
* kbx/keyboxd.h (enum kbxd_store_modes): here.
* kbx/keyboxd.c (main): USe pubring.db for now. This is a temporary
hack.
* kbx/backend-kbx.c (be_kbx_delete): Remove unused var cert.
--
Take care: This is not finished and in particular filling the database
takes quite long.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/frontend.c (the_database): New var.
(db_desc_t): Remove.
(kbxd_add_resource): Renamed to ...
(kbxd_set_database): this. Simplify.
(kbxd_search): Change to use only one database.
(kbxd_store): Ditto.
(kbxd_delete): Ditto.
--
The original implementation was way to complicated and would have only
brought back the problems deciding which database to use for each key.
The new scheme used one configured database and only that. That
database needs to be set right at the start.
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/util.h (UBID_LEN): New. Use it at all places.
* kbx/keybox-blob.c (create_blob_finish): Do not write the UBID item.
* kbx/keybox-dump.c (print_ubib): Remove.
(_keybox_dump_blob): Do not print the now removed ubid flag.
* kbx/keybox-search-desc.h (struct keydb_search_desc): Use constants
for the size of the ubid and grip.
* kbx/keybox-search.c (blob_cmp_ubid): New.
(has_ubid): Make it a simple wrapper around blob_cmp_ubid.
(keybox_get_data): Add arg 'r_ubid'.
* kbx/frontend.h (enum kbxd_store_modes): New.
* kbx/kbxserver.c (cmd_store): Add new option --insert.
* kbx/backend-cache.c (be_cache_initialize): New.
(be_cache_add_resource): Call it here.
* kbx/backend-kbx.c (be_kbx_seek): Remove args 'fpr' and 'fprlen'.
(be_kbx_search): Get the UBID from keybox_get_data.
* kbx/backend-support.c (be_fingerprint_from_blob): Replace by ...
(be_ubid_from_blob): new. Change all callers.
* kbx/frontend.c (kbxd_add_resource): Temporary disable the cache but
use the new cache init function.
(kbxd_store): Replace arg 'only_update' by 'mode'. Seek using the
ubid. Take care of the mode.
--
It turned out that using the hash of the entire blob was not helpful.
Thus we redefine the Unique-Blob-ID (UBID) as the primary fingerprint
of the blob. In case this is a v5 OpenPGP key a left truncated
version of the SHA-256 hash is used; in all other cases the full SHA-1
hash. Using a SHA-256 hash does not make sense because v4 keys are
and will for some time be the majority of keys and thus padding them
with zeroes won't make any difference. Even if fingerprint collisions
can eventually be created we will assume that the keys are bogus and
that it does not make sense to store its twin also in our key storage.
We can also easily extend the update code to detect a collision and
reject the update.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox.h: Fix naming of arguments.
* scd/ccid-driver.c (print_error) [TEST]: Add missing break. Note
that this is anyway an impossible case.
Signed-off-by: Werner Koch <wk@gnupg.org>
--
Resolved Conflicts:
* common/asshelp.c: Keep the new code in master for spawing under
Windows.
* g10/Makefile.am: Keep all new file.
* g10/photoid.c: Pass CTRL to pct_expando.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/frontend.c (kbxd_add_resource): Print a diagnostic on error.
* kbx/backend-kbx.c (be_kbx_add_resource): Acttually returh the error
code.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-blob.c (create_blob_header): New blob flag UBIB.
(create_blob_finish): Write the UBIB.
* kbx/keybox-dump.c (print_ubib): New.
(_keybox_dump_blob): Print UBIB flag.
* kbx/keybox-search.c (has_ubid): Compare the stored UBIB if
available.
--
This make scanning the keybox for a given UBIB much faster once it has
been stored.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/backend.h (enum database_types): Add DB_TYPE_CACHE.
(struct db_request_part_s): Add seqno fields.
(struct db_request_s): Add infos for the cache backend.
* kbx/backend-support.c (struct backend_handle_s): Add 'backend_id'.
(strdbtype): Support DB_TYPE_CACHE.
(be_generic_release_backend): Ditto.
(be_find_request_part): New.
(be_return_pubkey): New arg UBID and chnage status name.
* kbx/backend-cache.c: New.
* kbx/backend-kbx.c (be_kbx_init_request_part): New.
(be_kbx_search): Factor some code out to a support function.
(be_kbx_seek): New.
* kbx/frontend.c (kbxd_add_resource): Support DB_TYPE_CACHE.
(kbxd_search): Support the NEXR operation with the cache.
* kbx/keybox-search-desc.h (KEYDB_SEARCH_MODE_UBID): New.
(struct keydb_search_desc): Add field u.ubid.
* kbx/keybox-search.c (has_ubid): New.
(keybox_search): Support the UBID search.
--
This adds a caching backend to the keyboxd. This tries to accommodate
for duplicate use of fingerprints and thus be correct in case a
fingerprint is used in several keys. It also turned out that we need
to have a unique identifier (UBID) to identify a keyblock or X.509
certificate. In particular with an OpenPGP keyblob we can't easily
use the primary fingerprint as an identifier because that fingerprint
may also be used as subkey in another key. Thus using a hash of the
entire keyblock is a better identifier to be used to address a
keyblock for restarting a search or for identifying the keyblock to be
updated. Note that this new UBID is not a permanent identifier
because it changes with all keyblock update; it should be viewed as a
handle to the keyblock or X509 cert.
* kbx/keybox-dump.c (_keybox_dump_file): Take deleted records in
account.
--
This also changes the numbering of the records to reflect the real
record number.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-update.c (keybox_compress): Change condition back.
Also use make_timestamp for CUT_TIME.
--
Fixes-commit: 824ca6f042dc69edaf67bf9d4e875be75babab00
Note that the original change was not backported to 2.2.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-search.c (keybox_search): We need to seek to the last
position in all cases not just when doing a NEXT.
--
This is because search from the beginning needs a keybox_search_reset.
We can only make an exception for KEYDB_SEARCH_MODE_FIRST..
Fixes-commit: 49b236af0ecbb6df67513feb4b63851f2e159ea2
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-init.c (keybox_lock) [W32]: Use _keybox_close_file
instead of fclose so that a close is done if the file is opened by
another handle.
* kbx/keybox-search.c (keybox_search): Remember the last offset and
use that in NEXT search mode if we had to re-open the file.
--
GnuPG-bug-id: 4505
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-init.c (keybox_lock): New arg TIMEOUT. Change all
callers to pass -1 when locking.
* sm/keydb.c (struct resource_item): Remove LOCKANDLE.
(struct keydb_handle): Add KEEP_LOCK.
(keydb_add_resource): Use keybox locking instead of a separate dotlock
for testing whether we can run a compress.
(keydb_release): Reset KEEP_LOCK.
(keydb_lock): Set KEEP_LOCK.
(unlock_all): Take care of KEEP_LOCK.
(lock_all): Use keybox_lock instead of dotlock fucntions.
(keydb_delete): Remove arg UNLOCK.
* sm/delete.c (delete_one): Adjust keydb_delete. Due to the KEEP_LOCK
the keydb_release takes care of unlocking.
--
This aligns the code more with g10/keydb.c and avoids the separate
calls to dotlock_take.
GnuPG-bug-id: 4505
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-search-desc.h (KEYDB_SEARCH_MODE_FPR16)
(KEYDB_SEARCH_MODE_FPR20, KEYDB_SEARCH_MODE_FPR32): Remove. Switch
all users to KEYDB_SEARCH_MODE_FPR along with the fprlen value.
--
These search modes were added over time and there has until recently
be no incentive to remove the cruft. With the change for v5 keys I
finally went over all places and allowed the generic fingerprint mode
along with a given length of the fingerprint at all places.
Consequently the other modes can now be removed.
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/userids.c (classify_user_id): Support 32 byte fingerprints.
* kbx/keybox-search-desc.h (KEYDB_SEARCH_MODE_FPR32): New.
(struct keydb_search_desc): Add field fprlen.
* kbx/keybox-defs.h (struct _keybox_openpgp_key_info): Add field
version and increase size of fpr to 32.
* kbx/keybox-blob.c: Define new version 2 for PGP and X509 blobs.
(struct keyboxblob_key): Add field fprlen and increase size of fpr.
(pgp_create_key_part_single): Allow larger fingerprints.
(create_blob_header): Implement blob version 2 and add arg want_fpr32.
(_keybox_create_openpgp_blob): Detect the need for blob version 2.
* kbx/keybox-search.c (blob_get_first_keyid): Support 32 byte
fingerprints.
(blob_cmp_fpr): Ditto.
(blob_cmp_fpr_part): Ditto.
(has_fingerprint): Add arg fprlen and pass on.
(keybox_search): Support KEYDB_SEARCH_MODE_FPR32 and adjust for
changed has_fingerprint.
* kbx/keybox-openpgp.c (parse_key): Support version 5 keys.
* kbx/keybox-dump.c (_keybox_dump_blob): Support blob version 2.
* g10/delkey.c (do_delete_key): Support KEYDB_SEARCH_MODE_FPR32.
* g10/export.c (exact_subkey_match_p): Ditto.
* g10/gpg.c (main): Ditto.
* g10/getkey.c (get_pubkey_byfprint): Adjust for changed
KEYDB_SEARCH_MODE_FPR.
* g10/keydb.c (keydb_search_desc_dump): Support
KEYDB_SEARCH_MODE_FPR32 and adjust for changed KEYDB_SEARCH_MODE_FPR.
(keydb_search): Add new arg fprlen and change all callers.
* g10/keyedit.c (find_by_primary_fpr): Ditto.
* g10/keyid.c (keystr_from_desc): Ditto.
* g10/keyring.c (keyring_search): Ditto.
* g10/keyserver.c (print_keyrec): Ditto.
(parse_keyrec): Ditto.
(keyserver_export): Ditto.
(keyserver_retrieval_screener): Ditto.
(keyserver_import): Ditto.
(keyserver_import_fprint): Ditto.
(keyidlist): Ditto.
(keyserver_get_chunk): Ditto.
* g10/keydb.c (keydb_search): Add new arg fprlen and change all
callers.
* sm/keydb.c (keydb_search_fpr): Adjust for changed
KEYDB_SEARCH_MODE_FPR.
--
This prepares the support for OpenPGP v5 keys. The new version 2 blob
format is needed for the longer fingerprints and we also use this
opportunity to prepare for storing the keygrip in the blob for faster
lookup by keygrip. Right now this is not yet functional.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-defs.h (struct _keybox_openpgp_key_info): Add field grip.
* kbx/keybox-openpgp.c (struct keyparm_s): New.
(keygrip_from_keyparm): New.
(parse_key): Compute keygrip.
* kbx/keybox-search.c (blob_openpgp_has_grip): New.
(has_keygrip): Call it.
--
This has been marked for too long as not yet working. However, it is
a pretty useful feature and will come pretty handy when looking for
all keys matching one keygrip.
Can be optimized a lot by storing the keygrip in the meta data. This
will be done along with the upgrade of KBX for v5 fingerprints.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-search-desc.h (fpr): Increase the size.
--
In the function keydb_search_fpr in g10/keydb.c, it is copied using
MAX_FINGERPRINT_LEN. So, more size is required.
Fixes-commit: ecbbafb88d920e713439b6b1b8e1b41a6f8d0e38
Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
* kbx/keybox-search.c (blob_cmp_fpr): Avoid overflow in OFF+LEN
checking.
(blob_cmp_fpr_part): Ditto.
(blob_cmp_name): Ditto.
(blob_cmp_mail): Ditto.
(blob_x509_has_grip): Ditto.
(keybox_get_keyblock): Check OFF and LEN using a 64 bit var.
(keybox_get_cert): Ditto.
--
On most 32 bit systems size_t is 32 bit and thus the check
size_t cert_off = get32 (buffer+8);
size_t cert_len = get32 (buffer+12);
if (cert_off+cert_len > length)
return gpg_error (GPG_ERR_TOO_SHORT);
does not work as intended for all supplied values. The simplest
solution here is to cast them to 64 bit.
In general it will be better to avoid size_t at all and work with
uint64_t. We did not do this in the past because uint64_t was not
universally available.
GnuPG-bug-id: 3770
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-util.c (keybox_set_malloc_hooks): Remove.
(_keybox_malloc, _keybox_calloc, keybox_realloc)
(_keybox_free): Remove.
(keybox_file_rename): Remove. Was not used.
* sm/gpgsm.c (main): Remove call to keybox_set_malloc_hooks.
* kbx/kbxutil.c (main): Ditto.
* kbx/keybox-defs.h: Remove all separate includes. Include util.h.
remove convenience macros.
* common/logging.h (return_if_fail): New. Originally from
keybox-defs.h but now using log_debug.
(return_null_if_fail): Ditto.
(return_val_if_fail): Ditto.
(never_reached): Ditto.
--
Originally the KBX code was written to allow standalone use. However
this required lot of ugliness like separate memory allocators and
such. It also precludes the use of some standard functions from
common due to their use of the common gnupg malloc functions.
Dropping all that makes things easier. Minor disadvantages: the kbx
call done for gpg will now use gcry malloc fucntions and not the
standard malloc functions. This might be a bit slower but removing
them even fixes a possible bug in keybox_tmp_names which is used in
gpg and uses gpg's xfree which is actually gcry_free.
Signed-off-by: Werner Koch <wk@gnupg.org>
