* kbx/keybox-defs.h (KEYBOX_LL_OPEN_READ)
(KEYBOX_LL_OPEN_UPDATE, KEYBOX_LL_OPEN_CREATE): New.
* kbx/keybox-init.c (_keybox_ll_open): New. Replace all keybox use of
es_fopen by this function.
(_keybox_ll_close): New. Replace all keybox use of es_fclose by this
function.
--
Note that this has not been done for the utilities and the backend-kbx
of keyboxd.
* kbx/keybox-search.c (open_file): Use sysopen and sequential.
* kbx/keybox-update.c (create_tmp_file): Ditto.
(blob_filecopy): Ditto.
(keybox_set_flags): Ditto.
(keybox_delete): Ditto.
(keybox_compress): Ditto.
--
Under Windows "sysopen" requests that direct API calls (CreateFile et
al.) are used instead of the libc wrappers. This may or may not
improve the performance.
Using "sequential" is a hint to Windows to assume that a file is in
general access in a sequential manner. This will have an affect only
with a future libgpg-error.
* kbx/keybox-search.c (keybox_search_reset) [W32]: Always close.
* kbx/keybox-init.c (keybox_close_all_files): New.
* sm/keydb.c (keydb_close_all_files): New.
* sm/call-dirmngr.c (gpgsm_dirmngr_isvalid): Call new function.
(gpgsm_dirmngr_lookup): Ditto.
(gpgsm_dirmngr_run_command): Ditto.
--
We need to make sure that there are no open files on Windows. Thus we
close them at several strategic locations.
GnuPG-bug-id: 4505
--
We need to use es_fopen on Windows to cope with non-ascii file names.
This is quite a large but fortunately straightforward change. At a
very few places we keep using stdio (for example due to the use of
popen).
GnuPG-bug-id: 5098
Signed-off-by: Werner Koch <wk@gnupg.org>
Backported-from-master: 390497ea115e1aca93feec297a5bd6ae7b1ba6dd
* kbx/keybox-search.c (keybox_search): We need to seek to the last
position in all cases not just when doing a NEXT.
--
This is because search from the beginning needs a keybox_search_reset.
We can only make an exception for KEYDB_SEARCH_MODE_FIRST..
Fixes-commit: 6f72aa821407e47ad3963e72e139f2ca2c69d9dd
GnuPG-bug-id: 4505
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-init.c (keybox_lock) [W32]: Use _keybox_close_file
instead of fclose so that a close is done if the file is opened by
another handle.
* kbx/keybox-search.c (keybox_search): Remember the last offset and
use that in NEXT search mode if we had to re-open the file.
--
GnuPG-bug-id: 4505
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-defs.h (struct _keybox_openpgp_key_info): Add field grip.
* kbx/keybox-openpgp.c (struct keyparm_s): New.
(keygrip_from_keyparm): New.
(parse_key): Compute keygrip.
* kbx/keybox-search.c (blob_openpgp_has_grip): New.
(has_keygrip): Call it.
--
This has been marked for too long as not yet working. However, it is
a pretty useful feature and will come pretty handy when looking for
all keys matching one keygrip.
Can be optimized a lot by storing the keygrip in the meta data. This
will be done along with the upgrade of KBX for v5 fingerprints.
Signed-off-by: Werner Koch <wk@gnupg.org>
(cherry picked from commit c128667b3cba749dd14262e032d4c260a2b0acd3)
* kbx/keybox-search.c (blob_cmp_fpr): Avoid overflow in OFF+LEN
checking.
(blob_cmp_fpr_part): Ditto.
(blob_cmp_name): Ditto.
(blob_cmp_mail): Ditto.
(blob_x509_has_grip): Ditto.
(keybox_get_keyblock): Check OFF and LEN using a 64 bit var.
(keybox_get_cert): Ditto.
--
On most 32 bit systems size_t is 32 bit and thus the check
size_t cert_off = get32 (buffer+8);
size_t cert_len = get32 (buffer+12);
if (cert_off+cert_len > length)
return gpg_error (GPG_ERR_TOO_SHORT);
does not work as intended for all supplied values. The simplest
solution here is to cast them to 64 bit.
In general it will be better to avoid size_t at all and work with
uint64_t. We did not do this in the past because uint64_t was not
universally available.
GnuPG-bug-id: 3770
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-file.c (_keybox_read_blob): Remove.
(_keybox_read_blob2): Rename to ....
(_keybox_read_blob): this. Make arg options. Change all callers.
* kbx/keybox-search.c (keybox_search): Factor fopen call out to ...
(open_file): new.
(keybox_seek): Als use open_file.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.h (KEYDB_HANDLE): Move typedef to ...
* g10/gpg.h: here.
(struct server_control_s): Add field 'cached_getkey_kdb'.
* g10/gpg.c (gpg_deinit_default_ctrl): Release that keydb handle.
* g10/getkey.c (getkey_end): Cache keydb handle.
(get_pubkey): Use cached keydb handle.
* kbx/keybox-search.c (keybox_search_reset): Use lseek instead of
closing the file.
--
Before this patch a "gpg --check-sigs" opened and closed the keybox
file for almost every signature check. By caching the keydb handle
and using lseek(2) this can be limited to just 2 times. This might
speed up things on Windows.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (keyblock_cache): Remove field SIGSTATUS.
(keyblock_cache_clear): Adjust for that removal.
(parse_keyblock_image): Remove arg SIGSTATUS. Remove the signature
cache setting; this is now done in the parser.
(keydb_get_keyblock): Do not set SIGSTATUS.
(build_keyblock_image): Remove arg SIGSTATUS and simplify. Change
caller.
* kbx/keybox-blob.c: Explain that the signature information is not
anymore used.
(_keybox_create_openpgp_blob): Remove arg SIGSTATUS and change
callers.
* kbx/keybox-search.c (keybox_get_keyblock): Remove arg R_SIGSTATUS
and change callers.
* kbx/keybox-update.c (keybox_insert_keyblock): Likewise.
--
This thing was too complicated and has been replaced by the new ring
trust packet code.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-search.c (keybox_seek): New function.
* g10/keydb.c (keydb_search): When reading from the cache, seek to
just after the cached record.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* kbx/keybox-search.c (keybox_offset): New function.
* g10/keydb.c (struct keyblock_cache): Add fields resource and offset.
(keyblock_cache_clear): Reset HD->KEYBLOCK_CACHE.RESOURCE and
HD->KEYBLOCK_CACHE.OFFSET.
(keydb_search): Don't use the cached result if it comes before the
current file position. When caching an entry, also record the
position at which it was found.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
GnuPG-bug-id: 2187
* common/util.h: Remove replacement macros for libgpg-error<1.21.
* common/types.h: Ditto.
* common/mischelp.h: Ditto.
* common/t-mapstrings.c: Include t-support.h before stringhelp.h
* common/t-stringhelp.c: Ditto.
* common/t-support.h: Always include gpg-error.h.
* kbx/keybox-search.c: Do not include stringhelp.h so that keybox-defs
comes first.
--
This patch enhances commit d6e0149 from Dec 10.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-search-desc.h (struct keydb_search_desc.skipfnc): Change
third parameter to be the index of the user id packet in the keyblock
rather than the packet itself. Update users.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
The keybox code doesn't work directly with keyblocks. As such, the
matched user packet is not readily available to pass to
DESC[n].SKIPFNC. But, we do know the index of the user id packet that
matched. Thus, pass that instead. If the skip function needs the
user id packet, it can use the key id to look up the key block and
find the appropriate packet.
* kbx/keybox.h (KEYBOX_WITH_X509): Do not define.
* sm/Makefile.am (AM_CPPFLAGS): Define it here.
(common_libs): Change to libkeybox509.a
* g10/Makefile.am (AM_CFLAGS): remove KSBA_CFLAGS.
(gpg2_LDADD, gpgv2_LDADD): Remove KSBA_LIBS
* kbx/Makefile.am (noinst_LIBRARIES): Add libkeybox509.a.
(libkeybox509_a_SOURCES): New.
(libkeybox_a_CFLAGS): New.
(libkeybox509_a_CFLAGS): New.
(kbxutil_CFLAGS): New.
* kbx/keybox-search.c (has_keygrip) [!KEYBOX_WITH_X509]: Declare args
as unused.
--
There is no real need to link to Libksba in gpg.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-search.c (blob_cmp_mail): Stop comparing at the '>'.
--
This change allows to find mail addresses like
Joe Doe <joe@example.org> bar
Joe Doe <joe@example.org> (comment)
using the command
gpg -k '<joe@example.org'
or (with syntactic sugar)
gpg -k '<joe@example.org>'
These UIDs are ill-formed according to gpg checks but nevertheless are
seen in the wild.
Note, that it does only work with the new keybox format.
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/host2net.h (buf16_to_ulong, buf16_to_uint): New.
(buf16_to_ushort, buf16_to_u16): New.
(buf32_to_size_t, buf32_to_ulong, buf32_to_uint, buf32_to_u32): New.
--
Commit 91b826a38880fd8a989318585eb502582636ddd8 was not enough to
avoid all sign extension on shift problems. Hanno Böck found a case
with an invalid read due to this problem. To fix that once and for
all almost all uses of "<< 24" and "<< 8" are changed by this patch to
use an inline function from host2net.h.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-search.c (blob_get_keyid): Rename to
blob_get_first_keyid. Check number of keys and remove blob type check.
--
There is no need to check the blob type. We already know that it is a
key blob type and keyids are used for X.509 and OpenPGP. Also added
check for number of keys because the other parser functions do it as
well.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-search.c (blob_get_keyid): New.
(keybox-search): Call skipfnc callback function.
--
This patch (tentatively) fixes
GnuPG-bug-id: 1794
The keybox_search function in kbx/keybox-search.c currently ignores
the skipfnc callback, but the validate_key_list function in
g10/trustdb.c uses such a callback to exclude ultimately trusted keys.
* kbx/keybox.h (keybox_blobtype_t): New.
* kbx/keybox-defs.h (BLOBTYPE_*): Replace by KEYBOX_BLOBTYPE_*.
* kbx/keybox-search.c (keybox_search): Add arg want_blobtype and skip
non-matching blobs.
* sm/keydb.c (keydb_search): Pass KEYBOX_BLOBTYPE_X509 to keybox_search.
* g10/keydb.c (keydb_search): Pass KEYBOX_BLOBTYPE_PGP to keybox_search.
--
Without this fix a listing of all keys would fail because the wrong
blob type would be returned for the gpg or gpgsm.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-search.c (keybox_search): Add arg r_skipped and skip too
long blobs.
* sm/keydb.c (keydb_search): Call keybox_search with a dummy param.
* g10/keydb.c (struct keydb_handle): Add field skipped_long_blobs.
(keydb_search_reset): Reset that field.
(keydb_search): Update that field.
(keydb_get_skipped_counter): New.
* g10/keylist.c (list_all): Print count of skipped keys.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-search.c (keybox_search): Add arg R_DESCINDEX. Chnage
both callers.
* g10/keydb.c (keydb_search): Always set DESCINDEX.
--
This only affects the new keybox for OpenPGP keys in 2.1. The bug
exhibited itself by running GPA's backup command on Windows.
* kbx/keybox-update.c (keybox_update_keyblock): Implement.
* kbx/keybox-search.c (get_blob_flags): Move to ...
* kbx/keybox-defs.h (blob_get_type): here.
* kbx/keybox-file.c (_keybox_read_blob2): Fix calling without R_BLOB.
* g10/keydb.c (build_keyblock_image): Allow calling without
R_SIGSTATUS.
(keydb_update_keyblock): Implement for keybox.
* kbx/keybox-dump.c (_keybox_dump_blob): Fix printing of the unhashed
size. Print "does not expire" also on 64 bit platforms.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb.c (parse_keyblock_image): Add args PK_NO and UID_NO and
set the note flags accordingly.
(keydb_get_keyblock): Transfer PK_NO and UID_NO to parse_keyblock_image.
* kbx/keybox-search.c (blob_cmp_fpr, blob_cmp_fpr_part)
(blob_cmp_name, blob_cmp_mail): Return the key/user number.
(keybox_search): Set the key and user number into the found struct.
(keybox_get_keyblock): Add args R_PK_NO and R_UID_NO and set them from
the found struct.
--
getkey.c needs to know whether the correct subkey was found. Thus we
need to set the node flags the same way we did it with the keyring
storage.
* g10/keydb.c (parse_keyblock_image): Add arg SIGSTATUS.
(keydb_get_keyblock): Handle it.
(build_keyblock_image): Add arg SIGSTATUS.
(keydb_insert_keyblock): Handle it.
* kbx/keybox-blob.c (pgp_create_sig_part): Add arg SIGSTATUS.
(_keybox_create_openpgp_blob): Ditto.
* kbx/kbxutil.c (import_openpgp): Adjust for above change.
* kbx/keybox.h (KEYBOX_FLAG_SIG_INFO): New.
* kbx/keybox-search.c (_keybox_get_flag_location): Handle new flag.
(keybox_get_keyblock): Add arg R_SIGSTATUS.
* kbx/keybox-update.c (keybox_insert_keyblock): Add arg SIGSTATUS.
--
With this change a key listing using the keybox format is now double
as fast as using a keyring. The memory use dropped as well. Measured
with about 1500 keys.
* g10/getkey.c (get_pubkey_fast): Improve the assertion.
* kbx/keybox.h: Include iobuf.h.
* kbx/keybox-blob.c (keyboxblob_uid): Add field OFF.
(KEYBOX_WITH_OPENPGP): Remove use of this macro.
(pgp_create_key_part_single): New.
(pgp_temp_store_kid): Change to use the keybox-openpgp parser.
(pgp_create_key_part): Ditto.
(pgp_create_uid_part): Ditto.
(pgp_create_sig_part): Ditto.
(pgp_create_blob_keyblock): Ditto.
(_keybox_create_openpgp_blob): Ditto.
* kbx/keybox-search.c (keybox_get_keyblock): New.
* kbx/keybox-update.c (keybox_insert_keyblock): New.
* g10/keydb.c (parse_keyblock_image):
(keydb_get_keyblock): Support keybox.
(build_keyblock_image): New.
(keydb_insert_keyblock): Support keybox.
* kbx/kbxutil.c (import_openpgp, main): Add option --dry-run and print
a kbx file to stdout.
* kbx/keybox-file.c (_keybox_read_blob2): Allow keyblocks up to 10^6
bytes.
--
Import and key listing does now work with the keybox format. It is
still quite slow and signature caching is completely missing.
Increasing the maximum allowed length for a keyblock was required due
to a 700k keyblock which inhibited kbxutil to list the file.
kbxutil's option name --import-openpgp is not quite appropriate
because it only creates KBX blobs from OpenPGP data.
We better do this once and for all instead of cluttering all future
commits with diffs of trailing white spaces. In the majority of cases
blank or single lines are affected and thus this change won't disturb
a git blame too much. For future commits the pre-commit scripts
checks that this won't happen again.
* keybox.h (KEYBOX_FLAG_BLOB_SECRET, KEYBOX_FLAG_BLOB_EPHEMERAL):
New.
* keybox-update.c (keybox_compress): Use it here instead of a
magic constant.
sm/
* fingerprint.c (gpgsm_get_fingerprint): Add caching.
(gpgsm_get_fingerprint_string): Use bin2hexcolon().
(gpgsm_get_fingerprint_hexstring): Use bin2hex and allocate only
as much memory as required.
(gpgsm_get_keygrip_hexstring): Use bin2hex.
* certchain.c (gpgsm_validate_chain): Keep track of the
certificate chain and reset the ephemeral flags.
* keydb.c (keydb_set_cert_flags): New args EPHEMERAL and MASK.
Changed caller to use a mask of ~0. Return a proper error code if
the certificate is not available.
* keybox-search.c (get_flag_from_image, keybox_get_flags): New.
(_keybox_get_flag_location): New.
* certchain.c (gpgsm_validate_chain): Mark revoked certs in the
keybox.
* keylist.c (list_cert_colon): New arg VALIDITY; use it to print a
revoked flag.
(list_internal_keys): Retrieve validity flag.
(list_external_cb): Pass 0 as validity flag.
* keydb.c (keydb_get_flags, keydb_set_flags): New.
(keydb_set_cert_flags): New.
(lock_all): Return a proper error code.
(keydb_lock): New.
(keydb_delete): Don't lock but check that it has been locked.
(keydb_update_keyblock): Ditto.
* delete.c (delete_one): Take a lock.
