--
We need to use es_fopen on Windows to cope with non-ascii file names.
This is quite a large but fortunately straightforward change. At a
very few places we keep using stdio (for example due to the use of
popen).
GnuPG-bug-id: 5098
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-dump.c (_keybox_dump_blob): Fix the fixmes.
--
kbxutil is a debug tool but nevertheless it should behave well and not
read beyond allocated buffers and in turn crash.
Vincent Ulitzsch and Dominik Maier were kind enough to report these
crashes along with a couple of test keyboxes and crash analysis.
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/util.h (UBID_LEN): New. Use it at all places.
* kbx/keybox-blob.c (create_blob_finish): Do not write the UBID item.
* kbx/keybox-dump.c (print_ubib): Remove.
(_keybox_dump_blob): Do not print the now removed ubid flag.
* kbx/keybox-search-desc.h (struct keydb_search_desc): Use constants
for the size of the ubid and grip.
* kbx/keybox-search.c (blob_cmp_ubid): New.
(has_ubid): Make it a simple wrapper around blob_cmp_ubid.
(keybox_get_data): Add arg 'r_ubid'.
* kbx/frontend.h (enum kbxd_store_modes): New.
* kbx/kbxserver.c (cmd_store): Add new option --insert.
* kbx/backend-cache.c (be_cache_initialize): New.
(be_cache_add_resource): Call it here.
* kbx/backend-kbx.c (be_kbx_seek): Remove args 'fpr' and 'fprlen'.
(be_kbx_search): Get the UBID from keybox_get_data.
* kbx/backend-support.c (be_fingerprint_from_blob): Replace by ...
(be_ubid_from_blob): new. Change all callers.
* kbx/frontend.c (kbxd_add_resource): Temporary disable the cache but
use the new cache init function.
(kbxd_store): Replace arg 'only_update' by 'mode'. Seek using the
ubid. Take care of the mode.
--
It turned out that using the hash of the entire blob was not helpful.
Thus we redefine the Unique-Blob-ID (UBID) as the primary fingerprint
of the blob. In case this is a v5 OpenPGP key a left truncated
version of the SHA-256 hash is used; in all other cases the full SHA-1
hash. Using a SHA-256 hash does not make sense because v4 keys are
and will for some time be the majority of keys and thus padding them
with zeroes won't make any difference. Even if fingerprint collisions
can eventually be created we will assume that the keys are bogus and
that it does not make sense to store its twin also in our key storage.
We can also easily extend the update code to detect a collision and
reject the update.
Signed-off-by: Werner Koch <wk@gnupg.org>
--
Resolved Conflicts:
* common/asshelp.c: Keep the new code in master for spawing under
Windows.
* g10/Makefile.am: Keep all new file.
* g10/photoid.c: Pass CTRL to pct_expando.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-blob.c (create_blob_header): New blob flag UBIB.
(create_blob_finish): Write the UBIB.
* kbx/keybox-dump.c (print_ubib): New.
(_keybox_dump_blob): Print UBIB flag.
* kbx/keybox-search.c (has_ubid): Compare the stored UBIB if
available.
--
This make scanning the keybox for a given UBIB much faster once it has
been stored.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-dump.c (_keybox_dump_file): Take deleted records in
account.
--
This also changes the numbering of the records to reflect the real
record number.
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/userids.c (classify_user_id): Support 32 byte fingerprints.
* kbx/keybox-search-desc.h (KEYDB_SEARCH_MODE_FPR32): New.
(struct keydb_search_desc): Add field fprlen.
* kbx/keybox-defs.h (struct _keybox_openpgp_key_info): Add field
version and increase size of fpr to 32.
* kbx/keybox-blob.c: Define new version 2 for PGP and X509 blobs.
(struct keyboxblob_key): Add field fprlen and increase size of fpr.
(pgp_create_key_part_single): Allow larger fingerprints.
(create_blob_header): Implement blob version 2 and add arg want_fpr32.
(_keybox_create_openpgp_blob): Detect the need for blob version 2.
* kbx/keybox-search.c (blob_get_first_keyid): Support 32 byte
fingerprints.
(blob_cmp_fpr): Ditto.
(blob_cmp_fpr_part): Ditto.
(has_fingerprint): Add arg fprlen and pass on.
(keybox_search): Support KEYDB_SEARCH_MODE_FPR32 and adjust for
changed has_fingerprint.
* kbx/keybox-openpgp.c (parse_key): Support version 5 keys.
* kbx/keybox-dump.c (_keybox_dump_blob): Support blob version 2.
* g10/delkey.c (do_delete_key): Support KEYDB_SEARCH_MODE_FPR32.
* g10/export.c (exact_subkey_match_p): Ditto.
* g10/gpg.c (main): Ditto.
* g10/getkey.c (get_pubkey_byfprint): Adjust for changed
KEYDB_SEARCH_MODE_FPR.
* g10/keydb.c (keydb_search_desc_dump): Support
KEYDB_SEARCH_MODE_FPR32 and adjust for changed KEYDB_SEARCH_MODE_FPR.
(keydb_search): Add new arg fprlen and change all callers.
* g10/keyedit.c (find_by_primary_fpr): Ditto.
* g10/keyid.c (keystr_from_desc): Ditto.
* g10/keyring.c (keyring_search): Ditto.
* g10/keyserver.c (print_keyrec): Ditto.
(parse_keyrec): Ditto.
(keyserver_export): Ditto.
(keyserver_retrieval_screener): Ditto.
(keyserver_import): Ditto.
(keyserver_import_fprint): Ditto.
(keyidlist): Ditto.
(keyserver_get_chunk): Ditto.
* g10/keydb.c (keydb_search): Add new arg fprlen and change all
callers.
* sm/keydb.c (keydb_search_fpr): Adjust for changed
KEYDB_SEARCH_MODE_FPR.
--
This prepares the support for OpenPGP v5 keys. The new version 2 blob
format is needed for the longer fingerprints and we also use this
opportunity to prepare for storing the keygrip in the blob for faster
lookup by keygrip. Right now this is not yet functional.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-file.c (_keybox_read_blob): Remove.
(_keybox_read_blob2): Rename to ....
(_keybox_read_blob): this. Make arg options. Change all callers.
* kbx/keybox-search.c (keybox_search): Factor fopen call out to ...
(open_file): new.
(keybox_seek): Als use open_file.
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/host2net.h (buf16_to_ulong, buf16_to_uint): New.
(buf16_to_ushort, buf16_to_u16): New.
(buf32_to_size_t, buf32_to_ulong, buf32_to_uint, buf32_to_u32): New.
--
Commit 91b826a38880fd8a989318585eb502582636ddd8 was not enough to
avoid all sign extension on shift problems. Hanno Böck found a case
with an invalid read due to this problem. To fix that once and for
all almost all uses of "<< 24" and "<< 8" are changed by this patch to
use an inline function from host2net.h.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox.h (keybox_blobtype_t): New.
* kbx/keybox-defs.h (BLOBTYPE_*): Replace by KEYBOX_BLOBTYPE_*.
* kbx/keybox-search.c (keybox_search): Add arg want_blobtype and skip
non-matching blobs.
* sm/keydb.c (keydb_search): Pass KEYBOX_BLOBTYPE_X509 to keybox_search.
* g10/keydb.c (keydb_search): Pass KEYBOX_BLOBTYPE_PGP to keybox_search.
--
Without this fix a listing of all keys would fail because the wrong
blob type would be returned for the gpg or gpgsm.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-file.c (IMAGELEN_LIMIT): Change limit from 10^6 to 2MiB.
(_keybox_read_blob2): Skip too long record records.
(_keybox_write_blob): Do not accept too long record.
* kbx/keybox-dump.c (file_stats_s): Add field skipped_long_blobs.
(_keybox_dump_file): Print new counter.
(_keybox_dump_file): Skip too long records.
----
To test this feature you may set the limit back to 1MiB and use key
F7F0E70F307D56ED which is in my local copy close to 2MiB. Without
this patch it was possible to import the key but access to that key
and all keys stored after it was not possible.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-defs.h (struct keybox_handle): Add field for_openpgp.
* kbx/keybox-file.c (_keybox_write_header_blob): Set openpgp header
flag.
* kbx/keybox-blob.c (_keybox_update_header_blob): Add arg for_openpgp
and set header flag.
* kbx/keybox-init.c (keybox_new): Rename to do_keybox_new, make static
and add arg for_openpgp.
(keybox_new_openpgp, keybox_new_x509): New. Use them instead of the
former keybox_new.
* kbx/keybox-update.c (blob_filecopy): Add arg for_openpgp and set the
openpgp header flags.
* g10/keydb.c (rt_from_file): New. Factored out and extended from
keydb_add_resource.
(keydb_add_resource): Switch to the kbx file if it has the openpgp
flag set.
* kbx/keybox-dump.c (dump_header_blob): Print header flags.
--
The problem was reported by dkg on gnupg-devel (2014-10-07):
I just discovered a new problem, though, which will affect people on
systems that have gpg and gpg2 coinstalled:
0) create a new keyring with gpg2, and use it exclusively with gpg2
for a while.
1) somehow (accidentally?) use gpg (1.4.x) again -- this creates
~/.gnupg/pubring.gpg
2) future runs of gpg2 now only look at pubring.gpg and ignore
pubring.kbx -- the keys you had accumulated in the keybox are no
longer listed in the output of gpg2 --list-keys
Note that gpgsm has always used pubring.kbx and thus this file might
already be there but without gpg ever inserted a key. The new flag in
the KBX header gives us an indication whether a KBX file has ever been
written by gpg >= 2.1. If that is the case we will use it instead of
the default pubring.gpg.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-update.c (keybox_update_keyblock): Implement.
* kbx/keybox-search.c (get_blob_flags): Move to ...
* kbx/keybox-defs.h (blob_get_type): here.
* kbx/keybox-file.c (_keybox_read_blob2): Fix calling without R_BLOB.
* g10/keydb.c (build_keyblock_image): Allow calling without
R_SIGSTATUS.
(keydb_update_keyblock): Implement for keybox.
* kbx/keybox-dump.c (_keybox_dump_blob): Fix printing of the unhashed
size. Print "does not expire" also on 64 bit platforms.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-blob.c (put_membuf): Use a NULL buf to store zero bytes.
(create_blob_finish): Write just the needed space.
(create_blob_finish): Switch to SHA-1.
* kbx/keybox-dump.c (print_checksum): New.
(_keybox_dump_blob): Print the checksum and the verification status.
--
The checksum was never used in the past. Due to fast SHA-1
computations in modern CPUs we now use SHA-1. Eventually we will
support a First blob flag to enable the use of a secret or public
HMAC-SHA1. The first may be used for authentication of keyblocks and
the latter to mitigate collission attacks on SHA-1. It is not clear
whether this will be useful at all.
* kbx/keybox-defs.h (_keybox_openpgp_key_info): Add field ALGO.
* kbx/keybox-openpgp.c (parse_key): Store algo.
* kbx/kbxutil.c (dump_openpgp_key): Print algo number.
* kbx/keybox-dump.c (_keybox_dump_blob): Print identical Sig-Expire
value lines with a range of indices.
The asymmetric quotes used by GNU in the past (`...') don't render
nicely on modern systems. We now use two \x27 characters ('...').
The proper solution would be to use the correct Unicode symmetric
quotes here. However this has the disadvantage that the system
requires Unicode support. We don't want that today. If Unicode is
available a generated po file can be used to output proper quotes. A
simple sed script like the one used for en@quote is sufficient to
change them.
The changes have been done by applying
sed -i "s/\`\([^'\`]*\)'/'\1'/g"
to most files and fixing obvious problems by hand. The msgid strings in
the po files were fixed with a similar command.
We better do this once and for all instead of cluttering all future
commits with diffs of trailing white spaces. In the majority of cases
blank or single lines are affected and thus this change won't disturb
a git blame too much. For future commits the pre-commit scripts
checks that this won't happen again.
