* common/mbox-util.c (mailbox_from_userid): Add arg subaddress and
implement. Change all callers to pass false for it.
* common/t-mbox-util.c (run_mbox_no_sub_test): New.
(run_filter): Add arg no_sub.
(main): Call new test and add option --no-sub.
--
Some stats: In the about 5300000 keys on the SKS servers we found 3055
unique mailboxes with a '+' in it. After removing leading and
trailing '+' as well as multiple '+' (e.g. "c++" or "foo+bar+baz")
2697 were left which seem to be valid sub-addresses.
To filter mailboxes out from a line delimited list with
user-ids (e.g. an SQL output), the command
t-mbox-util --verbose --filter
can be used; to output w/o sub-addresses add --no-sub.
GnuPG-bug-id: 4200
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c (build_conflict_set): Do not assume MAX_FINGERPRINT_LEN
is the size of the fingerprint.
--
This problem was exhibited by
commit ecbbafb88d.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/card-util.c (change_cafpr): Use MAX_FINGERPRINT_LEN.
* g10/cipher.c (write_header): Use snprintf.
* g10/gpg.h (MAX_FINGERPRINT_LEN): Change to 32.
(MAX_FORMATTED_FINGERPRINT_LEN): Change to 59
* g10/keyid.c (format_hexfingerprint): Add v5 fingerprint format.
* g10/tofu.c (get_policy): Use MAX_FINGERPRINT_LEN for the buffer but
keep the raw length for now.
--
Note that this patch only increases the size of the buffer and adds a
new formatting for v5 fingerprints. Moe work is required to fix
internal data structures like those in trustdb.gpg and the tofu
tables.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/Makefile.am (tofu_source) [USE_TOFU]: Remove sqrtu32.h and
sqrtu32.c.
* g10/sqrtu32.h, g10/sqrtu32.c: Removed files.
* g10/tofu.c: Compare squares instead of square roots.
--
The original code is a factor 11.5 slower than using libm's sqrt(),
which in turn is a factor 3.5 slower than using one multiplication
on the other side of the comparison. Also, it's much simpler now.
Signed-off-by: Marcus Brinkmann <mb@g10code.com>
* g10/tofu.c (ask_about_binding): Remove useless translation markers.
--
Translation and in particular punctuation marks can only be translated
with context. Thus making a sole comma translatable is useless.
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/stringhelp.c (format_text): Return NULL on error.
* common/t-stringhelp.c (test_format_text): Adjust for change.
* g10/gpgcompose.c (show_help): Abort on out of core.
* g10/tofu.c (ask_about_binding): Abort on format_text error.
(show_statistics): Ditto.
(show_warning): Ditto.
--
For better re-usability function in common/ shot better not use
xmalloc functions.
Signed-off-by: Werner Koch <wk@gnupg.org>
--
For proper operations as a server we need to avoid global variables.
Thus we need to pass the session state CTRL to most functions. Quite
a lot of changes but fortunately straightforward to do.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c (get_trust): Sanity check CONFLICT_SET after calling
get_policy. If POLICY is 'auto' and the default policy is 'ask', make
sure CONFLICT_SET includes the current key.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
GnuPG-bug-id: 2959
Debian-bug-id: 854829
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (show_statistics): If there are not records, return 0
instead of NULL.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
GnuPG-bug-id: 2853
According to SUSv3:
If the subject sequence is empty or does not have the expected form,
no conversion is performed
...
If no conversion could be performed, 0 is returned and errno may be
set to [EINVAL].
http://pubs.opengroup.org/onlinepubs/007908799/xsh/strtol.html
It appears that MacOS X sets errno to EINVAL, but glibc doesn't.
Hence, we map NULL to 0 explicitly.
* g10/tofu.c (get_trust): Call get_policy before short-circuiting the
policy lookup for ultimately trusted keys to make sure the binding is
added to the bindings table, if necessary.
Signed-off-by: Neal H. Walfield <neal@g10code.com>
GnuPG-bug-id: 2929
* g10/tofu.c (tofu_get_validity): If there was a conflict, don't also
print out a warning about too few messages.
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (get_trust): Add arguments POLICYP and CONFLICT_SETP. If
they are not NULL, return the policy and conflict set (if there is
one), respectively. Update callers. If MAY_ASK is FALSE, don't print
out the statistics.
(tofu_register_encryption): If there is a conflict and we haven't yet
printed the statistics about the conflicting bindings, do so now.
(tofu_get_validity): Likewise.
Signed-off-by: Neal H. Walfield <neal@g10code.com>
GnuPG-bug-id: 2914
* g10/tofu.c (write_stats_status): Use the number of days with
signatures / encryptions to compute the validity, not the total number
of signatures / encryptions.
(BASIC_TRUST_THRESHOLD): Adjust given the new semantics.
(FULL_TRUST_THRESHOLD): Likewise.
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* doc/DETAILS: Add SIGN-DAYS and ENCRYPT-DAYS to the TOFU_STATS status
line.
* g10/tofu.c (write_stats_status): Take additional parameters
signature_days and encryption_days. Update callers. Include them in
the tfs record and TOFU status lines.
(show_statistics): Compute the number of days on which we saw a
message signed by FINGERPRINT, and the number of days on which we
encrypted a message to it.
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (ask_about_binding): Emit all of the conflicting keys and
their statistics on the status fd.
(get_trust): Likewise, if we don't call ask_about_binding.
(show_statistics): Have the caller pass the policy as returned by
get_policy. Add argument only_status_fd and don't emit any output on
stdout if it is set. Update callers.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (build_conflict_set): Revert to int* and fix calloc.
--
The original code used an int array and thus better keep that and do
not limit it to 128 entries.
Fixes-commit: c3008bffac
Fixes-commit: 3b5b94ceab
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c (build_conflict_set): Replace variable dynamic array.
--
Note that clang complained that CONFLICT_SET_COUNT could be negative.
Thus I added an assert.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c: Remove dead code.
(time_ago_str): Simplify implementation since we only want the most
significant unit.
(format_conflict_msg_part1): Use ngettext.
(ask_about_binding): Likewise and only emit full sentences.
(show_statistics): Likewise.
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* doc/DETAILS (TOFU_STATS): Rename the VALIDITY field to SUMMARY. Add
a new field called VALIDITY.
* g10/tofu.c (write_stats_status): Update output accordingly.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
Change TOFU_STATS as discussed offline with Werner, Justus and Andre.
* g10/tofu.c: Don't include <utime.h>, <fcntl.h> or <unistd.h>.
(busy_handler): Replace use of open with es_fopen.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
Thanks for Werner for pointing this out: es_fopen is more portable
than open.
* g10/trustdb.c (tdb_utks): New function.
* g10/tofu.c (check_utks): New function.
(initdb): Call it.
* tests/openpgp/tofu.scm: Modify test to check the effective policy of
keys whose effective policy changes when we change the set of UTKs.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
If the set of ultimately trusted keys changes, then it is possible
that a binding's effective policy changes. To deal with this, we
detect when the set of ultimately trusted keys changes and invalidate
all cached policies.
* g10/tofu.c (initdb): Add column effective_policy to the bindings
table.
(record_binding): New parameters effective_policy and set_conflict.
Save the effective policy. If SET_CONFLICT is set, then set conflict
according to CONFLICT. Otherwise, preserve the current value of
conflict. Update callers.
(get_trust): Don't compute the effective policy here...
(get_policy): ... do it here, if it was not cached. Take new
parameters, PK, the public key, and NOW, the time that the operation
started. Update callers.
(show_statistics): New parameter PK. Pass it to get_policy. Update
callers.
(tofu_notice_key_changed): New function.
* g10/gpgv.c (tofu_notice_key_changed): New stub.
* g10/import.c (import_revoke_cert): Take additional argument CTRL.
Pass it to keydb_update_keyblock.
* g10/keydb.c (keydb_update_keyblock): Take additional argument CTRL.
Update callers.
[USE_TOFU]: Call tofu_notice_key_changed.
* g10/test-stubs.c (tofu_notice_key_changed): New stub.
* tests/openpgp/tofu.scm: Assume that manually setting a binding's
policy to auto does not cause the tofu engine to forget about any
conflict.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
We now store the computed policy in the tofu DB (in the
effective_policy column of the bindings table) to avoid computing it
every time, which is expensive. Further, policy is never overridden
in case of a conflict. Instead, we detect a conflict if CONFLICT is
not empty.
This change is backwards compatible to existing DBs. The only minor
incompatibility is that unresolved conflicts won't be automatically
resolved in case we import a direct signature, or cross signatures.
* g10/tofu.c (struct tofu_dbs_s): Remove unused field
record_binding_update2. Replace register_insert with
register_signature and register_encryption.
(tofu_register_signature): Don't use dbs->s.register_insert, but
dbs->s.register_signature.
(tofu_register_encryption): Don't use dbs->s.register_insert, but
dbs->s.register_encryption.
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (busy_handler): Fix the flags and utime is not needed.
--
The argument flags must include one of O_RDONLY, O_WRONLY, or O_RDWR.
Adding O_TRUNC, the file is updated. So, utime is not needed.
Fixes-commit: b2e1b17efa
Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
* g10/tofu.c (busy_handler): Add third argument which is mandatory for
O_CREATE flag.
--
Reported-by: Kristian Fiskerstrand
Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
* g10/tofu.c: Include <time.h>, <utime.h>, <fcntl.h> and <unistd.h>.
(tofu_dbs_s): Add fields want_lock_file and want_lock_file_ctime.
(begin_transaction): Only yield if DBS->WANT_LOCK_FILE_CTIME has
changed since we took the lock. Don't use gpgrt_yield to yield, but
sleep for 100ms. After taking the batch lock, update
DBS->WANT_LOCK_FILE_CTIME. Also take the batch lock the first time we
take the real lock. When taking the real lock, use immediate not
deferred mode to avoid gratuitious aborts.
(end_transaction): When dropping the outermost real lock, drop the
batch lock.
(busy_handler): New function.
(opendbs): Set the busy handler to it when opening the DB. Initialize
CTRL->TOFU.DBS->WANT_LOCK_FILE.
(tofu_closedbs): Free DBS->WANT_LOCK_FILE.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
By default, SQLite defers transactions until they are actually needed.
A consequence of this is that if we have two readers and both decide
to do a write, then one has to abort. To avoid this problem, we can
make the outermost transaction an immediate transaction. This has the
disadvantage that we only allow a single reader at a time, but at
least we don't have gratuitous aborts anymore.
A second problem is that SQLite apparently doesn't actually create a
queue of waiters. The result is that doing a sched_yield between
dropping and retaking the batch transaction is not enough to allow the
other process to make progress. Instead, we need to wait a
while (emperically: 100ms seems reasonable). To avoid waiting when
there is no contention, we use a new file's timestamp to signal that
there is a waiter.
* g10/tofu.c (build_conflict_set): If CONFLICT_SET contains a single
element, don't bother to check for cross sigs. Add parameter PK.
Update callers.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
