* g10/tofu.c: Include <time.h>, <utime.h>, <fcntl.h> and <unistd.h>.
(tofu_dbs_s): Add fields want_lock_file and want_lock_file_ctime.
(begin_transaction): Only yield if DBS->WANT_LOCK_FILE_CTIME has
changed since we took the lock. Don't use gpgrt_yield to yield, but
sleep for 100ms. After taking the batch lock, update
DBS->WANT_LOCK_FILE_CTIME. Also take the batch lock the first time we
take the real lock. When taking the real lock, use immediate not
deferred mode to avoid gratuitious aborts.
(end_transaction): When dropping the outermost real lock, drop the
batch lock.
(busy_handler): New function.
(opendbs): Set the busy handler to it when opening the DB. Initialize
CTRL->TOFU.DBS->WANT_LOCK_FILE.
(tofu_closedbs): Free DBS->WANT_LOCK_FILE.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
By default, SQLite defers transactions until they are actually needed.
A consequence of this is that if we have two readers and both decide
to do a write, then one has to abort. To avoid this problem, we can
make the outermost transaction an immediate transaction. This has the
disadvantage that we only allow a single reader at a time, but at
least we don't have gratuitous aborts anymore.
A second problem is that SQLite apparently doesn't actually create a
queue of waiters. The result is that doing a sched_yield between
dropping and retaking the batch transaction is not enough to allow the
other process to make progress. Instead, we need to wait a
while (emperically: 100ms seems reasonable). To avoid waiting when
there is no contention, we use a new file's timestamp to signal that
there is a waiter.
* g10/tofu.c (build_conflict_set): If CONFLICT_SET contains a single
element, don't bother to check for cross sigs. Add parameter PK.
Update callers.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (signed_by_utk): When checking if a key is signed by an
ultimately trusted key, only consider the signatures on the specified
user id.
* tests/openpgp/tofu.scm: Add test for the above.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (cross_sigs): When checking cross signatures, only
consider the signatures on the specified user id.
* tests/openpgp/tofu.scm: Add test for the above.
* tests/openpgp/tofu/cross-sigs/
1938C3A0E4674B6C217AC0B987DB2814EC38277E-1.gpg:
New file.
* tests/openpgp/tofu/cross-sigs/
1938C3A0E4674B6C217AC0B987DB2814EC38277E-1.txt: New file.
* tests/openpgp/tofu/cross-sigs/
1938C3A0E4674B6C217AC0B987DB2814EC38277E-2.gpg: New file.
* tests/openpgp/tofu/cross-sigs/
1938C3A0E4674B6C217AC0B987DB2814EC38277E-2.txt: New file.
* tests/openpgp/tofu/cross-sigs/
1938C3A0E4674B6C217AC0B987DB2814EC38277E-3.txt: New file.
* tests/openpgp/tofu/cross-sigs/
1938C3A0E4674B6C217AC0B987DB2814EC38277E-secret.gpg: New file.
* tests/openpgp/tofu/cross-sigs/
DC463A16E42F03240D76E8BA8B48C6BD871C2247-1.gpg: New file.
* tests/openpgp/tofu/cross-sigs/
DC463A16E42F03240D76E8BA8B48C6BD871C2247-1.txt: New file.
* tests/openpgp/tofu/cross-sigs/
DC463A16E42F03240D76E8BA8B48C6BD871C2247-2.gpg: New file.
* tests/openpgp/tofu/cross-sigs/
DC463A16E42F03240D76E8BA8B48C6BD871C2247-2.txt: New file.
* tests/openpgp/tofu/cross-sigs/
DC463A16E42F03240D76E8BA8B48C6BD871C2247-3.gpg: New file.
* tests/openpgp/tofu/cross-sigs/
DC463A16E42F03240D76E8BA8B48C6BD871C2247-3.txt: New file.
* tests/openpgp/tofu/cross-sigs/
DC463A16E42F03240D76E8BA8B48C6BD871C2247-4.gpg: New file.
* tests/openpgp/tofu/cross-sigs/
DC463A16E42F03240D76E8BA8B48C6BD871C2247-secret.gpg: New file.
* tests/openpgp/tofu/cross-sigs/README: New file.
--
Signed-off-by: Neal H. Walfield
* g10/tofu.c (get_trust): If POLICY is ask, but we can't ask, don't
bail immediately. Instead, check if the key in question is an
ultimately trusted key or cross signed.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (record_binding): Take an additional argument, CONFLICT.
Set the binding's conflict accordingly. Update callers.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (build_conflict_set): A binding has a conflict is
conflict is *not* NULL, not if it is NULL.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (initdb): Change policy from a boolean to an integer.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
Reported-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Note: sqlite ignores type information so this change has no real
impact.
--
This reverts commit b0d2526bc4e5c663eeffe04500420c70cee98712.
The number of format elements may not change in ngettext. The entire
construct is anyway wrong because ngettext is passed a different value
than what is used in the printf.
We need to rework the use of most strings in tofu.c.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c (tofu_set_policy): If record_binding fails, fail. If the
function fails, rollback the transaction.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (get_trust): Use the pk_main_keyid accessor function.
(tofu_register_signature): Likewise.
(tofu_register_encryption): Likewise.
(tofu_set_policy): Likewise and also use pk_keyid and keyid_cmp.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (signed_by_utk): New function.
(get_trust): If a key is signed by an ultimately trusted key, then
set any bindings to good.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (ask_about_binding): Use the time that a signature was
seen, not allegedly generated, when generating statistics.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (build_conflict_set): Check for the current key after
looking for conflicts and removing any '!'.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
Fixes-commit: 1f1f56e6
* g10/tofu.c (record_binding): New parameter now. Update callers.
Don't use SQLite's strftime('%s','now') to get the current time, use
NOW.
(ask_about_binding): Likewise.
(get_trust): New parameter now. Update callers.
(show_statistics): Likewise.
(tofu_register_signature): Don't use SQLite's strftime('%s','now') to
get the current time, use gnupg_get_time().
(tofu_register_encryption): Likewise.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (write_stats_status): Use ulong for MESSSAGES. Fix
format strings. Simplify by using the new write_status_printf.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c (get_trust): Make sure the caller provides a primary key.
(tofu_register_signature): Likewise.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (time_ago_str): Only show the most significant unit.
* g10/tofu.c (show_statistics): Tweak the output.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (show_statistics): Return whether to call show_warning.
Move the warning from here...
(show_warning): ... to this new function.
(tofu_get_validity): If show_statistics returns a non-zero value, call
show_warning.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c: Include "sqrtu32.h".
(struct tofu_dbs_s.s): Rename get_trust_gather_other_keys to
get_trust_gather_signature_stats. Add new field
get_trust_gather_encryption_stats.
(initdb): Create the encryptions table.
(ask_about_binding): Show the encryption statistics too.
(tofu_register): Rename from this...
(tofu_register_signature): ... to this and update callers.
(tofu_register_encryption): New function.
(write_stats_status): Add parameters encryption_count,
encryption_first_done and encryption_most_recent. Update callers.
Compute the trust using the euclidean distance of the signature and
signature count. Compare with twice the threshold. Include
encryption count information in the TFS and TOFU_STATS lines.
(show_statistics): Also get information about the encrypted messages.
* g10/trustdb.c (tdb_get_validity_core): Use it.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (BINDING_NEW): New enum value.
(BINDING_CONFLICT): Likewise.
(BINDING_EXPIRED): Likewise.
(BINDING_REVOKED): Likewise.
(ask_about_binding): Move cross sig check from here...
(get_trust): ... and the conflict set building from here...
(build_conflict_set): ... to this new function.
(format_conflict_msg_part1): Replace parameter conflict with
conflict_set. Drop parameter fingerprint. Update callers.
(ask_about_binding): Drop unused parameter conflict and redundant
parameter bindings_with_this_email_count. Rename parameter
bindings_with_this_email to conflict_set. Update callers.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (cross_sigs): New function.
(ask_about_binding): If apparently conflicting keys are cross signed,
then don't mark them as conflicting.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
If two keys are cross signed, then the same person (probably)
controlled them both. In this case, don't raise a TOFU conflict.
This usually occurs when someone transitions to a new key. When that
person rotates to a third key, she will typically only cross sign it
with the second key. As such, we check this transitively to avoid
declaring a conflict between the 1st and 3rd key.
* g10/tofu.c (begin_transaction): New parameter only_batch. If set,
only start a batch transaction if there is none and one has been
requested. Update callers.
(tofu_suspend_batch_transaction): New function.
(tofu_resume_batch_transaction): Likewise.
(ask_about_binding): Take a ctrl_t, not a tofu_dbs_t. Update
callers. Gather statistics within a transaction. Suspend any batch
transaction when getting user input.
(get_trust): Take a ctrl_t, not a tofu_dbs_t. Update callers.
Enclose in a transaction.
(tofu_get_validity): Use a batch transaction, not a normal
transaction.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (struct signature_stats): Add fields is_expired and
is_revoked.
(signature_stats_prepend): Clear *stats when allocating it.
(ask_about_binding): Also show whether the user ids are expired or
revoked.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c: first_seen == 0 is not an error.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
Fixes-commit: 0f1f02ac
Regression-due-to: 45bb9a2a
* g10/tofu.c (tofu_set_policy): Do the update in a transaction.
* g10/gpg.c (main): Do a TOFU policy update in a batch transaction.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/gpgsql.h (gpgsql_arg_type): Rename SQLITE_ARG_END to
GPGSQL_ARG_END, SQLITE_ARG_INT to GPGSQL_ARG_INT, SQLITE_ARG_LONG_LONG
to GPGSQL_ARG_LONG_LONG, SQLITE_ARG_STRING to GPGSQL_ARG_STRING, and
SQLITE_ARG_BLOB to GPGSQL_ARG_BLOB.
--
This commit completes the work started in b1ba460.
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/gpg.h (tofu): Rename field batch_update_ref to
batch_updated_wanted.
* g10/tofu.c (struct tofu_dbs_s): Rename field batch_update to
in_batch_transaction.
(begin_transaction): Only end an extant batch transaction if we are
not in a normal transaction. When ending a batch transaction, really
end it. Update ctrl->tofu.batch_update_started when starting a batch
transaction.
(end_transaction): Only release a batch transaction if ONLY_BATCH is
true. When releasing a batch transaction, assert that there is no
open normal transaction. Only allow DBS to be NULL if ONLY_BATCH is
true.
(tofu_begin_batch_update): Don't update
ctrl->tofu.batch_update_started.
(opendbs): Call end_transaction unconditionally.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (tofu_get_validity): If a key has no valid (non-expired)
user ids, change TOFU to return TRUST_NEVER.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (tofu_register): Take a list of user ids, not a single
user id. Only register the bindings, don't compute the trust. Thus,
change return type to an int and remove the may_ask parameter. Update
callers.
(tofu_get_validity): Take a list of user ids, not a single user id.
Update callers. Observe signatures made by expired user ids, but
don't include them in the trust calculation.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/gpg.h (struct server_control_s): New field in_transaction.
* g10/tofu.c (struct tofu_dbs_s): Remove fields savepoint_inner and
savepoint_inner_commit.
(begin_transaction): Increment CTRL->TOFU.IN_TRANSACTION. Name the
savepoint according to the nesting level.
(end_transaction): Name the savepoint according to the nesting level.
Decrement CTRL->TOFU.IN_TRANSACTION.
(rollback_transaction): Likewise. Only ever rollback a non-batch
transaction.
(opendbs): Assert that there are no outstanding transactions.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (ask_about_binding): Print the info text when the policy
is ask and there are multiple bindings with the email address.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/gpg.c (oWithTofuInfo): New.
(opts): Add --with-tofu-info.
(main): Set opt.with_tofu_info.
* g10/options.h (struct opt): Add field WITH_TOFU_INFO.
* g10/tofu.c (show_statistics): Add optional arg OUTFP and enter
special mode if not NULL. Change all callers.
(tofu_write_tfs_record): New.
* g10/keylist.c (list_keyblock_colon): Do not print the tofu policy as
part of the "uid" record. Print a new "tfs" record if the new option
is set.
* tests/openpgp/tofu.scm (getpolicy): Change from UID to TFS record.
--
A separate option is required to avoid slowing down key listings.
Foer example the current code takes for a keylisting in tofu+pgp mode
17 seconds while it takes more than 5 minutes if the option is used.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c (write_stats_status): Add arg FP to print a colon
formated line. Adjust for changed TOFU_STATS interface.
(show_statistics): Let the query return timestamps and use
gnupg_get-time to compute the "time ago" values.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/options.h (struct opt): Remove field tofu_db_format.
* g10/gpg.h (server_control_s): Add fields tofu.batch_update_ref and
tofu.batch_update_started.
* g10/gpg.c (parse_tofu_db_format): Remove.
(main): Make option --tofu-db-format obsolete.
* g10/tofu.c: Major rework. Remove the pretty complicated and slower
split format and with that all the caching. Use the dbs struct
directly. Move global vars for batch update into CTRL. Change
calling conventions of some function to take CTRL or DBS pointers
instead of the former low-level database pointer.
--
The split database format might have been nice for use with Unison but
it bypasses the concept of a relational database by doing parts of
this itself and also risking deadlocks. Working with the Tofu
database for debugging or experiments is also not possible with parts
of the database logic implemented in gpg.
The Tofu support is quite new and we can assume that it is not in real
use now. Thus we better remove that now so that we do not need to
maintain it for all future.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c (tofu_closedbs): Fix freeing database handles up to the
cache limit. Previously, this would crash if db_cache_count == count.
Reported-by: Ben Kibbey <bjk@luxsci.net>
Signed-off-by: Justus Winter <justus@g10code.com>
* g10/tofu.c (get_trust): Factor code out to ...
(format_conflict_msg_part1): new and to ...
(ask_about_binding): new.
--
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c (get_trust): Add arg PK. Uses this instead of a an extra
lookup of the public key by fingerrpint.
(tofu_register): Pass PK to get_trust.
(tofu_get_validity): Ditto.
*g10/tofu.c (tofu_register): Remove unused FINGERPRINT_PP.
--
With my test keybox I see a speedup of 10 times (33s to 3.1s). The
reason for this was the extra key lookup which I hacked in at some
point to make the extraction of a keyid correct also for non v4 keys.
However our caller already has the public key and thus can easily pass
it to get_trust along with the fingerprint.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/sqlite.c: Rename to gpgsql.c. Change function prefixes to
gpgsql_.
* g10/sqlite.h: Rename to gpgsql.h.
* g10/tofu.c: Adjust for changes.
--
We used for our own extensions symbols with an sqlite_ names prefix.
This may in theory lead to duplicated symbols but more important, it
is harder to understand what is from gpg and what is from libsqlite.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/gpg.h (struct tofu_dbs_s, tofu_dbs_t): New declarations.
(struct server_control_s): Add field tofu.dbs.
* g10/tofu.c (struct dbs): Rename to tofu_dbs_s. Replace all users by
by tofu_dbs_t.
(opendbs): Add arg CTRL. Cache the DBS in CTRL.
(closedbs): Rename to tofu_closedbs and make global. Add arg CTRL.
(tofu_register): Add arg CTRL. Change all callers. Do not call
closedbs.
(tofu_get_validity): Ditto.
(tofu_set_policy): Ditto.
(tofu_get_policy): Ditto.
(tofu_set_policy_by_keyid): Add arg CTRL.
* g10/gpg.c (gpg_deinit_default_ctrl): Call tofu_closedbs.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c (show_statistics): Print TOFU STATS with formatting
characters.
--
We better leave the non-breaking space character in the status
messages so that the caller can make use of them.
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/status.h (STATUS_TOFU_USER, STATUS_TOFU_STATS)
(STATUS_TOFU_STATS_SHORT, STATUS_TOFU_STATS_LONG): New.
* g10/tofu.c (NO_WARNING_THRESHOLD): New.
(record_binding, tofu_register): Take care of --dry-run.
(show_statistics): Print STATUS_TOFU_USER. Reformat some messages.
Fix the ngettext/strcmp thing. Use log_string instead of log_info.
Use NO_WARNING_THRESHOLD constant.
(get_trust): Use format_text and print a compact fingerprint.
--
The use of log_string makes long messages better readable; instead of
gpg: Warning: if you think you've seen more[...]
key, then this key might be a forgery! Car[...]
address for small variations. If the key i[...]
we now have
gpg: Warning: if you think you've seen more[...]
key, then this key might be a forgery![...]
address for small variations. If the [...]
We also put the key information after the message and not between the
user id and the last used info like here:
gpg: Verified 7 messages signed by "Werner Koch <werner@eifzilla.de>"
in the past 4 days, 16 hours.
The most recent message was verified 3 days, 13 hours ago.
(key: 8061 5870 F5BA D690 3336 [...] 1E42 B367, policy: auto)
This also makes the key info a separate translatable string.
Further a compact version of the fingerprint (hex w/o spaces) is
printed in some messages.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c (string_to_long): New.
(string_to_ulong): New.
(get_single_unsigned_long_cb): Replace strtol/strtoul by new function.
(get_single_long_cb): Ditto.
(signature_stats_collect_cb): Ditto.
(get_policy): Ditto.
(show_statistics): Ditto. Uese es_free instead of free.
--
There is one minor semantic change: We now accept "nnn.0" always. The
old code did not checked for ".0: in show_statistics.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c (opendb): Guard call to timeout function.
--
GnuPG-bug-id: 2294
Fix not tested but is pretty obvious.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c: Make use of print_further_info to reduce the number of
different error messages to be translated. Also streamline some
messages.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c: Include ttyio.h. Change many strings to help
translating. Make use of ngettext wehere needed.
(CONTROL_L): New.
(TIME_AGO_UNIT_SMALL_NAME): Remove this and all similar *_NAME macros.
(time_ago_unit): Remove.
(get_trust): Use tty_prints and cpr_get only for the actual prompt.
Add Ctrl-L hack.
(show_statistics): Use two English strings for singular and plural.
* po/POTFILES.in: Add tofu.c.
--
These changes are required for proper translation. More to changes
may be needed, though.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c (record_binding): Change policy_old's type from an enum
tofu_policy to a long: this variable is passed by reference and a long
is expected.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
Reported-by: Justus Winter <justus@g10code.com>
Fixes-commit: f77913e
* common/stringhelp.c (format_text): New function.
* common/t-stringhelp.c (stresc): New function.
(test_format_text): New function. Test format_text.
* g10/tofu.c (get_trust): Use format_text to reflow long texts.
(show_statistics): Likewise.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (fingerprint_str): Remove.
(tofu_register): Take a public key instead of a fingerprint as arg.
Use hexfingerprint() to get a fpr from the PK.
(tofu_get_validity): Ditto.
(tofu_set_policy, tofu_get_policy): Simplify by using hexfingerprint.
* g10/trustdb.c (tdb_get_validity_core): Pass the primary key PK to
instead of the fingerprint to the tofu functions.
--
This change has the advantage that we are not bound to a specific
fingerprint length and will thus helps us to implement rfc4880bis.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c (get_trust): For the UTK check lookup the key by
fingerprint.
--
Extracting the keyid form the fingerprint is not a good idea because
that only works for v4 keys. It is also better to first read the key
and then extract the keyid from the actual available key.
The entire trusted-key stuff should be reworked to make use of
fingerprints.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c (opendbs): If the TOFU DB format is set to auto and there
is no TOFU DB, default to the flat format.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (fingerprint_str): Die with the error code returned by
the failed function.
(time_ago_str): Ditto. Do not make a comma translatable.
(fingerprint_format): Use "%zu" for a size_t.
--
Also wrapped some long strings.
In general we should not use log_fatal or use xmalloc functions but
properly return an error code and use xtrymalloc like functions.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/tofu.c (show_statistics): Also show when the most recently
signed message was observed.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
Suggested-by: MFPA <2014-667rhzu3dc-lists-groups@riseup.net>
* g10/tofu.c (show_statistics): Break the time delta to string code
into...
(time_ago_str): ... this new function.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/tofu.c (fingerprint_pp): Split this function into...
(fingerprint_str): ... this function...
(fingerprint_format): ... and this function.
(record_binding): Store the unformatted fingerprint in the DB. Only
use the formatting fingerprint when displaying a message to the user.
(get_trust): Likewise.
(show_statistics): Likewise.
(tofu_register): Likewise.
(tofu_get_validity): Likewise.
(tofu_set_policy): Likewise.
(tofu_get_policy): Likewise.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
