* kbx/backend-sqlite.c (struct be_sqlite_local_s): Add fields
lastubid_valid and lastubid.
(run_sql_prepare): Add optional extra2 arg and chage callers.
(run_select_statement): Add an ORDER BY clause to most SELECTs.
(be_sqlite_search): Skip duplicated keyblocks in a search.
--
See the comment in the code for the background.
Beware: This change lets tests/openpgp/key-selection.scm fail. Needs
to be fixed.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/backend-sqlite.c (DATABASE_VERSION): New.
(table_definitions): Make column name of table config unique.
(create_or_open_database): Read and set the database version.
(get_config_value, set_config_value): New.
--
Well, kind of. For now we just store the database version. Note that
for existing databases the config table should first be dropped so
that the UNIQUE constraint can be used.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/backend-sqlite.c (be_sqlite_rollback): New.
(be_sqlite_commit): New.
(be_sqlite_search): Take care of global transactions.
(be_sqlite_store): Ditto.
(be_sqlite_delete): Ditto.
* kbx/frontend.c (kbxd_rollback, kbxd_commit): New.
* kbx/keyboxd.h (opt): Add vars for transactions.
* kbx/kbxserver.c (struct server_local_s): Add fields next_session and
client_pid.
(session_list): New var.
(cmd_transaction): New.
(register_commands): Register command.
(kbxd_start_command_handler): Store pids and track sessions. Do a
final rollback.
--
This command is currently an experiment to allow a client to run
everything in one session.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/kbx-client-util.c (kbx_client_data_simple): New.
* kbx/backend-sqlite.c (struct be_sqlite_local_s): Add field descidx.
(be_sqlite_search): Use that.
* g10/call-keyboxd.c (keydb_search): Implement multi mode.
--
With that change the keyboxd is at par with the keybox code.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/backend-sqlite.c: Always use ?NNN for SQL parameters.
--
Using :NNN is obviously wrong to get the parameters in the right
order.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/backend-sqlite.c (table_definitions): Add column UINO to
userids.
(be_sqlite_local_s): Add fields select_col_uidno and
select_col_subkey.
(run_select_statement): Also select subkey or uidno column.
(be_sqlite_search): Return their values.
(store_into_userid): Store the UIDNO.
* kbx/backend-support.c (be_return_pubkey): Extend PUBKEY_INFO.
--
For an existing database adding the new column to the table userid is
straightforward. However if the original version of the schema used an
integer for the keyid column, that column has likely be renamed. Make
sure that the NOT NULL constraint has also be removed; check the
SQLite documentation on how this can be done.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-dump.c (_keybox_dump_blob): Fix the fixmes.
--
kbxutil is a debug tool but nevertheless it should behave well and not
read beyond allocated buffers and in turn crash.
Vincent Ulitzsch and Dominik Maier were kind enough to report these
crashes along with a couple of test keyboxes and crash analysis.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/backend-sqlite.c: Change definition of column KID.
(kid_from_mem): Remove.
(kid_from_u32): Rewrite.
(run_sql_bind_int64): Remove.
(run_select_statement): Implement lookup by short keyid. Fix lookup
by long keyid.
(store_into_fingerprint): Adjust kid arg.
--
The original idea of using an INTEGER, which has the required 64 bits,
didn't worked out due to problems with signed/unsigned mismatch and
the required truncation in a short keyid lookup. Thus we change the
definition to a blob.
For a database currently in use the change can be done by hand:
alter table fingerprint rename column kid to deletedkid;
alter table fingerprint add column kid not null
default X'0000000000000000';
update fingerprint set kid = substr (fpr, 13);
The update does only work with 20 octet fingerprints; that is not with
the new v5 OpenPGP keys. Note that sqlite does not allow to drip a
column, thus we rename it.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/backend-support.c (be_return_pubkey): Add args is_ephemeral and
is_revoked. Adjust callers.
* kbx/backend-sqlite.c: Alter table pubkey to add new columns.
(run_select_statement): Add new column to all selects.
(be_sqlite_search): Return the new flags.
--
For existing test databases the new column can be added with:
alter table pubkey add ephemeral integer not null default 0;
alter table pubkey add revoked integer not null default 0;
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/kbx-client-util.c (kbx_client_data_new): Add arg 'dlines'.
* g10/call-keyboxd.c (open_context): Set DLINES to true.
* sm/keydb.c (open_context): Ditto.
--
This allows to compile time switch between the D-line and the
fd-passing data communication between gpg/gpgsm and keyboxd. A quick
test with about 3000 OpenPGP keys showed that D-lines are only 10%
slower than the fd-passing based implementation. Given that the
thread adds extra complexity we go for now with the D-line approach.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keyboxd.h (struct server_control_s): Replace the two request
objects by just one. Add filter flags.
* kbx/kbxserver.c (cmd_search): Add options --openpgp and --x509.
(cmd_killkeyboxd): Do not return GPG_ERR_EOF.
* kbx/frontend.c (kbxd_release_session_info): Adjust for the new
request object.
(kbxd_search, kbxd_store, kbxd_delete): Ditto.
* kbx/backend-sqlite.c (struct be_sqlite_local_s): Add filter flags.
(run_sql_prepare): Add optional arg 'extra'. Change callers.
(run_sql_bind_ntext): New.
(run_sql_bind_text): Just call run_sql_bind_ntext.
(run_select_statement): Add ctrl arg. Implement the filter flags.
* g10/call-keyboxd.c (keydb_search): Use the --openpgp option.
--
As soon as we implement X.509 we need to have a way to return only
openpgp or x.509 certificates. Gpg/gpgsm will then use the respective
flag.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/kbx-client-util.c: New.
* kbx/kbx-client-util.h: New.
* kbx/Makefile.am (client_sources): New.
* g10/keydb.c (parse_keyblock_image): Rename to keydb_parse_keyblock
and make global.
* g10/call-keyboxd.c: Include kbx-client-util.h.
(struct keyboxd_local_s): Remove struct datastream. Add field kcd.
Remove per_session_init_done.
(lock_datastream, unlock_datastream): Remove.
(prepare_data_pipe, datastream_thread): Remove.
(keydb_get_keyblock_do_parse): Remove.
(gpg_keyboxd_deinit_session_data): Release the KCD object.
(open_context): Use of kbx_client_data_new.
(keydb_get_keyblock): Simplify.
(keydb_search): Use kbx_client_data_cmd and _wait.
--
The data specific part of the code has been moved from gpg to a new
module in kbx/ so that it can also be used by gpgsm. The OpenPGP
parsing while reading the data has been replaced by storing the data
in memory and parse it later. That makes a nice interface and
abstracts the fd-passing/D-lines handling away.
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/gc-opt-flags.h (GC_OPT_FLAG_RUNTIME): Move to ...
* tools/gpgconf-comp.c: here.
(known_options_scdaemon): Remove "options".
(known_options_dirmngr): Remove "options".
(known_options_gpgsm): Remove "options".
(known_options_gpg): Remove "options" and "keyserver".
(struct gc_option_s): Rename active t gpgconf_list.
(gc_component_list_options): Do not act upon active.
(option_check_validity): Ditto.
(is_known_option): Make it work correctly for unknown options.
(retrieve_options_from_program): Use renamed flag gpgconf_list only to
detect duplicated items from --gpgconf-list. Do not set runtime.
Only e set the options if set by --gpgconf-list; never clear them.
* agent/gpg-agent.c: Simplify the --gpgconf-list output.
* dirmngr/dirmngr.c: Ditto.
* g10/gpg.c: Ditto.
* kbx/keyboxd.c: Ditto.
* scd/scdaemon.c: Ditto.
* sm/gpgsm.c: Ditto.
* tests/openpgp/gpgconf.scm: Use "compliance" instead of "keyserver"
for the string arg test.
--
There is no need to read the list of options from the components
unless they convey a default value. It is better to consult only the
list we have in gpgconf-comp.c to decide on whether an option should
be displayed. Right, this might mess up thing if a newer gpgconf
version is used with an older component, but we already print warnings
in this case and in general we do not want to support this anymore -
the times of gpg 1.4. and 2.0 are long over now.
GnuPG-bug-id: 4788
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/util.h: Remove argparse.h.
* common/argparse.c: Undef GPGRT_ENABLE_ARGPARSE_MACROS.
* configure.ac (GPGRT_ENABLE_ARGPARSE_MACROS): Define.
* agent/gpg-agent.c: Undef GPGRT_ENABLE_ARGPARSE_MACROS and include
argparse.h. Do this also for all main modules which use our option
parser except for gpg. Replace calls to strusage by calls to
gpgrt_strusage everywhere.
* g10/gpg.c (opts): Change type to gpgrt_opt_t. Flag oOptions and
oNoOptions with ARGPARSE_conffile and ARGPARSE_no_conffile.
(main): Change type of pargs to gpgrt_argparse_t. Rework the option
parser to make use of the new gpgrt_argparser.
--
This is not yet finished but a make check works. gpg has the most
complex and oldest option handling and thus this is the first
migration target. SE-Linux checks and version-ed config files are
missing and will be added later.
GnuPG-bug-id: 4788
Signed-off-by: Werner Koch <wk@gnupg.org>
No functional changes, just fixing minor spelling issues.
---
Most of these were identified from the command line by running:
codespell \
--ignore-words-list fpr,stati,keyserver,keyservers,asign,cas,iff,ifset \
--skip '*.po,ChangeLog*,help.*.txt,*.jpg,*.eps,*.pdf,*.png,*.gpg,*.asc' \
doc g13 g10 kbx agent artwork scd tests tools am common dirmngr sm \
NEWS README README.maint TODO
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
* configure.ac: Add option --dsiable-keyboxd
* kbx/Makefile.am: Do not build keyboxd in that case.
--
This is useful to build a minimal version of gpgv.
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/util.h (EXTERN_UNLESS_MAIN_MODULE): Add the definion only
here but now without the Norcroft-C. Change all other places where it
gets defined.
* common/iobuf.h (iobuf_debug_mode): Declare unconditionally as
extern.
* common/iobuf.c (iobuf_debug_mode): Define it here.
* agent/gpg-agent.c (INCLUDED_BY_MAIN_MODULE): Define here and also in
all main modules of all other programs.
* g10/main.h: Put util.h before the local header files.
--
This change is required for use with gcc/ld's LTO feature which does
not allow common blocks. Further gcc 10 will make -fno-common the
default and thus this chnage is always needed. What a pitty.
Co-authored-by: Tomáš Mráz
GnuPG-bug-id: 4831
Signed-off-by: Werner Koch <wk@gnupg.org>
(cherry picked from commit 21d9bd8b87a9f793a106095e3838eb71825189d7)
- Applied respective chnages also to gpg-card and keyboxd.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/backend-sqlite.c: New.
* kbx/Makefile.am (keyboxd_SOURCES): Add it.
(keyboxd_CFLAGS, keyboxd_LDADD): Add SQLite flags.
* kbx/backend.h (enum database_types): Add DB_TYPE_SQLITE.
(be_sqlite_local_t): New typedef.
(struct db_request_part_s): Add field besqlite.
* kbx/backend-support.c (strdbtype): Add string for DB_TYPE_SQLITE.
(be_generic_release_backend): Support SQLite.
(be_release_request): Ditto.
(be_find_request_part): Ditto.
(is_x509_blob): Rename to ...
(be_is_x509_blob): this and make global.
* kbx/frontend.c (kbxd_set_database): Detect ".db" suffix and use that
for SQLite.
(kbxd_search): Support SQLite
(kbxd_store): Ditto.
(kbxd_delete): Ditto.
* kbx/frontend.h (kbxd_store_modes): Move to ...
* kbx/keyboxd.h (enum kbxd_store_modes): here.
* kbx/keyboxd.c (main): USe pubring.db for now. This is a temporary
hack.
* kbx/backend-kbx.c (be_kbx_delete): Remove unused var cert.
--
Take care: This is not finished and in particular filling the database
takes quite long.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/frontend.c (the_database): New var.
(db_desc_t): Remove.
(kbxd_add_resource): Renamed to ...
(kbxd_set_database): this. Simplify.
(kbxd_search): Change to use only one database.
(kbxd_store): Ditto.
(kbxd_delete): Ditto.
--
The original implementation was way to complicated and would have only
brought back the problems deciding which database to use for each key.
The new scheme used one configured database and only that. That
database needs to be set right at the start.
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/util.h (UBID_LEN): New. Use it at all places.
* kbx/keybox-blob.c (create_blob_finish): Do not write the UBID item.
* kbx/keybox-dump.c (print_ubib): Remove.
(_keybox_dump_blob): Do not print the now removed ubid flag.
* kbx/keybox-search-desc.h (struct keydb_search_desc): Use constants
for the size of the ubid and grip.
* kbx/keybox-search.c (blob_cmp_ubid): New.
(has_ubid): Make it a simple wrapper around blob_cmp_ubid.
(keybox_get_data): Add arg 'r_ubid'.
* kbx/frontend.h (enum kbxd_store_modes): New.
* kbx/kbxserver.c (cmd_store): Add new option --insert.
* kbx/backend-cache.c (be_cache_initialize): New.
(be_cache_add_resource): Call it here.
* kbx/backend-kbx.c (be_kbx_seek): Remove args 'fpr' and 'fprlen'.
(be_kbx_search): Get the UBID from keybox_get_data.
* kbx/backend-support.c (be_fingerprint_from_blob): Replace by ...
(be_ubid_from_blob): new. Change all callers.
* kbx/frontend.c (kbxd_add_resource): Temporary disable the cache but
use the new cache init function.
(kbxd_store): Replace arg 'only_update' by 'mode'. Seek using the
ubid. Take care of the mode.
--
It turned out that using the hash of the entire blob was not helpful.
Thus we redefine the Unique-Blob-ID (UBID) as the primary fingerprint
of the blob. In case this is a v5 OpenPGP key a left truncated
version of the SHA-256 hash is used; in all other cases the full SHA-1
hash. Using a SHA-256 hash does not make sense because v4 keys are
and will for some time be the majority of keys and thus padding them
with zeroes won't make any difference. Even if fingerprint collisions
can eventually be created we will assume that the keys are bogus and
that it does not make sense to store its twin also in our key storage.
We can also easily extend the update code to detect a collision and
reject the update.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox.h: Fix naming of arguments.
* scd/ccid-driver.c (print_error) [TEST]: Add missing break. Note
that this is anyway an impossible case.
Signed-off-by: Werner Koch <wk@gnupg.org>
--
Resolved Conflicts:
* common/asshelp.c: Keep the new code in master for spawing under
Windows.
* g10/Makefile.am: Keep all new file.
* g10/photoid.c: Pass CTRL to pct_expando.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/frontend.c (kbxd_add_resource): Print a diagnostic on error.
* kbx/backend-kbx.c (be_kbx_add_resource): Acttually returh the error
code.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-blob.c (create_blob_header): New blob flag UBIB.
(create_blob_finish): Write the UBIB.
* kbx/keybox-dump.c (print_ubib): New.
(_keybox_dump_blob): Print UBIB flag.
* kbx/keybox-search.c (has_ubid): Compare the stored UBIB if
available.
--
This make scanning the keybox for a given UBIB much faster once it has
been stored.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/backend.h (enum database_types): Add DB_TYPE_CACHE.
(struct db_request_part_s): Add seqno fields.
(struct db_request_s): Add infos for the cache backend.
* kbx/backend-support.c (struct backend_handle_s): Add 'backend_id'.
(strdbtype): Support DB_TYPE_CACHE.
(be_generic_release_backend): Ditto.
(be_find_request_part): New.
(be_return_pubkey): New arg UBID and chnage status name.
* kbx/backend-cache.c: New.
* kbx/backend-kbx.c (be_kbx_init_request_part): New.
(be_kbx_search): Factor some code out to a support function.
(be_kbx_seek): New.
* kbx/frontend.c (kbxd_add_resource): Support DB_TYPE_CACHE.
(kbxd_search): Support the NEXR operation with the cache.
* kbx/keybox-search-desc.h (KEYDB_SEARCH_MODE_UBID): New.
(struct keydb_search_desc): Add field u.ubid.
* kbx/keybox-search.c (has_ubid): New.
(keybox_search): Support the UBID search.
--
This adds a caching backend to the keyboxd. This tries to accommodate
for duplicate use of fingerprints and thus be correct in case a
fingerprint is used in several keys. It also turned out that we need
to have a unique identifier (UBID) to identify a keyblock or X.509
certificate. In particular with an OpenPGP keyblob we can't easily
use the primary fingerprint as an identifier because that fingerprint
may also be used as subkey in another key. Thus using a hash of the
entire keyblock is a better identifier to be used to address a
keyblock for restarting a search or for identifying the keyblock to be
updated. Note that this new UBID is not a permanent identifier
because it changes with all keyblock update; it should be viewed as a
handle to the keyblock or X509 cert.
* kbx/keybox-dump.c (_keybox_dump_file): Take deleted records in
account.
--
This also changes the numbering of the records to reflect the real
record number.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-update.c (keybox_compress): Change condition back.
Also use make_timestamp for CUT_TIME.
--
Fixes-commit: 824ca6f042dc69edaf67bf9d4e875be75babab00
Note that the original change was not backported to 2.2.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keybox-search.c (keybox_search): We need to seek to the last
position in all cases not just when doing a NEXT.
--
This is because search from the beginning needs a keybox_search_reset.
We can only make an exception for KEYDB_SEARCH_MODE_FIRST..
Fixes-commit: 49b236af0ecbb6df67513feb4b63851f2e159ea2
Signed-off-by: Werner Koch <wk@gnupg.org>
