* g10/options.h (IMPORT_BULK): New.
* g10/import.c (parse_import_options): Add "bulk-import".
* g10/call-keyboxd.c (in_transaction): New var.
(gpg_keyboxd_deinit_session_data): Run a commit if in bulk import
mode.
(create_new_context): Run a begin transaction if in bulk import mode.
--
Initial tests with this option are not very promising. Importing
about 3000 real world keys with --use-keyboxd and full logging took:
real 33m31.724s
user 19m54.265s
sys 2m49.662s
With bulk-import this saves a mere 12%:
real 29m36.542s
user 19m3.391s
sys 2m46.728s
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/kbx-client-util.c (kbx_client_data_simple): New.
* kbx/backend-sqlite.c (struct be_sqlite_local_s): Add field descidx.
(be_sqlite_search): Use that.
* g10/call-keyboxd.c (keydb_search): Implement multi mode.
--
With that change the keyboxd is at par with the keybox code.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb-private.h (struct keydb_handle_s): Add fields to return
the ordinals of the last found blob.
* g10/call-keyboxd.c (keydb_get_keyblock): Pass them to the keyblock
parser.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/kbx-client-util.c (kbx_client_data_new): Add arg 'dlines'.
* g10/call-keyboxd.c (open_context): Set DLINES to true.
* sm/keydb.c (open_context): Ditto.
--
This allows to compile time switch between the D-line and the
fd-passing data communication between gpg/gpgsm and keyboxd. A quick
test with about 3000 OpenPGP keys showed that D-lines are only 10%
slower than the fd-passing based implementation. Given that the
thread adds extra complexity we go for now with the D-line approach.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/keyboxd.h (struct server_control_s): Replace the two request
objects by just one. Add filter flags.
* kbx/kbxserver.c (cmd_search): Add options --openpgp and --x509.
(cmd_killkeyboxd): Do not return GPG_ERR_EOF.
* kbx/frontend.c (kbxd_release_session_info): Adjust for the new
request object.
(kbxd_search, kbxd_store, kbxd_delete): Ditto.
* kbx/backend-sqlite.c (struct be_sqlite_local_s): Add filter flags.
(run_sql_prepare): Add optional arg 'extra'. Change callers.
(run_sql_bind_ntext): New.
(run_sql_bind_text): Just call run_sql_bind_ntext.
(run_select_statement): Add ctrl arg. Implement the filter flags.
* g10/call-keyboxd.c (keydb_search): Use the --openpgp option.
--
As soon as we implement X.509 we need to have a way to return only
openpgp or x.509 certificates. Gpg/gpgsm will then use the respective
flag.
Signed-off-by: Werner Koch <wk@gnupg.org>
* kbx/kbx-client-util.c: New.
* kbx/kbx-client-util.h: New.
* kbx/Makefile.am (client_sources): New.
* g10/keydb.c (parse_keyblock_image): Rename to keydb_parse_keyblock
and make global.
* g10/call-keyboxd.c: Include kbx-client-util.h.
(struct keyboxd_local_s): Remove struct datastream. Add field kcd.
Remove per_session_init_done.
(lock_datastream, unlock_datastream): Remove.
(prepare_data_pipe, datastream_thread): Remove.
(keydb_get_keyblock_do_parse): Remove.
(gpg_keyboxd_deinit_session_data): Release the KCD object.
(open_context): Use of kbx_client_data_new.
(keydb_get_keyblock): Simplify.
(keydb_search): Use kbx_client_data_cmd and _wait.
--
The data specific part of the code has been moved from gpg to a new
module in kbx/ so that it can also be used by gpgsm. The OpenPGP
parsing while reading the data has been replaced by storing the data
in memory and parse it later. That makes a nice interface and
abstracts the fd-passing/D-lines handling away.
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/util.h: Remove argparse.h.
* common/argparse.c: Undef GPGRT_ENABLE_ARGPARSE_MACROS.
* configure.ac (GPGRT_ENABLE_ARGPARSE_MACROS): Define.
* agent/gpg-agent.c: Undef GPGRT_ENABLE_ARGPARSE_MACROS and include
argparse.h. Do this also for all main modules which use our option
parser except for gpg. Replace calls to strusage by calls to
gpgrt_strusage everywhere.
* g10/gpg.c (opts): Change type to gpgrt_opt_t. Flag oOptions and
oNoOptions with ARGPARSE_conffile and ARGPARSE_no_conffile.
(main): Change type of pargs to gpgrt_argparse_t. Rework the option
parser to make use of the new gpgrt_argparser.
--
This is not yet finished but a make check works. gpg has the most
complex and oldest option handling and thus this is the first
migration target. SE-Linux checks and version-ed config files are
missing and will be added later.
GnuPG-bug-id: 4788
Signed-off-by: Werner Koch <wk@gnupg.org>
No functional changes, just fixing minor spelling issues.
---
Most of these were identified from the command line by running:
codespell \
--ignore-words-list fpr,stati,keyserver,keyservers,asign,cas,iff,ifset \
--skip '*.po,ChangeLog*,help.*.txt,*.jpg,*.eps,*.pdf,*.png,*.gpg,*.asc' \
doc g13 g10 kbx agent artwork scd tests tools am common dirmngr sm \
NEWS README README.maint TODO
Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
* g10/call-keyboxd.c (struct store_parm_s): New.
(store_inq_cb): New.
(keydb_update_keyblock): Implement the keyboxd part.
(keydb_insert_keyblock): Ditto.
(keydb_delete_keyblock): Ditto.
(keydb_search_reset): Clear ubid flag. Also use the correct union
member for building the search string.
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/util.h (UBID_LEN): New. Use it at all places.
* kbx/keybox-blob.c (create_blob_finish): Do not write the UBID item.
* kbx/keybox-dump.c (print_ubib): Remove.
(_keybox_dump_blob): Do not print the now removed ubid flag.
* kbx/keybox-search-desc.h (struct keydb_search_desc): Use constants
for the size of the ubid and grip.
* kbx/keybox-search.c (blob_cmp_ubid): New.
(has_ubid): Make it a simple wrapper around blob_cmp_ubid.
(keybox_get_data): Add arg 'r_ubid'.
* kbx/frontend.h (enum kbxd_store_modes): New.
* kbx/kbxserver.c (cmd_store): Add new option --insert.
* kbx/backend-cache.c (be_cache_initialize): New.
(be_cache_add_resource): Call it here.
* kbx/backend-kbx.c (be_kbx_seek): Remove args 'fpr' and 'fprlen'.
(be_kbx_search): Get the UBID from keybox_get_data.
* kbx/backend-support.c (be_fingerprint_from_blob): Replace by ...
(be_ubid_from_blob): new. Change all callers.
* kbx/frontend.c (kbxd_add_resource): Temporary disable the cache but
use the new cache init function.
(kbxd_store): Replace arg 'only_update' by 'mode'. Seek using the
ubid. Take care of the mode.
--
It turned out that using the hash of the entire blob was not helpful.
Thus we redefine the Unique-Blob-ID (UBID) as the primary fingerprint
of the blob. In case this is a v5 OpenPGP key a left truncated
version of the SHA-256 hash is used; in all other cases the full SHA-1
hash. Using a SHA-256 hash does not make sense because v4 keys are
and will for some time be the majority of keys and thus padding them
with zeroes won't make any difference. Even if fingerprint collisions
can eventually be created we will assume that the keys are bogus and
that it does not make sense to store its twin also in our key storage.
We can also easily extend the update code to detect a collision and
reject the update.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/keydb-private.h (struct keydb_handle_s): Add fields for UBID.
* g10/call-keyboxd.c (search_status_cb): New.
(keydb_search): Set new UBID fields.
--
The UBID is not yet used but may come handy later.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/Makefile.am: Add nPth flags.
* g10/gpg.c: Include npth.h.
(gpg_deinit_default_ctrl): Deinit call-keyboxd local data.
(main): Init nPth.
* g10/keydb-private.h (struct keydb_handle_s): Add field 'kbl' and
remove the search result and the assuan context.
* g10/call-keyboxd.c (struct keyboxd_local_s): Add more fields.
(lock_datastream, unlock_datastream): New.
(gpg_keyboxd_deinit_session_data): Adjust for changed data structures.
(prepare_data_pipe): New.
(open_context): Return kbl instead of an Assuan context. Init mutexes
etc.
(close_context): Merge into ...
(keydb_release): here. Adjust for changed data structures.
(datastream_thread): New.
(keydb_get_keyblock): Implement datastream stuff.
(keydb_search): Ditto.
* common/asshelp.c (wait_for_sock): Add arg connect_flags.
(start_new_service): Set FDPASSING flag for the keyboxd.
--
This code as a lot of rough edges, in particular it relies on a well
behaving keyboxd. We need to add code to shutdown the datastream
reader thread in case of errors and to properly get it up again. We
also need to make really sure that both threads run in lockstep so
that the datastream thread is only active while we are sending a
command to the keyboxd.
We should also see whether we can depend nPth initialization on the
--use-keyboxd option to avoid any problems with nPth.
And we need to test on Windows.
Signed-off-by: Werner Koch <wk@gnupg.org>
* g10/gpg.c (oUseKeyboxd,oKeyboxdProgram): New consts.
(opts): New options --use-keyboxd and --keyboxd-program.
(main): Implement them.
* g10/keydb.c: Move some defs out to ...
* g10/keydb-private.h: new file.
* g10/keydb.c: prefix function names with "internal" and move original
functions to ...
* g10/call-keyboxd.c: new file. Divert to the internal fucntion if
--use-keyboxd is used. Add a CTRL arg to most fucntions and change
all callers.
* g10/Makefile.am (common_source): Add new files.
(noinst_PROGRAMS): Do bot build gpgcompose.
--
Note that this is just the framework with only a basic implementation
of searching via keyboxd.
Signed-off-by: Werner Koch <wk@gnupg.org>
