* sm/minip12.c: Factor parsing code out to ...
* common/tlv-parser.c: new. Extend function names and provide a few
extra functions.
* common/Makefile.am (common_sources): Add new file.
* sm/minip12.c: Adjust to use the new parser API.
* sm/minip12.c (struct tlv_ctx_s): Add origbuffer and origbufsize.
Remove pop_count. Rename offset to length.
(dump_tag_info, _dump_tag_info): Rewrite.
(dump_tlv_ctx, _dump_tlv_ctx): Rewrite.
(tlv_new): Init origbuffer.
(_tlv_peek): Add arg ti.
(tlv_peek): New.
(tlv_peek_null): New.
(_tlv_push): Rewrite.
(_tlv_pop): Rewrite.
(tlv_next): New macro. Move old code to ...
(_tlv_next): this. Add arg lno. Pop remaining end tags.
(tlv_popped): Remove.
(tlv_expect_object): Handle ndef.
(tlv_expect_octet_string): Ditto.
(parse_bag_encrypted_data): Use nesting level to control the inner
loop.
(parse_shrouded_key_bag): Likewise.
(parse_bag_data): Handle surplus octet strings.
(p12_parse): Ditto.
* sm/minip12.c (decrypt_block): Strip the padding.
(tlv_expect_top_sequence): Remove. Replace callers by
tlv_expect_sequence.
* tests/cms/samplekeys/t6752-ov-user-ff.p12: New sample key.
* tests/cms/samplekeys/Description-p12: Add its description
--
This patch improves the BER parser by simplifying it. Now tlv_next
pops off and thus closes all containers regardless on whether they are
length bounded or ndef. tlv_set_pending is now always used to undo
the effect of a tlv_next in a loop condition which was terminated by a
nesting level change.
Instead of using the length as seen in the decrypted container we now
remove the padding and let the BER parser do its work. This might
have a negative effect on pkcs#12 objects which are not correctly
padded but we don't have any example of such broken objects.
GnuPG-bug-id: 6752
* sm/minip12.c (_tlv_push): Handle NDEF more correctly.
(tlv_expect_octet_string): Do not bail out on NDEF.
(dump_tag_info): Print some more infos.
--
We do not have a complete test case for this. We need to further
analyze T6752 to see what Mozilla is doing here. In any case with
this patch we get a bit further and don't bail out at the ndef.
GnuPG-bug-id: 6536, 6752
* sm/minip12.c (oid_hmacWithSHA1): New. Also for the SHA-2 algos.
(digest_algo_from_oid): New.
(set_key_iv_pbes2): Add arg digest_algo.
(crypt_block): Ditto.
(decrypt_block): Ditto.
(parse_bag_encrypted_data): Parse the optional prf part and get the
hmac algorithm.
(parse_shrouded_key_bag): Ditto.
(p12_build): Pass SHA1 for digest_algo.
* sm/t-minip12.c (run_one_test): Print failed values in verbose mode.
* tests/cms/samplekeys/nistp256-openssl-self-signed.p12: New.
* tests/cms/samplekeys/Description-p12: Add this one.
* tests/cms/Makefile.am (EXTRA_DIST): Ditto.
--
This supports the modern algorithms, i.e. using SHA256 for the KDF
which is the default in openssl unless the -legacy option is used.
GnuPG-bug-id: 6536
* sm/minip12.c (need_octet_string_cramming): New.
(tlv_expect_object, tlv_expect_octet_string): Run the test before
cramming.
* sm/minip12.c (ENABLE_DER_STRUCT_DUMPING): New but undefined macro
for debug purposes.
(bag_decrypted_data_p, bag_data_p): Use macro to allow dumping.
--
This bug was exhibited by importing a gpgsm exported EC certificate.
We use an extra test instead of retrying to allow retruning an error
from malloc failure. And well, for easier reading of the code.
GnuPG-bug-id: 6536
* common/asshelp.h (ASSHELP_FLAG_AUTOSTART): New.
* common/asshelp.c (start_new_service): Rename arg autostart to flags
and adjust checks.
(start_new_gpg_agent): Likewise. Change all callers.
(start_new_keyboxd): Likewise. Change all callers.
(start_new_dirmngr): Likewise. Change all callers.
--
It is easier to have a generic flags arg instead of adding more and
more dedicated args. verbose and debug are kept as they are because
they are not boolean.
* sm/decrypt.c (gpgsm_decrypt): Use estream for the input.
* sm/encrypt.c (gpgsm_encrypt): Likewise.
* sm/gpgsm.c (open_read): Remove.
(main): Use open_es_fread for gpgsm_import_files. Fix call of
gpgsm_encrypt, gpgsm_sign, gpgsm_verify and gpgsm_decrypt.
(open_es_fread): Use gnupg_check_special_filename and open_stream_nc.
* sm/gpgsm.h: Fix function declarations.
* sm/import.c (import_one): Use estream for the input.
(reimport_one, gpgsm_import, gpgsm_import_files): Likewise.
* sm/server.c (struct server_local_s): Rename MESSAGE_FD to
MESSAGE_FP.
(close_message_fp): Rename from close_message_fd.
(reset_notify): Follow the change of close_message_fp.
(cmd_encrypt, cmd_decrypt, cmd_verify, cmd_sign): Follow the change of
close_message_fp. Use open_stream_nc to get estream.
(cmd_import): Likewise.
(cmd_export, cmd_delkeys, gpgsm_server): Follow the change of
close_message_fp.
(cmd_message): Setup MESSAGE_FP with open_stream_nc.
* sm/sign.c (hash_data): Use estream for the input.
(hash_and_copy_data): Likewise.
(gpgsm_sign): Likewise.
* sm/verify.c (hash_data): Use estream_t for FP.
(gpgsm_verify): Use estream_t for IN_FP and DATA_FP.
--
GnuPG-bug-id: 6592
Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
* sm/server.c (cmd_genkey): Use open_stream_nc for input and output.
(cmd_getauditlog): Use open_stream_nc for output.
--
GnuPG-bug-id: 6580
Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
* sm/minip12.c (struct bufferlist_s): New.
(struct tlv_ctx_s): Add bufferlist.
(tlv_register_buffer): New.
(tlv_release): Release bufferlist.
(tlv_expect_object): Handle octet string cramming.
(tlv_expect_octet_string): Ditto.
(cram_octet_string): Changed interface. We don't need the
input_consumed value anymore.
* sm/minip12.c (parse_shrouded_key_bag): Also parse the attribute set.
* sm/t-minip12.c (main): Add option --no-extra.
(cert_collect_cb, run_tests_from_file): Fix memory leak
* tests/cms/samplekeys/t5793-openssl.pfx: New from T5793.
* tests/cms/samplekeys/t5793-test.pfx: Ditto.
* tests/cms/samplekeys/Description-p12: Add them.
* tests/cms/Makefile.am (EXTRA_DIST): Add samplekeys.
--
This should finish the rewrite of the pkcsc#12 parser for now. More
fun is likely to come.
GnuPG-bug-id: 6536, 5793
* sm/minip12.c: Reworked most of the parser.
(p12_set_verbosity): Add arg debug and change all callers.
* sm/t-minip12.c: Major rewrite to run regression tests unattended.
* sm/Makefile.am (module_maint_tests): Move t-Minit to ...
(module_tests): here.
* tests/cms/samplekeys/Description-p12: New.
--
Note that cram_octet_string stuff has not yet been reworked. I need
to locate the sample files first.
GnuPG-bug-id: 6536
* g10/gpg.c (opts): Use string for oLoggerFD, oOverrideSessionKeyFD,
oStatusFD, oAttributeFD, oCommandFD, and oPassphraseFD.
(main): Use translate_sys2libc_fdstr.
* g10/gpgv.c (opts): Use string for oLoggerFD, and oStatusFD.
(main): Use translate_sys2libc_fdstr.
* sm/gpgsm.c (opts): Use string for oLoggerFD, oStatusFD, and
oPassphraseFD.
(main): Use translate_sys2libc_fdstr.
* tools/gpg-auth.c (opts): Use string for oStatusFD.
(main): Use translate_sys2libc_fdstr.
tools/gpg-card.c (opts): Use string for oStatusFD.
(main): Use translate_sys2libc_fdstr.
* tools/gpg-pair-tool.c (opts): Use string for oStatusFD.
(main): Use translate_sys2libc_fdstr.
* tools/gpg-wks-client.c (opts): Use string for oStatusFD.
(main): Use translate_sys2libc_fdstr.
* tools/gpgconf.c (opts): Use string for oStatusFD.
(main): Use translate_sys2libc_fdstr.
* tools/gpgtar-create.c (gpgtar_create): Fix for opt.status_fd.
* tools/gpgtar-extract.c (gpgtar_extract): Fix for opt.status_fd.
* tools/gpgtar-list.c (gpgtar_list): Fix for opt.status_fd.
* tools/gpgtar.c (opts): Use string for oStatusFD.
(main): Use translate_sys2libc_fdstr.
* tools/gpgtar.h (opts): Use string for oStatusFD.
--
GnuPG-bug-id: 6551
Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
* sm/call-dirmngr.c (run_command_inq_cb): Support SENDCERT_SKI.
* dirmngr/crlcache.c (crl_cache_insert): Print the CRL name along with
the unknown OID nortice.
* sm/gpgsm.c (oInputSizeHint): New.
(opts): Add "--input-size-hint".
(main): Set option.
* sm/server.c (option_handler): Add option "input-size-hint".
* sm/gpgsm.h (struct server_control_s): Add field input_size_hint.
* sm/encrypt.c (gpgsm_encrypt): Set the toatl file size.
* sm/decrypt.c (gpgsm_decrypt): Ditto.
* sm/sign.c (gpgsm_sign): Ditto.
* sm/verify.c (gpgsm_verify): Ditto.
--
This option allows to set a value for the progress output line. Note
that as of now there is no other way to set the file size.
GnuPG-bug-id: 6534
* g10/call-keyboxd.c (gpg_keyboxd_deinit_session_data): Release
the assuan connection before kbx_client_data_release.
(open_context): Enable use of the data pipe.
* sm/keydb.c (gpgsm_keydb_deinit_session_data): Release the
assuan connection before kbx_client_data_release.
(open_context): Enable use of the data pipe.
* kbx/kbx-client-util.c (struct kbx_client_data_s): Add THD field.
(prepare_data_pipe): Close the pipe output end as it's been sent
already. Remember the KCD->THD, so that it can be joined later.
(datastream_thread): Finish when reading no data from the pipe.
(kbx_client_data_release): Join the thread. Then, we can safely
call es_fclose on the FP.
--
GnuPG-bug-id: 6512
Signed-off-by: NIIBE Yutaka <gniibe@fsij.org>
* kbx/keybox-init.c (ll_buffer_size): New var intialized to 128k
(stream_buffers): New var.
(keybox_set_buffersize): New.
(_keybox_ll_open, _keybox_ll_close): Implement buffering.
* sm/gpgsm.c (oKbxBufferSize): New.
(opts): Add option --kbx-buffer-size.
(main): Call keybox_set_buffersize.
* g10/gpg.c: Include keybox.h.
* (oKbxBufferSize): New.
(opts): Add option --kbx-buffer-size.
(main): Call keybox_set_buffersize.
--
Running a test on Windows using a pubring.kbx with
Total number of blobs: 2098
openpgp: 1294
x509: 803
and a size of 42MiB with
gpgsm -k --with-validation --disable-dirmngr --kbx-buffer-size N >nul
gives these performance figures using procmon
| N(k) | file events | time(s) |
|------+-------------+---------|
| 0 | 4900000 | 86 |
| 16 | 2456000 | 58 |
| 32 | 1233000 | 43 |
| 64 | 622000 | 37 |
| 128 | 317000 | 32 |
| 256 | 164000 | 31 |
| 512 | 88000 | 30 |
Using _open instead of CreateFile give the same number of file events
but increased the time slight by one second for the measured buffer
size of 64k and 128k. Benchmarks for gpg have not been conducted.
* kbx/keybox-search.c (open_file): Use sysopen and sequential.
* kbx/keybox-update.c (create_tmp_file): Ditto.
(blob_filecopy): Ditto.
(keybox_set_flags): Ditto.
(keybox_delete): Ditto.
(keybox_compress): Ditto.
--
Under Windows "sysopen" requests that direct API calls (CreateFile et
al.) are used instead of the libc wrappers. This may or may not
improve the performance.
Using "sequential" is a hint to Windows to assume that a file is in
general access in a sequential manner. This will have an affect only
with a future libgpg-error.
* sm/certchain.c (check_cert_policy): Add simple static cache.
--
It is quite common that a policy file does not exist. Thus we can
avoid the overhead of trying to open it over and over again just to
assert that it does not exists.
* common/homedir.c (gnupg_maybe_make_homedir): Also create a
common.conf.
* g10/keydb.c: Include comopt.h.
(maybe_create_keyring_or_box): Detect the creation of a common.conf.
* g10/gpg.c (main): Avoid adding more resources in this case.
* sm/keydb.c: Include comopt.h.
(maybe_create_keybox): Detect the creation of a common.conf.
* common/comopt.h (comopt): Remove the conditional "extern".
* agent/trustlist.c (struct trustitem_s): Add field de_vs.
(read_one_trustfile): Parse it.
(istrusted_internal): Emit TRUSTLISTFLAG status line.
* sm/gpgsm.h (struct rootca_flags_s): Add field de_vs.
* sm/call-agent.c (istrusted_status_cb): Detect the flags.
* sm/sign.c (write_detached_signature): Remove unused vars.
--
Right now this flag has no effect; we first need to specify the exact
behaviour.
GnuPG-bug-id: 5079
* sm/sign.c: Include tlv.h.
(write_detached_signature): New,
(gpgsm_sign): Fixup binary detached signatures.
--
This helps some other software to verify detached signatures.
* dirmngr/server.c (cmd_getinfo): New sub-command "stats".
(dirmngr_status_helpf): Allow for a CTRL of NULL.
* dirmngr/certcache.c (cert_cache_print_stats): Add arg ctrl and use
dirmngr_status_helpf. Adjust all callers.
* dirmngr/domaininfo.c (domaininfo_print_stats): Ditto.
* sm/certchain.c (ask_marktrusted): Flush stdout before printing the
fingerprint.
* dirmngr/crlcache.h (CRL_CACHE_NOTTRUSTED): New.
* dirmngr/crlcache.c (cache_isvalid): Set this status.
(crl_cache_cert_isvalid): Map it to GPG_ERR_NOT_TRUSTED.
(crl_cache_reload_crl): Move diagnostic to ...
* dirmngr/crlfetch.c (crl_fetch): here.
* dirmngr/server.c (cmd_isvalid): Map it to GPG_ERR_NOT_TRUSTED.
* dirmngr/validate.c (check_revocations): Handle new status. Improve
diagnostics.
* common/status.c (get_inv_recpsgnr_code): Map INV_CRL_OBJ.
* common/audit.c (proc_type_verify): Ditto.
--
This avoids repeated loading of CRLs in case of untrusted root
certificates.
