* common/iobuf.c (iobuf_copy): Change buffer size from 1 MB to 32 KB.
--
Change suggested by Werner based on the observation that other buffers
are of a similar size.
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* g10/build-packet.c (do_symkey_enc): Use iobuf_write instead of
iobuf_put in a loop. Use iobuf_copy instead of iobuf_read and
iobuf_write in a loop. Move the memory wiping from here...
* common/iobuf.c (iobuf_copy): ... to here.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>
* common/b64enc.c (b64enc_finish): Remove var assignment which is not
used later.
* common/iobuf.c (file_filter): Ditto.
* common/tlv.c (do_find_tlv): Ditto.
* common/userids.c (classify_user_id): Ditto.
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/iobuf.c (iobuf_esopen): Initialize 'len' as 'file_es_filter'
will make use of it.
--
Found using the Clang Static Analyzer.
Signed-off-by: Justus Winter <justus@g10code.com>
* common/iobuf.h (enum iobuf_use): Add new member, IOBUF_INPUT_TEMP.
* common/iobuf.c (iobuf_temp_with_content): Create the iobuf as an
IOBUF_INPUT_TEMP, not an IOBUF_INPUT buffer. Assert that LENGTH ==
A->D.SIZE.
(iobuf_push_filter2): If A is an IOBUF_INPUT_TEMP, then make the new
filter an IOBUF_INPUT filter and set its buffer size to
IOBUF_BUFFER_SIZE.
(underflow): If A is an IOBUF_INPUT_TEMP, then just return EOF; don't
remove already read data.
(iobuf_seek): If A is an IOBUF_INPUT_TEMP, don't discard the buffered
data.
(iobuf_alloc): Allow USE == IOBUF_INPUT_TEMP.
(pop_filter): Allow USE == IOBUF_INPUT_TEMP.
(iobuf_peek): Allow USE == IOBUF_INPUT_TEMP.
(iobuf_writebyte): Fail if USE == IOBUF_INPUT_TEMP.
(iobuf_write): Fail if USE == IOBUF_INPUT_TEMP.
(iobuf_writestr): Fail if USE == IOBUF_INPUT_TEMP.
(iobuf_flush_temp): Fail if USE == IOBUF_INPUT_TEMP.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
Introduce a new iobuf type, IOBUF_INPUT_TEMP. Use this for the iobuf
created by iobuf_temp_with_content instead of IOBUF_INPUT. This was
necessary so that seeking and peeking correctly work on this type of
iobuf. In particular, seeking didn't work because we discarded the
buffered data and peeking didn't work because we discarded data which
was already read, which made seeking later impossible.
* common/iobuf.c (iobuf_read): Make control flow more obvious.
(iobuf_get_filelength): Likewise.
(iobuf_get_fd): Likewise.
(iobuf_seek): Likewise.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* common/iobuf.c (iobuf_alloc): Check that BUFSIZE is not 0.
(iobuf_readbyte): Check that A is an input filter. Check that the
amount of read data is at most the amount of buffered data.
(iobuf_read): Check that A is an input filter.
(iobuf_writebyte): Check that A is not an input filter.
(iobuf_writestr): Check that A is not an input filter.
(iobuf_flush_temp): Check that A is not an input filter.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* common/iobuf.c (iobuf_push_filter2): If the head filter is a temp
filter, use IOBUF_BUFFER_SIZE for the new filter.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* common/iobuf.c (iobuf_push_filter2): If the pipeline is an output or
temp pipeline, the new filter shouldn't assume ownership of the old
head's internal buffer: the data was written before the filter was
added.
* common/t-iobuf.c (double_filter): New function.
(main): Add test cases for the above bug.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* common/iobuf.c (iobuf_temp_to_buffer): Flush each filter in the
pipeline and copy the data from the last (not the first) filter's
internal buffer.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* common/iobuf.c (do_open): New function, which is a generalization of
iobuf_open, iobuf_Create, iobuf_openrw.
(iobuf_open): Call do_open.
(iobuf_create): Likewise.
(iobuf_openrw): Likewise.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* common/iobuf.h (iobuf_open_fd_or_name): Remove prototype. Replace
use with either iobuf_open or iobuf_fdopen_nc, as appropriate.
* common/iobuf.c (iobuf_open): Remove function.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* common/iobuf.c (iobuf_cancel): Don't abort freeing a pipeline if
freeing a filter fails. This needs to a memory leak. Instead, keep
freeing and return the error code of the first filter that fails.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* common/iobuf.c (underflow): Take additional parameter
clear_pending_eof. If not set, don't clear a pending eof when
returning EOF. Update callers.
(iobuf_peek): Fill the internal buffer, if needed, to be able to
better satisfy any request.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* common/iobuf.c (underflow): Don't require that the buffer be empty.
When called, fill any available space.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* common/iobuf.c (iobuf_read_line): Be more careful with boundary
conditions.
* common/iobuf.h: Include <gpg-error.h>.
* common/t-iobuf.c: New file.
* common/Makefile.am (module_tests): Add t-iobuf.
(t_mbox_util_LDADD): New variable.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* common/iobuf.c (iobuf_temp_with_content): Set the filter type to
IOBUF_INPUT, not IOBUF_TEMP, which is only for output filters that
write into a dynamic buffer.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* common/iobuf.h (struct iobuf_struct): Remove field desc.
* common/iobuf.c (iobuf_desc): New function. When a filter's
description is needed, use this instead of the filter's desc field.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* common/iobuf.h (struct iobuf_struct): Clarify semantics of nofast.
Simplify use of nofast to implement just these semantics.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* common/iobuf.c: Move BLOCK_FILTER_INPUT,
BLOCK_FILTER_OUTPUT_BLOCK_FILTER_TEMP from here...
* common/iobuf.h: ... to here and rename to IOBUF_INPUT, IOBUF_OUTPUT
and IOBUF_TEMP, respectively. Where appropriate, use these macros
instead of a literal.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* common/iobuf.c (BLOCK_FILTER_INPUT): Define. Where appropriate, use
this instead of a literal.
(BLOCK_FILTER_OUTPUT): Likewise.
(BLOCK_FILTER_TEMP): Likewise.
--
Signed-off-by: Neal H. Walfield <neal@g10code.com>.
* common/host2net.h (buf16_to_ulong, buf16_to_uint): New.
(buf16_to_ushort, buf16_to_u16): New.
(buf32_to_size_t, buf32_to_ulong, buf32_to_uint, buf32_to_u32): New.
--
Commit 91b826a38880fd8a989318585eb502582636ddd8 was not enough to
avoid all sign extension on shift problems. Hanno Böck found a case
with an invalid read due to this problem. To fix that once and for
all almost all uses of "<< 24" and "<< 8" are changed by this patch to
use an inline function from host2net.h.
Signed-off-by: Werner Koch <wk@gnupg.org>
* common/iobuf.c: (iobuf_open): initialize len
--
In iobuf_open, IOBUFCTRL_DESC and IOBUFCTRL_INIT commands are invoked
(via file_filter()) on fcx, passing in a pointer to an uninitialized
len.
With these two commands, file_filter doesn't actually do anything with
the value of len, so there's no actual risk of use of uninitialized
memory in the code as it stands.
However, some static analysis tools might flag this situation with a
warning, and initializing the value doesn't hurt anything, so i think
this trivial cleanup is warranted.
Debian-Bug-Id: 773469
* common/iobuf.c (direct_open): Add arg MODE700.
(iobuf_create): Ditto.
* g10/openfile.c (open_outfile): Add arg RESTRICTEDPERM. Change call
callers to pass 0 for it.
* g10/revoke.c (gen_desig_revoke, gen_revoke): Here pass true for new
arg.
* g10/export.c (do_export): Pass true for new arg if SECRET is true.
--
GnuPG-bug-id: 1653.
Note that this works only if --output has been used.
* common/iobuf.c (MAX_NESTING_FILTER): New.
(iobuf_push_filter2): Limit the nesting level.
* g10/mainproc.c (mainproc_context): New field ANY. Change HAVE_DATA
and ANY_SIG_SIGN to bit fields of ANY. Add bit field
UNCOMPRESS_FAILED.
(proc_compressed): Avoid printing multiple Bad Data messages.
(check_nesting): Return GPG_ERR_BAD_DATA instead of UNEXPECTED_DATA.
--
This is a more general fix for the nested compression packet bug. In
particular this helps g10/import.c:read_block to stop pushing
compression filters onto an iobuf stream. This patch also reduces the
number of error messages for the non-import case.
Signed-off-by: Werner Koch <wk@gnupg.org>
(cherry picked from commit 35e40e2d514223c950c2f6d1214e02e92d87e997)
Resolved conflicts:
common/iobuf.c
g10/mainproc.c
* common/iobuf.c (iobuf_seek): Fix for temp streams.
* g10/pubkey-enc.c (get_session_key, get_it): Add some log_clock calls.
* g10/keydb.c (dump_search_desc): New.
(enum_keyblock_states, struct keyblock_cache): New.
(keyblock_cache_clear): New.
(keydb_get_keyblock, keydb_search): Implement a keyblock cache.
(keydb_update_keyblock, keydb_insert_keyblock, keydb_delete_keyblock)
(keydb_rebuild_caches, keydb_search_reset): Clear the cache.
--
Gpg uses the key database at several places without a central
coordination. This leads to several scans of the keybox for the same
key. To improve that we now use a simple cache to store a retrieved
keyblock in certain cases. In theory this caching could also be done
for old keyrings, but it is a bit more work and questionable whether
it is needed; the keybox scheme is anyway much faster than keyrings.
Using a keybox with 20000 384 bit ECDSA/ECHD keypairs and a 252 byte
sample text we get these values for encrypt and decrypt operations on
an Core i5 4*3.33Ghz system. The option --trust-model=always is used.
Times are given in milliseconds wall time.
| | enc | dec | dec,q |
|-----------+-----+-----+-------|
| key 1 | 48 | 96 | 70 |
| key 10000 | 60 | 98 | 80 |
| key 20000 | 69 | 106 | 88 |
| 10 keys | 540 | 290 | 70 |
The 10 keys test uses a mix of keys, the first one is used for
decryption but all keys are looked up so that information about are
printed. The last column gives decryption results w/o information
printing (--quiet).
The keybox is always scanned sequentially without using any index. By
adding an index to the keybox it will be possible to further reduce
the time required for keys stored to the end of the file.
The asymmetric quotes used by GNU in the past (`...') don't render
nicely on modern systems. We now use two \x27 characters ('...').
The proper solution would be to use the correct Unicode symmetric
quotes here. However this has the disadvantage that the system
requires Unicode support. We don't want that today. If Unicode is
available a generated po file can be used to output proper quotes. A
simple sed script like the one used for en@quote is sufficient to
change them.
The changes have been done by applying
sed -i "s/\`\([^'\`]*\)'/'\1'/g"
to most files and fixing obvious problems by hand. The msgid strings in
the po files were fixed with a similar command.
We better do this once and for all instead of cluttering all future
commits with diffs of trailing white spaces. In the majority of cases
blank or single lines are affected and thus this change won't disturb
a git blame too much. For future commits the pre-commit scripts
checks that this won't happen again.
