This fixes the problematic #define fstat _fstat which would only support files up to 2 GB.
Also refactored it as evutil_fd_filesize to avoid exposing stat when not necessary.
Found by oss-fuzz, after coverage had been improved in google/oss-fuzz#11257
v2: adjust test
v3: fix for windows (_get_osfhandle() crashes when called on closed fd)
v4: fix for EVENT__DISABLE_MM_REPLACEMENT
In buffer.c a variable "flags" and a label "done" are defined but
never used if "EVENT__HAVEMMAP" is not defined.
The code does not work on platforms which do not provide
the function `socketpair()`. Introduce EVENT__HAVE_SOCKETPAIR flag
which determines if `socketpair()` or `evutil_ersatz_socketpair()`
is used.
The `readv`/`writev` functions are designed for scattered I/O optimally,
their logic in the kernel is more sophisticated, compared to read/write,
which includes extra on-stack `iovec` in the kernel space, importing `iovec`
array from user space to kernel space, reading/writing with `iov_iter`, etc.
As a result, using `readv`/`writev` on single-segment `iovec` will fall into
the special branch in the kernel where it is imported as `ITER_UBUF` differed
from `ITER_IOVEC` for multiple-segments `iovec`.
Thus, it is just not worth calling `readv`/`writev` for single-segment `iovec`,
we should use `read`/`write` instead, to save it from going through the
sophisticated yet unnecessary kernel code path, circumvent a waste of
kernel on-stack memory, copying `iovec` between user space and kernel space, etc.
The variable chainp is only used if USE_IOVEC_IMPL is defined.
This makes a strict compiler complain about unused variables,
since chainp is declared outside of an USE_IOVEC_IMPL block.
This is the same as evbuffer_add_reference(), but allows to specify
offset in the @data
v2: rename evbuffer_add_reference_misalign() to evbuffer_add_reference_with_offset()
If pread(2) is available, prefer it over double lseek(2)
and read(2) in evbuffer_file_segment_materialize().
Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
There can be issues on 32-bit architectures to mmap 2+GiB file, and to
make this portable between different version of glibc, mmap64 was
prefered over _FILE_OFFSET_BITS
Skip rounding up memory allocations for:
* evbuffer_add_reference()
* evbuffer_add_buffer_reference()
* evbuffer_add_file_segment()
These chain objects only store small structs with references to
other things, and these small structs do not themselves grow, so
bumping up the allocation to MIN_BUFFER_SIZE (512 bytes) is wasteful.
Looks like a `splice` implementation was planned, but has clearly never
eventuated (the TODO comment is from ~12 years ago, in
8b5bd77415fb6634fadf08357676926fecf5f032). For now, it's probably better
to remove the unused code/correct the docs.
UBSAN reports:
evbuffer/remove_buffer_with_empty3: ../buffer.c:1443:3: runtime error: null pointer passed as argument 2, which is declared to never be null
#0 0x7ffff6cd0410 in evbuffer_pullup ../buffer.c:1443
#1 0x5555556d68b9 in test_evbuffer_remove_buffer_with_empty3 ../test/regress_buffer.c:408
#2 0x5555557b95ee in testcase_run_bare_ ../test/tinytest.c:173
#3 0x5555557ba048 in testcase_run_one ../test/tinytest.c:333
#4 0x5555557bc0f8 in tinytest_main ../test/tinytest.c:527
#5 0x555555787702 in main ../test/regress_main.c:528
#6 0x7ffff606c001 in __libc_start_main (/usr/lib/libc.so.6+0x27001)
#7 0x55555569436d in _start (/src/le/libevent/.cmake-debug/bin/regress+0x14036d)
if evbuffer_add_file_segment() fails it returns -1, so we should call
evbuffer_file_segment_free() only on error, and this -1 not 0.
Fixes: 6a81b1f5 ("Avoid double-free on error in evbuffer_add_file. Found by coverity.")
Backport-to: 2.1
[ @azat:
- add return heredoc for evbuffer_setcb()
- add unit test with event_set_mem_functions()
- look through the report from abi-compliance-checker/abi-dumper
]
Closes: #855
Before this patch evbuffer always reads 4K at a time, while this is fine
most of time you can find an example when this will decrease throughput.
So add an API to change default limit:
- evbuffer_set_max_read()
- evbuffer_get_max_read()
And a notice that most of time default is sane.
advance_last_with_data() adjusts evbuffer.last_with_datap, and if we
will have empty chain in the middle advance_last_with_data() will stop,
while it should not, since while empty chains is not regular thing they
can pops up in various places like, and while I did not look through all
of them the most tricky I would say is:
evbuffer_reverse_space()/evbuffer_commit_space()
evbuffer_add_reference()
Test case from:
https://github.com/envoyproxy/envoy/pull/6062Fixes: #778
v2: keep last_with_datap really last with data, i.e. update only if
chain has data in it
In case we have empty chain (chain that do not have any data, i.e. ->off
== 0) at the beginning of the buffer, and no more full chains to move to
the dst, we will skip moving of this empty chain, and hence
last_with_datap will not be adjusted, and things will be broken after.
Fix this by not relying on ->off, just count if we have something to
move that's it.
Test case from:
https://github.com/envoyproxy/envoy/pull/6062Fixes: #774
As pointed by @yankeehacker in #590:
Signed to Unsigned Conversion Error - buffer.c:1623
Description: This assignment creates a type mismatch by populating an
unsigned variable with a signed value. The signed integer will be
implicitly cast to an unsigned integer, converting negative values into
positive ones. If an attacker can control the signed value, it may be
possible to trigger a buffer overflow if the value specifies the length
of a memory write.
Remediation: Do not rely on implicit casts between signed and unsigned
values because the result can take on an unexpected value and violate
weak assumptions made elsewhere in the program.
Fixes: #590
../buffer.c:2231:6: warning: Access to field 'flags' results in a dereference of a null pointer
if (CHAIN_SPACE_LEN(*firstchainp) == 0) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../buffer.c:130:30: note: expanded from macro 'CHAIN_SPACE_LEN'
#define CHAIN_SPACE_LEN(ch) ((ch)->flags & EVBUFFER_IMMUTABLE ? \
Despite the presence of 'sys/queue.h' in some stdlib implementations
(i.e. uclibc) 'LIST_HEAD' macro can be missing. This fix defines this
macro in the same manner as was done previously for 'TAILQ_'.
Fixes: #539Closes: #639 (cherry-picked)
Backport: 2.1.9
In the case of iocp, in the for loop above, there is a situation where:
remaining == chain->off == 0
And this happens due to CHAIN_PINNED_R() case (that is used only in
buffer_iocp.c)
Closes: #630 (picked)
"chain" cannot be NULL here because we have at least one chain (we
handle empty buffer separatelly) and hence loop will be executed at
least once.
Link: 841ecbd961 (commitcomment-23631347)
Signed-off-by: Ivan Maidanski <i.maidanski@samsung.com>
Signed-off-by: Azat Khuzhin <a3at.mail@gmail.com>
@EMPanisset reported a problem (#358) with evbuffer_remove_buffer(), but
actually I think that the problem is in evbuffer_add_buffer() which introduces
this empty chain, all other callers (except evbuffer_prepend_buffer(), but it
doesn't have this problem though) should be safe.
And FWIW the only API that allows empty chains is evbuffer_add_reference(), and
we can add check there to avoid such issues, but for now I leaved this without
fixing, since I think that evbuffer_add_reference() with empty chains can be
used as a barrier (but this can be tricky).
Fixes: regress evbuffer/remove_buffer_with_empty2
v2: introduce/fixes evbuffer/add_buffer_with_empty
evbuffer_add() would always put data in the last chain, even if there
was available space in a previous chain, and in doing so it also
failed to update last_with_datap, causing subsequent calls to other
functions that do look at last_with_datap to add data in the middle
of the evbuffer instead of at the end.
Fixes the evbuffer_add() part of issue #335, and the evbuffer/add2 and
evbuffer/add3 tests, and also prevents wasting space available in the
chain pointed to by last_with_datap.
These types are not part of POSIX. As we only use them in a small number
of places, we'd better replace them by C standard types. This makes a
larger part of the code build for CloudABI.
For this fix, we need to make sure that passing too-large inputs to
the evbuffer functions can't make us do bad things with the heap.
Also, lower the maximum chunk size to the lower of off_t, size_t maximum.
This is necessary since otherwise we could get into an infinite loop
if we make a chunk that 'misalign' cannot index into.
Since the bufferevents' events are now EV_FINALIZE (name pending),
they won't deadlock. To clean up properly, though, we must use the
finalization feature.
This patch also split bufferevent deallocation into an "unlink" step
that happens fast, and a "destruct" step that happens after
finalization.
More work is needed: there needs to be a way to specify a finalizer
for the bufferevent's argument itself. Also, this finalizer business
makes lots of the reference counting we were doing unnecessary.
Also, more testing is needed.
