You could previously do this with evbuffer_peek() and some memcpys,
but it was a bit more work than most folks wanted to get into.
Closes sourceforge ticket 3108072
Instead of having a file segment born as one type and stay that way
forever, let them start out unmapped, but map themselves as needed
if they need to get written out on a non-drains_to_fd evbuffer.
The sendfile() implementation for evbuffer_add_file is potentially more
efficient, but it has a problem: you can only use it to send bytes over
a socket using sendfile(). If you are writing bytes via SSL_send() or
via a filter, or if you need to be able to inspect your buffer, it
doesn't work.
As an easy fix, this patch disables the sendfile-based implementation of
evbuffer_add_file on an evbuffer unless the user sets a new
EVBUFFER_FLAG_DRAINS_TO_FD flag on that evbuffer, indicating that the
evbuffer will not be inspected, but only written out via
evbuffer_write(), evbuffer_write_atmost(), or drained with stuff like
evbuffer_drain() or evbuffer_add_buffer(). This flag is off by
default, except for evbuffers used for output on bufferevent_socket.
In the future, it could be interesting to make a best-effort file
segment implementation that tries to send via sendfile, but mmaps on
demand. That's too much complexity for a stable release series, though.
Our evbuffer_strchr() function [which was only used for
search_eol(EOL_LF) could give incorrect results if it found its answer
in the first chunk but didn't start searching from the front of the
chunk.
Also, this patch adds unit tests for evbuffer_search_eol, particularly
in those cases that evbuffer_readln() tests didn't exercise.
Importantly, we don't actually want to call evbuffer_write() when
the buffer is empty. This makes it an error to ever get a -1 return
value from evbuffer_add_file(), which makes it safe for us to test
the return value.
The EVUTIL_CLOSESOCKET() macro required you to include unistd.h in your
source for POSIX. We might as well turn it into a function: an extra
function call is going to be cheap in comparison with the system call.
We retain the EVUTIL_CLOSESOCKET() macro as an alias for the new
evutil_closesocket() function.
(commit message from email by Nick and Sebastian)
The previous evbuffer_expand was not only incorrect; it was
inefficient too. On all questions of time vs memory tradeoffs, it
chose to burn time in order to avoid wasting memory. The new code
tries to be a little more balanced: it only resizes an existing chain
when doing so doesn't require too much copying, and when failing to do
so would waste a lot of the chain's space.
This patch also rewrites evbuffer_chain_insert to work properly with
last_with_datap, and adds a few convenience functions to buffer.c.
To implement evbuffer_expand() properly, you need to be able to
replace the last chunk that has data, which means that we need to keep
track of the the next pointer pointing to the last_with_data chunk,
not the last_with_data chunk itself.
This actually makes some of the code a lot simpler. The only
ones that actually used previous_to_last for anything were reserving
and committing space.
This is the first patch in a series to replace previous_to_last with
last_with_data. Currently, we can only use two partially empty chains
at the end of an evbuffer, so if we have one with 511 bytes free, and
another with 512 bytes free, and we try to do a 1024 byte read, we
can't just stick another chain on the end: we need to reallocate the
last one. That's stupid and inefficient.
Instead, this patch adds a last_with_data pointer to eventually
replace previous_to_last. Instead of pointing to the penultimated
chain (if any) as previous_to_last does, last_with_data points to the
last chain that has any data in it, if any. If all chains are empty,
last_with_data points to the first chain. If there are no chains,
last_with_data is NULL.
The next step is to start using last_with_data everywhere that we
currently use previous_to_last. When that's done, we can remove
previous_to_last and the code that maintains it.
