- http/basic_trailing_space -- covers cases when there is trailing space
after the request line (nginx handles this)
- http/simple_nonconformant -- covers non RFC3986 conformant URIs
This is important, as otherwise clients can easily exhaust the file
descriptors available on a libevent HTTP server, which can cause
problems in other code which does not handle EMFILE well: for example,
see https://github.com/bitcoin/bitcoin/issues/11368Closes: #578 (patch cherry picked)
This is the second hunk of the first patch
5ff8eb26371c4dc56f384b2de35bea2d87814779 ("Fix crashing http server when
callback do not reply in place")
Fixes: #567
General http callback looks like:
static void http_cb(struct evhttp_request *req, void *arg)
{
evhttp_send_reply(req, HTTP_OK, "Everything is fine", NULL);
}
And they will work fine becuase in this case http will write request
first, and during write preparation it will disable *read callback* (in
evhttp_write_buffer()), but if we don't reply immediately, for example:
static void http_cb(struct evhttp_request *req, void *arg)
{
return;
}
This will leave connection in incorrect state, and if another request
will be written to the same connection libevent will abort with:
[err] ../http.c: illegal connection state 7
Because it thinks that read for now is not possible, since there were no
write.
Fix this by disabling EV_READ entirely. We couldn't just reset callbacks
because this will leave EOF detection, which we don't need, since user
hasn't replied to callback yet.
Reported-by: Cory Fields <cory@coryfields.com>
Tests:
- http/https_simple_dirty # not affected, since dirty is the default
- http/https_simple # affected
v2: fix compilation with -DEVENT__DISABLE_OPENSSL=ON
Like in `sockaddr_in` structure in /usr/include/netinet/in.h
@azat: convert all other users (bench, compat, ..) and tweak message
Fixes: #178Fixes: #196
Refs: 6bf1ca78
Link: https://codereview.appspot.com/156040043/#msg4
Seems that the hack with filling BACKLOG didn't work on win32, and hence we
stuck in write() waiting, not in connect()
And:
$ time regress http/cancel_server_timeout
- on linux: 10secs
- on win32: 2-5secs
I tried to debug this but you can't sniff TCP packages (wireshark/rawpcap) on
localhost in windows xp (according to [RAWPCAP] and my testing).
RAWPCAP: http://www.netresec.com/?page=RawCap
This patch adds 8 new tests:
- http/cancel
- http/cancel_by_host
- http/cancel_by_host_no_ns
- http/cancel_by_host_inactive_server
- http/cancel_inactive_server
- http/cancel_by_host_no_ns_inactive_server
- http/cancel_by_host_server_timeout
- http/cancel_server_timeout
- http/cancel_by_host_no_ns_server_timeout
This patches not 100% for http layer, but more for be_sock, but it was simpler
to add them here, plus it also shows some bugs with fd leaking in http layer.
Right now we have next picture (we can also play with timeouts/attempts for
evdns to make tests fail, IOW to track the failures even without valgrind):
$ valgrind --leak-check=full --show-reachable=yes --track-fds=yes --error-exitcode=1 regress --no-fork http/cancel..
http/cancel: OK
http/cancel_by_host: OK
http/cancel_by_host_no_ns: [msg] Nameserver 127.0.0.1:42489 has failed: request timed out.
[msg] All nameservers have failed
OK
http/cancel_by_host_inactive_server: OK
http/cancel_inactive_server: OK
http/cancel_by_host_no_ns_inactive_server: [msg] Nameserver 127.0.0.1:51370 has failed: request timed out.
[msg] All nameservers have failed
OK
http/cancel_by_host_server_timeout: OK
http/cancel_server_timeout: OK
http/cancel_by_host_no_ns_server_timeout: [msg] Nameserver 127.0.0.1:45054 has failed: request timed out.
[msg] All nameservers have failed
OK
9 tests ok. (0 skipped)
==3202==
==3202== FILE DESCRIPTORS: 2309 open at exit.
...
==8403== HEAP SUMMARY:
==8403== in use at exit: 1,104 bytes in 5 blocks
==8403== total heap usage: 10,916 allocs, 10,911 frees, 1,458,818 bytes allocated
==8403==
==8403== 40 bytes in 1 blocks are indirectly lost in loss record 1 of 5
==8403== at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8403== by 0x4AAD2D: event_mm_calloc_ (event.c:3459)
==8403== by 0x498E48: evbuffer_add_cb (buffer.c:3309)
==8403== by 0x4A0DE2: bufferevent_socket_new (bufferevent_sock.c:366)
==8403== by 0x4BF8BA: evhttp_connection_base_bufferevent_new (http.c:2369)
==8403== by 0x4BFA6A: evhttp_connection_base_new (http.c:2421)
==8403== by 0x460CFC: http_cancel_test (regress_http.c:1413)
==8403== by 0x490965: testcase_run_bare_ (tinytest.c:105)
==8403== by 0x490C47: testcase_run_one (tinytest.c:252)
==8403== by 0x491586: tinytest_main (tinytest.c:434)
==8403== by 0x47DFCD: main (regress_main.c:461)
==8403==
==8403== 136 bytes in 1 blocks are indirectly lost in loss record 2 of 5
==8403== at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8403== by 0x4AAD2D: event_mm_calloc_ (event.c:3459)
==8403== by 0x491EDD: evbuffer_new (buffer.c:365)
==8403== by 0x49A0AB: bufferevent_init_common_ (bufferevent.c:300)
==8403== by 0x4A0D31: bufferevent_socket_new (bufferevent_sock.c:353)
==8403== by 0x4BF8BA: evhttp_connection_base_bufferevent_new (http.c:2369)
==8403== by 0x4BFA6A: evhttp_connection_base_new (http.c:2421)
==8403== by 0x460CFC: http_cancel_test (regress_http.c:1413)
==8403== by 0x490965: testcase_run_bare_ (tinytest.c:105)
==8403== by 0x490C47: testcase_run_one (tinytest.c:252)
==8403== by 0x491586: tinytest_main (tinytest.c:434)
==8403== by 0x47DFCD: main (regress_main.c:461)
==8403==
==8403== 136 bytes in 1 blocks are indirectly lost in loss record 3 of 5
==8403== at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8403== by 0x4AAD2D: event_mm_calloc_ (event.c:3459)
==8403== by 0x491EDD: evbuffer_new (buffer.c:365)
==8403== by 0x49A0E8: bufferevent_init_common_ (bufferevent.c:305)
==8403== by 0x4A0D31: bufferevent_socket_new (bufferevent_sock.c:353)
==8403== by 0x4BF8BA: evhttp_connection_base_bufferevent_new (http.c:2369)
==8403== by 0x4BFA6A: evhttp_connection_base_new (http.c:2421)
==8403== by 0x460CFC: http_cancel_test (regress_http.c:1413)
==8403== by 0x490965: testcase_run_bare_ (tinytest.c:105)
==8403== by 0x490C47: testcase_run_one (tinytest.c:252)
==8403== by 0x491586: tinytest_main (tinytest.c:434)
==8403== by 0x47DFCD: main (regress_main.c:461)
==8403==
==8403== 528 bytes in 1 blocks are indirectly lost in loss record 4 of 5
==8403== at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8403== by 0x4AAD2D: event_mm_calloc_ (event.c:3459)
==8403== by 0x4A0D02: bufferevent_socket_new (bufferevent_sock.c:350)
==8403== by 0x4BF8BA: evhttp_connection_base_bufferevent_new (http.c:2369)
==8403== by 0x4BFA6A: evhttp_connection_base_new (http.c:2421)
==8403== by 0x460CFC: http_cancel_test (regress_http.c:1413)
==8403== by 0x490965: testcase_run_bare_ (tinytest.c:105)
==8403== by 0x490C47: testcase_run_one (tinytest.c:252)
==8403== by 0x491586: tinytest_main (tinytest.c:434)
==8403== by 0x47DFCD: main (regress_main.c:461)
==8403==
==8403== 1,104 (264 direct, 840 indirect) bytes in 1 blocks are definitely lost in loss record 5 of 5
==8403== at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8403== by 0x4AAD2D: event_mm_calloc_ (event.c:3459)
==8403== by 0x4D0326: evdns_getaddrinfo (evdns.c:4682)
==8403== by 0x4B1213: evutil_getaddrinfo_async_ (evutil.c:1568)
==8403== by 0x4A1255: bufferevent_socket_connect_hostname (bufferevent_sock.c:517)
==8403== by 0x4C00B6: evhttp_connection_connect_ (http.c:2582)
==8403== by 0x4C02B8: evhttp_make_request (http.c:2637)
==8403== by 0x4614EC: http_cancel_test (regress_http.c:1496)
==8403== by 0x490965: testcase_run_bare_ (tinytest.c:105)
==8403== by 0x490C47: testcase_run_one (tinytest.c:252)
==8403== by 0x491586: tinytest_main (tinytest.c:434)
==8403== by 0x47DFCD: main (regress_main.c:461)
==8403==
==8403== LEAK SUMMARY:
==8403== definitely lost: 264 bytes in 1 blocks
==8403== indirectly lost: 840 bytes in 4 blocks
==8403== possibly lost: 0 bytes in 0 blocks
==8403== still reachable: 0 bytes in 0 blocks
==8403== suppressed: 0 bytes in 0 blocks
This will make cancellation tests more graceful, that said that error_cb can
not be called sometimes if you will break the loop in cancel.
Plus drop that define for function generations, since function body changed,
and it is not generic anymore, plus that macro didn't used by anyone else.
With greater buffer it can't be written with one writev(2), and hence we can
trigger more tricky cases, like calling writecb/readcb more then once.
Refs: #321
