3640 Commits

Author SHA1 Message Date
David Paschich
285188963d Fix bufferevent_pair to properly set BEV_EVENT_{READING,WRITING} on flush.
Here's some fun. From `bufferevent.h`:
  ```
  #define BEV_EVENT_READING	0x01	/**< error encountered while reading */
  #define BEV_EVENT_WRITING	0x02	/**< error encountered while writing */
  ```

And from `event.h`:
  ```
  /** Wait for a socket or FD to become readable */
  #define EV_READ		0x02
  /** Wait for a socket or FD to become writeable */
  #define EV_WRITE	0x04
  ```

Library users have to be very careful to get this right; it turns out, the
library itself got this wrong in the `bufferevent_pair` code. It appears that
in most of the code, only `BEV_EVENT_FINISHED` will indicate whether it's read
or write; on error or timeout, it appears that "both" is assumed and not set in
the callback. I read through all the other places where `BEV_EVENT_FINISHED` is
passed to an event callback; it appears that the pair code is the only spot
that got it wrong.

azat: add TT_FORK to avoid breaking clean env, and rebase commit message
(copied from #359)
Fixes: #359
2016-06-17 18:21:45 +03:00
Azat Khuzhin
2a4bf29470 test/http: avoid using conditionals with omitted operands (fixes VS2015)
But we need to add VS2015 build on appveyor.

Fixes: #361
Reported-by: @nntrab
2016-06-17 17:04:37 +03:00
Azat Khuzhin
aabf1c2dd1 test/http: don't mix declarations and code (fixes -Wdeclaration-after-statement) 2016-06-17 16:07:15 +03:00
Azat Khuzhin
c08d90ba5a test/buffer: fix leak in test_evbuffer_prepend() 2016-06-17 16:06:12 +03:00
Azat Khuzhin
e7d1e3926b test/buffer: avoid errors with --no-fork (reinitialize static vars) 2016-06-17 15:44:13 +03:00
Rainer Keller
b5ca365626 For non GCC/clang on OSX the -Wno-deprecated-declarations may not be valid
Closes: nmathewson/Libevent#129
2016-06-08 14:31:29 +03:00
Azat Khuzhin
41e54464b9 Merge branch 'travis-ci-os-matrix-v2'
This increases libevent coverage to:
- os:osx

- cmake -DEVENT__DISABLE_MM_REPLACEMENT=ON
- cmake -DEVENT__ENABLE_VERBOSE_DEBUG=ON
- configure --disable-openssl
- configure --disable-thread-support
- configure --disable-malloc-replacement

- fix travis-ci builds under automake >1.11

Possible failures after this patch set (not always, IOW in some builds this
issues aren't real issues):
- some failures but mostly because of timing issues, must be fixed separately.
- https://travis-ci.org/azat/libevent/jobs/129430229 # on brew update
- https://travis-ci.org/azat/libevent/jobs/129430221 # some locking issues

* travis-ci-os-matrix-v2:
  automake: define serial-tests only if automake have this option
  test/automake: don't use paralell test harness (since automake 1.12)
  travis-ci/osx: relink gcc/g++ instead of clang
  travis-ci: enable multi-os mode (osx, linux)
  travis-ci: increase matrix (--disable-foo)
  travis-ci: adjust alignment

Fixes: #356
Travis-CI: https://travis-ci.org/azat/libevent/builds/129430181
2016-05-12 00:48:15 +03:00
Azat Khuzhin
61179dec77 automake: define serial-tests only if automake have this option
P.S. I did try a lot of other ways, but seems that this is the only one that
will work.

Fixes: automake 1.11
Travis-CI: https://travis-ci.org/azat/libevent/jobs/129398265
2016-05-11 16:47:46 +03:00
Azat Khuzhin
44d755e00e test/automake: don't use paralell test harness (since automake 1.12)
Starting from automake 1.2 there is parallel test harness, that redirects all
output to some log, which serial-test doesn't do.
So in case of new runner we can get no output for 10 minutes, for example on my
desktop:
  $ time make verify VERBOSE=1
  PASS: test/test-script.sh
  ============================================================================
  Testsuite summary for libevent 2.1.5-beta
  ============================================================================
  # TOTAL: 1
  # PASS:  1
  # SKIP:  0
  # XFAIL: 0
  # FAIL:  0
  # XPASS: 0
  # ERROR: 0
  ============================================================================

  real    25m31.735s
  user    0m13.753s
  sys     0m7.648s

And this means that this will fail on travis-ci, since it has timeout for 10
minutes. Sure we can use `travis wait 60` instead, but I think that it is
better to fix this by writing result to output, instead of hacking around, so
let's use serial-tests instead of parallel always.
And now it works on travis-ci under linux because it has automake 1.11 while
osx has at least 1.12.

Links:
https://docs.travis-ci.com/user/common-build-problems/
https://www.gnu.org/software/automake/manual/html_node/Serial-Test-Harness.html#Serial-Test-Harness
https://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html

CI:
https://travis-ci.org/azat/libevent/jobs/129171497 # ok on linux
https://travis-ci.org/azat/libevent/jobs/129171532 # no output for 10 min on osx
2016-05-11 14:28:47 +03:00
Azat Khuzhin
481481d3f5 travis-ci/osx: relink gcc/g++ instead of clang
Since by default osx replaces /usr/bin/gcc with it's apple clang, while we have
gcc in our build matrix, so use real gcc.
2016-05-10 18:17:25 +03:00
Azat Khuzhin
79917e48ba travis-ci: enable multi-os mode (osx, linux)
- use addons.apt.packages instead of `apt-get install`
- add `brew update`/`brew install ...`

Link: https://docs.travis-ci.com/user/installing-dependencies/
Link: https://docs.travis-ci.com/user/multi-os/
2016-05-10 08:29:56 +03:00
Azat Khuzhin
59649f74c2 travis-ci: increase matrix (--disable-foo)
Use --disable-foo from
https://github.com/libevent/libevent/wiki/ReleaseChecklist
2016-05-09 23:30:56 +03:00
Azat Khuzhin
c8be33952e travis-ci: adjust alignment 2016-05-09 18:58:42 +03:00
Azat Khuzhin
38e0f4a5d6 be_openssl: clear all pending errors before SSL_*() calls
Refs: #350
Reported-by: @CapSel
2016-05-02 15:26:27 +03:00
Azat Khuzhin
ce38993333 Ignore all pkgconfig generated stuff
In the referenced commit new *.pc added, and I think it is better to ignore
them all.

Refs: b8d7c6211a965c19c7c5de414135ff13b5fa2476 ("libevent_core and
libevent_extra also deserve a pkgconfig file")
2016-04-21 02:16:21 +03:00
Azat Khuzhin
e77ff4187f test/buffer: cover n_add_for_cb when evbuffer_prepend() need to allocate buffer
Regression-for: 0abd0393eaf029e1ead8a09b479ea6830f7152ee ("Fix n_add_for_cb in
evbuffer_prepend() in case of new buffer required")
2016-04-21 02:15:08 +03:00
Azat Khuzhin
bd19a28ac4 test/tinytest_macros: add new one tt_nstr_op() 2016-04-21 02:15:08 +03:00
Azat Khuzhin
0abd0393ea Fix n_add_for_cb in evbuffer_prepend() in case of new buffer required
Signed-off-by: @luoming1224
Fixes: #349
2016-04-21 01:59:30 +03:00
Jan Heylen
b8d7c6211a libevent_core and libevent_extra also deserve a pkgconfig file 2016-04-20 07:39:42 +02:00
Azat Khuzhin
d5e634a80e Merge branch 'be-filter-outputcb-disable-during-execution'
* be-filter-outputcb-disable-during-execution:
  test/bufferevent: check that output_filter disabled during processing output
  be_filter: actually disable output_filter during processing output
2016-04-18 13:39:54 +03:00
Azat Khuzhin
ae2881254a test/bufferevent: check that output_filter disabled during processing output
Regression-for: c031215d532c97f1d82efd672ecd622d31d3342d ("be_filter: actually
disable output_filter during processing output")
2016-04-18 13:39:03 +03:00
Simon Perreault
c031215d53 be_filter: actually disable output_filter during processing output
IOW: Make the code do what the comment says it should do.
2016-04-18 13:32:05 +03:00
Azat Khuzhin
2a71b33233 listener: unlock lev on error in listener_read_cb()
Without this patch:
  $ regress --no-fork +listener/error_unlock
  listener/error_unlock: [warn] Error from accept() call: Too many open files
  [err] ../evthread.c:220: Assertion lock->count == 0 failed in ../evthread.c
  Aborted (core dumped)

Fixes: #341
Fixes: listener/error_unlock
2016-04-01 00:04:35 +03:00
Azat Khuzhin
7d85651694 test/listener: regression for missing unlock in listener_read_cb()
P.S. it triggers even without pthread, but this makes checks more strict.

Refs: #341
2016-04-01 00:04:35 +03:00
Azat Khuzhin
b59ef3bd63 Merge remote-tracking branch 'origin/pr/339'
* origin/pr/339:
  evbuffer_add: Use last_with_datap if set, not last.
  test/regress: add tests for evbuffer_add() breakage on empty last chain

Fixes: #335
2016-03-29 19:26:18 +03:00
Marcus Sundberg
a8769ef12d evbuffer_add: Use last_with_datap if set, not last.
evbuffer_add() would always put data in the last chain, even if there
was available space in a previous chain, and in doing so it also
failed to update last_with_datap, causing subsequent calls to other
functions that do look at last_with_datap to add data in the middle
of the evbuffer instead of at the end.

Fixes the evbuffer_add() part of issue #335, and the evbuffer/add2 and
evbuffer/add3 tests, and also prevents wasting space available in the
chain pointed to by last_with_datap.
2016-03-26 20:52:07 +01:00
Marcus Sundberg
d5ee739153 test/regress: add tests for evbuffer_add() breakage on empty last chain
The evbuffer/add* tests currenly break on 2.0.21, 2.0.22 and 2.1 HEAD
due to issue #335. The evbuffer/reference2 test breaks on 2.0.21 and
2.0.22 due to commit b18c04dd not being applied.
2016-03-26 20:51:46 +01:00
Azat Khuzhin
bddad71e51 test/http: fix running some tests sequential (with --no-fork)
After this patch
$ regress --no-fork +http/..

Passed without failures.
2016-03-25 11:32:33 +03:00
Azat Khuzhin
cbc3209d61 test/http: localize evhttp server structure 2016-03-25 11:20:11 +03:00
Azat Khuzhin
24b521499a evhttp_have_expect(): fix -Wlogical-not-parentheses
../http.c:589:6: warning: logical not is only applied to the left hand side of this comparison [-Wlogical-not-parentheses]
        if (!req->kind == EVHTTP_REQUEST || !REQ_VERSION_ATLEAST(req, 1, 1))
                        ^          ~~
2016-03-25 10:21:48 +03:00
Azat Khuzhin
ec65c42052 evdns: fix searching empty hostnames
From #332:
  Here follows a bug report by **Guido Vranken** via the _Tor bug bounty program_. Please credit Guido accordingly.

  ## Bug report

  The DNS code of Libevent contains this rather obvious OOB read:

  ```c
  static char *
  search_make_new(const struct search_state *const state, int n, const char *const base_name) {
      const size_t base_len = strlen(base_name);
      const char need_to_append_dot = base_name[base_len - 1] == '.' ? 0 : 1;
  ```

  If the length of ```base_name``` is 0, then line 3125 reads 1 byte before the buffer. This will trigger a crash on ASAN-protected builds.

  To reproduce:

  Build libevent with ASAN:
  ```
  $ CFLAGS='-fomit-frame-pointer -fsanitize=address' ./configure && make -j4
  ```
  Put the attached ```resolv.conf``` and ```poc.c``` in the source directory and then do:

  ```
  $ gcc -fsanitize=address -fomit-frame-pointer poc.c .libs/libevent.a
  $ ./a.out
  =================================================================
  ==22201== ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60060000efdf at pc 0x4429da bp 0x7ffe1ed47300 sp 0x7ffe1ed472f8
  READ of size 1 at 0x60060000efdf thread T0
  ```

P.S. we can add a check earlier, but since this is very uncommon, I didn't add it.

Fixes: #332
2016-03-25 00:37:46 +03:00
Azat Khuzhin
d7348bab60 test/dns: regression for empty hostname
Refs: #332
2016-03-25 00:33:43 +03:00
Azat Khuzhin
d49a65879f test/http: fix SERVER_TIMEOUT tests under win32
Seems that the hack with filling BACKLOG didn't work on win32, and hence we
stuck in write() waiting, not in connect()

And:
$ time regress http/cancel_server_timeout
- on linux: 10secs
- on win32: 2-5secs

I tried to debug this but you can't sniff TCP packages (wireshark/rawpcap) on
localhost in windows xp (according to [RAWPCAP] and my testing).

RAWPCAP: http://www.netresec.com/?page=RawCap
2016-03-24 21:01:16 +03:00
Azat Khuzhin
376f107318 test/http: add a helper for creating timedout/failed request 2016-03-24 20:28:20 +03:00
Azat Khuzhin
d02a2858fa test/http: adopt for C90 (mixed code and declarations) 2016-03-24 14:11:28 +03:00
Azat Khuzhin
4db15e09a9 evdns: avoid double-free in evdns_base_free() for probing requests
http/cancel_by_host_no_ns:
    OK ../test/regress_http.c:1384: assert(regress_dnsserver(data->base, &portnum, search_table))
    OK ../test/regress_http.c:1387: assert(dns_base)
    OK ../test/regress_http.c:1423: assert(evcon)
         OK ../test/regress_http.c:1444: assert(evhttp_make_request(evcon, req, EVHTTP_REQ_GET, "/delay") != -1): 0 vs -1
         OK ../test/regress_http.c:1455: assert(test_ok == 2): 2 vs 2
         OK ../test/regress_http.c:1480: assert(evhttp_make_request(evcon, req, EVHTTP_REQ_GET, "/test") != -1): 0 vs -1[msg] Nameserver 127.0.0.1:55948 has failed: request timed out.
[msg] All nameservers have failed

    OK ../test/regress_http.c:1274: assert(!req)
         OK ../test/regress_http.c:1505: assert(evhttp_make_request(evcon, req, EVHTTP_REQ_GET, "/test") != -1): 0 vs -1
    OK ../test/regress_http.c:1274: assert(!req)==19199== Invalid read of size 8
==19199==    at 0x4CC285: evdns_cancel_request (evdns.c:2849)
==19199==    by 0x4CEDB2: evdns_nameserver_free (evdns.c:4018)
==19199==    by 0x4CEF5B: evdns_base_free_and_unlock (evdns.c:4052)
==19199==    by 0x4CF13B: evdns_base_free (evdns.c:4088)
==19199==    by 0x4617A3: http_cancel_test (regress_http.c:1518)
==19199==    by 0x490A78: testcase_run_bare_ (tinytest.c:105)
==19199==    by 0x490D5A: testcase_run_one (tinytest.c:252)
==19199==    by 0x491699: tinytest_main (tinytest.c:434)
==19199==    by 0x47E0E0: main (regress_main.c:461)
==19199==  Address 0x61e56d0 is 0 bytes inside a block of size 48 free'd
==19199==    at 0x4C2AE6B: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==19199==    by 0x4AAFFF: event_mm_free_ (event.c:3516)
==19199==    by 0x4C5ADD: request_finished (evdns.c:693)
==19199==    by 0x4CEE95: evdns_base_free_and_unlock (evdns.c:4040)
==19199==    by 0x4CF13B: evdns_base_free (evdns.c:4088)
==19199==    by 0x4617A3: http_cancel_test (regress_http.c:1518)
==19199==    by 0x490A78: testcase_run_bare_ (tinytest.c:105)
==19199==    by 0x490D5A: testcase_run_one (tinytest.c:252)
==19199==    by 0x491699: tinytest_main (tinytest.c:434)
==19199==    by 0x47E0E0: main (regress_main.c:461)
==19199==  Block was alloc'd at
==19199==    at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==19199==    by 0x4AAEB2: event_mm_calloc_ (event.c:3459)
==19199==    by 0x4CAAA2: nameserver_send_probe (evdns.c:2327)
==19199==    by 0x4C50FF: nameserver_prod_callback (evdns.c:494)
==19199==    by 0x4A564C: event_process_active_single_queue (event.c:1646)
==19199==    by 0x4A5B95: event_process_active (event.c:1738)
==19199==    by 0x4A6296: event_base_loop (event.c:1961)
==19199==    by 0x4A5C1D: event_base_dispatch (event.c:1772)
==19199==    by 0x46172C: http_cancel_test (regress_http.c:1507)
==19199==    by 0x490A78: testcase_run_bare_ (tinytest.c:105)
==19199==    by 0x490D5A: testcase_run_one (tinytest.c:252)
==19199==    by 0x491699: tinytest_main (tinytest.c:434)
==19199==
2016-03-24 14:09:04 +03:00
Azat Khuzhin
5830e331e1 Merge branch 'bufev-cancellations-v5'
This patch set fixes bufferevent via http request cancellations (connect() and
dns-request), it survives tests and cancel.. with --no-fork, so this must be ok
(though I have one patch for dns layer pending).
But I'm not sure about cancel.. unit tests on win32, will fix disable them
later if they will differs (plus maybe we must make them skip-by-default?).

Fixes: #333

* bufev-cancellations-v5:
  http: set fd to -1 unconditioally, to avoid leaking of DNS requests
  test/http: cover NS timed out during request cancellations separatelly
  http: avoid leaking of fd in evhttp_connection_free()
  http: get fd from be layer during connection reset
  be_sock: cancel in-progress dns requests
  evdns: export cancel via callbacks in util (like async lib core/extra issues)
  test/http: request cancellation with resolving/{conn,write}-timeouts in progress
2016-03-24 14:06:19 +03:00
Azat Khuzhin
7a4b472925 http: set fd to -1 unconditioally, to avoid leaking of DNS requests
Otherwise:
http/cancel_by_host_ns_timeout_inactive_server: [msg] Nameserver 127.0.0.1:37035 has failed: request timed out.
[msg] All nameservers have failed
OK
1 tests ok.  (0 skipped)
==26211==
==26211== FILE DESCRIPTORS: 3 open at exit.
==26211== Open file descriptor 2: /dev/pts/47
==26211==    <inherited from parent>
==26211==
==26211== Open file descriptor 1: /dev/pts/47
==26211==    <inherited from parent>
==26211==
==26211== Open file descriptor 0: /dev/pts/47
==26211==    <inherited from parent>
==26211==
==26211==
==26211== HEAP SUMMARY:
==26211==     in use at exit: 1,112 bytes in 5 blocks
==26211==   total heap usage: 149 allocs, 144 frees, 18,826 bytes allocated
==26211==
==26211== 40 bytes in 1 blocks are indirectly lost in loss record 1 of 5
==26211==    at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26211==    by 0x4AAEB2: event_mm_calloc_ (event.c:3459)
==26211==    by 0x498F5B: evbuffer_add_cb (buffer.c:3309)
==26211==    by 0x4A0EF5: bufferevent_socket_new (bufferevent_sock.c:366)
==26211==    by 0x4BFADF: evhttp_connection_base_bufferevent_new (http.c:2375)
==26211==    by 0x4BFC8F: evhttp_connection_base_new (http.c:2427)
==26211==    by 0x460DAA: http_cancel_test (regress_http.c:1417)
==26211==    by 0x490A78: testcase_run_bare_ (tinytest.c:105)
==26211==    by 0x490D5A: testcase_run_one (tinytest.c:252)
==26211==    by 0x491699: tinytest_main (tinytest.c:434)
==26211==    by 0x47E0E0: main (regress_main.c:461)
==26211==
==26211== 136 bytes in 1 blocks are indirectly lost in loss record 2 of 5
==26211==    at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26211==    by 0x4AAEB2: event_mm_calloc_ (event.c:3459)
==26211==    by 0x491FF0: evbuffer_new (buffer.c:365)
==26211==    by 0x49A1BE: bufferevent_init_common_ (bufferevent.c:300)
==26211==    by 0x4A0E44: bufferevent_socket_new (bufferevent_sock.c:353)
==26211==    by 0x4BFADF: evhttp_connection_base_bufferevent_new (http.c:2375)
==26211==    by 0x4BFC8F: evhttp_connection_base_new (http.c:2427)
==26211==    by 0x460DAA: http_cancel_test (regress_http.c:1417)
==26211==    by 0x490A78: testcase_run_bare_ (tinytest.c:105)
==26211==    by 0x490D5A: testcase_run_one (tinytest.c:252)
==26211==    by 0x491699: tinytest_main (tinytest.c:434)
==26211==    by 0x47E0E0: main (regress_main.c:461)
==26211==
==26211== 136 bytes in 1 blocks are indirectly lost in loss record 3 of 5
==26211==    at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26211==    by 0x4AAEB2: event_mm_calloc_ (event.c:3459)
==26211==    by 0x491FF0: evbuffer_new (buffer.c:365)
==26211==    by 0x49A1FB: bufferevent_init_common_ (bufferevent.c:305)
==26211==    by 0x4A0E44: bufferevent_socket_new (bufferevent_sock.c:353)
==26211==    by 0x4BFADF: evhttp_connection_base_bufferevent_new (http.c:2375)
==26211==    by 0x4BFC8F: evhttp_connection_base_new (http.c:2427)
==26211==    by 0x460DAA: http_cancel_test (regress_http.c:1417)
==26211==    by 0x490A78: testcase_run_bare_ (tinytest.c:105)
==26211==    by 0x490D5A: testcase_run_one (tinytest.c:252)
==26211==    by 0x491699: tinytest_main (tinytest.c:434)
==26211==    by 0x47E0E0: main (regress_main.c:461)
==26211==
==26211== 536 bytes in 1 blocks are indirectly lost in loss record 4 of 5
==26211==    at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26211==    by 0x4AAEB2: event_mm_calloc_ (event.c:3459)
==26211==    by 0x4A0E15: bufferevent_socket_new (bufferevent_sock.c:350)
==26211==    by 0x4BFADF: evhttp_connection_base_bufferevent_new (http.c:2375)
==26211==    by 0x4BFC8F: evhttp_connection_base_new (http.c:2427)
==26211==    by 0x460DAA: http_cancel_test (regress_http.c:1417)
==26211==    by 0x490A78: testcase_run_bare_ (tinytest.c:105)
==26211==    by 0x490D5A: testcase_run_one (tinytest.c:252)
==26211==    by 0x491699: tinytest_main (tinytest.c:434)
==26211==    by 0x47E0E0: main (regress_main.c:461)
==26211==
==26211== 1,112 (264 direct, 848 indirect) bytes in 1 blocks are definitely lost in loss record 5 of 5
==26211==    at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26211==    by 0x4AAEB2: event_mm_calloc_ (event.c:3459)
==26211==    by 0x4D0564: evdns_getaddrinfo (evdns.c:4685)
==26211==    by 0x4B13BA: evutil_getaddrinfo_async_ (evutil.c:1575)
==26211==    by 0x4A139E: bufferevent_socket_connect_hostname (bufferevent_sock.c:524)
==26211==    by 0x4C02DB: evhttp_connection_connect_ (http.c:2588)
==26211==    by 0x4C04DD: evhttp_make_request (http.c:2643)
==26211==    by 0x4615FF: http_cancel_test (regress_http.c:1504)
==26211==    by 0x490A78: testcase_run_bare_ (tinytest.c:105)
==26211==    by 0x490D5A: testcase_run_one (tinytest.c:252)
==26211==    by 0x491699: tinytest_main (tinytest.c:434)
==26211==    by 0x47E0E0: main (regress_main.c:461)
==26211==
==26211== LEAK SUMMARY:
==26211==    definitely lost: 264 bytes in 1 blocks
==26211==    indirectly lost: 848 bytes in 4 blocks
==26211==      possibly lost: 0 bytes in 0 blocks
==26211==    still reachable: 0 bytes in 0 blocks
==26211==         suppressed: 0 bytes in 0 blocks
2016-03-24 14:01:31 +03:00
Azat Khuzhin
0c343afea9 test/http: cover NS timed out during request cancellations separatelly 2016-03-24 14:00:44 +03:00
Azat Khuzhin
f0e1341179 http: avoid leaking of fd in evhttp_connection_free()
Since we do close fd there if we don't have BEV_OPT_CLOSE_ON_FREE, and
evcon->fd can be incorrect (non -1), so just get it from the underlying
bufferevent to fix this.

And after this patch the following tests report 0 instead of 2307 fd leaks:
$ valgrind --leak-check=full --show-reachable=yes --track-fds=yes --error-exitcode=1 regress --no-fork http/cancel..
==11299== FILE DESCRIPTORS: 3 open at exit.

And this is stdin/stderr/stdout.
2016-03-23 12:53:51 +03:00
Azat Khuzhin
4a53c54bf7 http: get fd from be layer during connection reset
Since it can be non -1, and we must close it, otherwise we will have problems.

And after this patch the following tests report fd 2307 instead of 2309 fd leaks:
$ valgrind --leak-check=full --show-reachable=yes --track-fds=yes --error-exitcode=1 regress --no-fork http/cancel..
==10853== FILE DESCRIPTORS: 2307 open at exit.
2016-03-23 12:53:03 +03:00
Azat Khuzhin
86dfd2ced1 be_sock: cancel in-progress dns requests
Before this patch we didn't have such functionality and this can cause some
issues when we are cancelling HTTP request while after this we will get a
response/timeout from the NS and in this case error_cb will be called and can
damage newly started HTTP request.
We could also have problems with connect, but we don't have them since we
closes the fd in HTTP layer.

This is not so good that this is done in be_sock, but it is internal, so we can
change without pain. Plus I don't like that callback-via-evutil, but since we
have event_extra we need do like that.

And after this patch the following tests doesn't report leaks:
$ valgrind --leak-check=full --show-reachable=yes --track-fds=yes --error-exitcode=1 regress --no-fork http/cancel..
...
==10469== FILE DESCRIPTORS: 2309 open at exit.
...
==10469== HEAP SUMMARY:
==10469==     in use at exit: 0 bytes in 0 blocks
==10469==   total heap usage: 33,846 allocs, 33,846 frees, 4,617,651 bytes allocated
==10469==
==10469== All heap blocks were freed -- no leaks are possible

v2: do under lock
v3: reset dns request
v4: ignore EVUTIL_EAI_CANCEL in regular callback
2016-03-23 12:51:44 +03:00
Azat Khuzhin
8cbe65d5f4 evdns: export cancel via callbacks in util (like async lib core/extra issues) 2016-03-23 12:46:47 +03:00
Azat Khuzhin
334340da51 test/http: request cancellation with resolving/{conn,write}-timeouts in progress
This patch adds 8 new tests:
- http/cancel
- http/cancel_by_host
- http/cancel_by_host_no_ns
- http/cancel_by_host_inactive_server
- http/cancel_inactive_server
- http/cancel_by_host_no_ns_inactive_server
- http/cancel_by_host_server_timeout
- http/cancel_server_timeout
- http/cancel_by_host_no_ns_server_timeout

This patches not 100% for http layer, but more for be_sock, but it was simpler
to add them here, plus it also shows some bugs with fd leaking in http layer.

Right now we have next picture (we can also play with timeouts/attempts for
evdns to make tests fail, IOW to track the failures even without valgrind):
$ valgrind --leak-check=full --show-reachable=yes --track-fds=yes --error-exitcode=1 regress --no-fork http/cancel..
http/cancel: OK
http/cancel_by_host: OK
http/cancel_by_host_no_ns: [msg] Nameserver 127.0.0.1:42489 has failed: request timed out.
[msg] All nameservers have failed
OK
http/cancel_by_host_inactive_server: OK
http/cancel_inactive_server: OK
http/cancel_by_host_no_ns_inactive_server: [msg] Nameserver 127.0.0.1:51370 has failed: request timed out.
[msg] All nameservers have failed
OK
http/cancel_by_host_server_timeout: OK
http/cancel_server_timeout: OK
http/cancel_by_host_no_ns_server_timeout: [msg] Nameserver 127.0.0.1:45054 has failed: request timed out.
[msg] All nameservers have failed
OK
9 tests ok.  (0 skipped)
==3202==
==3202== FILE DESCRIPTORS: 2309 open at exit.
...
==8403== HEAP SUMMARY:
==8403==     in use at exit: 1,104 bytes in 5 blocks
==8403==   total heap usage: 10,916 allocs, 10,911 frees, 1,458,818 bytes allocated
==8403==
==8403== 40 bytes in 1 blocks are indirectly lost in loss record 1 of 5
==8403==    at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8403==    by 0x4AAD2D: event_mm_calloc_ (event.c:3459)
==8403==    by 0x498E48: evbuffer_add_cb (buffer.c:3309)
==8403==    by 0x4A0DE2: bufferevent_socket_new (bufferevent_sock.c:366)
==8403==    by 0x4BF8BA: evhttp_connection_base_bufferevent_new (http.c:2369)
==8403==    by 0x4BFA6A: evhttp_connection_base_new (http.c:2421)
==8403==    by 0x460CFC: http_cancel_test (regress_http.c:1413)
==8403==    by 0x490965: testcase_run_bare_ (tinytest.c:105)
==8403==    by 0x490C47: testcase_run_one (tinytest.c:252)
==8403==    by 0x491586: tinytest_main (tinytest.c:434)
==8403==    by 0x47DFCD: main (regress_main.c:461)
==8403==
==8403== 136 bytes in 1 blocks are indirectly lost in loss record 2 of 5
==8403==    at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8403==    by 0x4AAD2D: event_mm_calloc_ (event.c:3459)
==8403==    by 0x491EDD: evbuffer_new (buffer.c:365)
==8403==    by 0x49A0AB: bufferevent_init_common_ (bufferevent.c:300)
==8403==    by 0x4A0D31: bufferevent_socket_new (bufferevent_sock.c:353)
==8403==    by 0x4BF8BA: evhttp_connection_base_bufferevent_new (http.c:2369)
==8403==    by 0x4BFA6A: evhttp_connection_base_new (http.c:2421)
==8403==    by 0x460CFC: http_cancel_test (regress_http.c:1413)
==8403==    by 0x490965: testcase_run_bare_ (tinytest.c:105)
==8403==    by 0x490C47: testcase_run_one (tinytest.c:252)
==8403==    by 0x491586: tinytest_main (tinytest.c:434)
==8403==    by 0x47DFCD: main (regress_main.c:461)
==8403==
==8403== 136 bytes in 1 blocks are indirectly lost in loss record 3 of 5
==8403==    at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8403==    by 0x4AAD2D: event_mm_calloc_ (event.c:3459)
==8403==    by 0x491EDD: evbuffer_new (buffer.c:365)
==8403==    by 0x49A0E8: bufferevent_init_common_ (bufferevent.c:305)
==8403==    by 0x4A0D31: bufferevent_socket_new (bufferevent_sock.c:353)
==8403==    by 0x4BF8BA: evhttp_connection_base_bufferevent_new (http.c:2369)
==8403==    by 0x4BFA6A: evhttp_connection_base_new (http.c:2421)
==8403==    by 0x460CFC: http_cancel_test (regress_http.c:1413)
==8403==    by 0x490965: testcase_run_bare_ (tinytest.c:105)
==8403==    by 0x490C47: testcase_run_one (tinytest.c:252)
==8403==    by 0x491586: tinytest_main (tinytest.c:434)
==8403==    by 0x47DFCD: main (regress_main.c:461)
==8403==
==8403== 528 bytes in 1 blocks are indirectly lost in loss record 4 of 5
==8403==    at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8403==    by 0x4AAD2D: event_mm_calloc_ (event.c:3459)
==8403==    by 0x4A0D02: bufferevent_socket_new (bufferevent_sock.c:350)
==8403==    by 0x4BF8BA: evhttp_connection_base_bufferevent_new (http.c:2369)
==8403==    by 0x4BFA6A: evhttp_connection_base_new (http.c:2421)
==8403==    by 0x460CFC: http_cancel_test (regress_http.c:1413)
==8403==    by 0x490965: testcase_run_bare_ (tinytest.c:105)
==8403==    by 0x490C47: testcase_run_one (tinytest.c:252)
==8403==    by 0x491586: tinytest_main (tinytest.c:434)
==8403==    by 0x47DFCD: main (regress_main.c:461)
==8403==
==8403== 1,104 (264 direct, 840 indirect) bytes in 1 blocks are definitely lost in loss record 5 of 5
==8403==    at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8403==    by 0x4AAD2D: event_mm_calloc_ (event.c:3459)
==8403==    by 0x4D0326: evdns_getaddrinfo (evdns.c:4682)
==8403==    by 0x4B1213: evutil_getaddrinfo_async_ (evutil.c:1568)
==8403==    by 0x4A1255: bufferevent_socket_connect_hostname (bufferevent_sock.c:517)
==8403==    by 0x4C00B6: evhttp_connection_connect_ (http.c:2582)
==8403==    by 0x4C02B8: evhttp_make_request (http.c:2637)
==8403==    by 0x4614EC: http_cancel_test (regress_http.c:1496)
==8403==    by 0x490965: testcase_run_bare_ (tinytest.c:105)
==8403==    by 0x490C47: testcase_run_one (tinytest.c:252)
==8403==    by 0x491586: tinytest_main (tinytest.c:434)
==8403==    by 0x47DFCD: main (regress_main.c:461)
==8403==
==8403== LEAK SUMMARY:
==8403==    definitely lost: 264 bytes in 1 blocks
==8403==    indirectly lost: 840 bytes in 4 blocks
==8403==      possibly lost: 0 bytes in 0 blocks
==8403==    still reachable: 0 bytes in 0 blocks
==8403==         suppressed: 0 bytes in 0 blocks
2016-03-23 12:46:10 +03:00
Azat Khuzhin
dbff101b59 be_sock: evutil_getaddrinfo_async_() always return 0 2016-03-23 12:12:13 +03:00
Azat Khuzhin
927ab33f3b test/http: exit from the loop in the errorcb to wait cancellation
This will make cancellation tests more graceful, that said that error_cb can
not be called sometimes if you will break the loop in cancel.

Plus drop that define for function generations, since function body changed,
and it is not generic anymore, plus that macro didn't used by anyone else.
2016-03-23 12:11:29 +03:00
Azat Khuzhin
351207f400 regress_clean_dnsserver(): reset global port vars
This will fix some test chains with --no-fork.
2016-03-23 12:09:37 +03:00
Azat Khuzhin
d4054928c7 http: make fallback for EVHTTP_CON_READ_ON_WRITE_ERROR more cleaner 2016-03-11 20:59:58 +03:00
Azat Khuzhin
2ff164abac http: fix EVHTTP_CON_READ_ON_WRITE_ERROR when it doesn't supported by OS
For example win32 doesn't accept such things (maybe via overloaded IO, I'm not
sure), also I looked into curl and seems that the behaviour is the same (IOW
like with EVHTTP_CON_READ_ON_WRITE_ERROR on linux/win32).

Fixes: https://ci.appveyor.com/project/nmathewson/libevent/build/2.1.5.216#L499 (win32)
Fixes: 680742e1665b85487f10c0ef3df021e3b8e98634 ("http: read server response
even after server closed the connection")
v2: v0 was just removing that flag, i.e. make it deprecated and set_flags() will return -1
2016-03-11 20:59:58 +03:00
Azat Khuzhin
3b581693ac test/http: read_on_write_error: fix it for win32
Fixes: https://ci.appveyor.com/project/nmathewson/libevent/build/2.1.5.216#L499 (win32)
2016-03-11 20:59:58 +03:00