This is necessary for making some thread libraries work with
event.c, and might get better performance with others.
The biggest change required here was that we needed to make some
internal code that had previously called event_add and event_del
call the nolock variants.
The original code needlessly called open in its two- or three-
argument format depending on the O_CREAT flag; this should not be
needed.
The code also leaked an fd if fcntl() failed.
Reported by Dave Hart.
(Tweaked by nickm: Fix up the arcr4andom_buf OSX hack so that the
fallback case isn't compiled into the code when we have
arc4random_buf() and we are not on OSX. Also add a comment
explaining what's up.)
Back when deferred_cb stuff had its own queue, the queue was always
executed, but we never ran more than 16 callbacks per iteration.
That made for two problems:
1: Because deferred_cb stuff would always run, and had no priority,
it could cause priority inversion.
2: It doesn't respect the max_dispatch_interval code.
Then, when I refactored deferred_cb to be a special case of
event_callback, that solved the above issues, but made for two more
issues:
3: Because deferred_cb stuff would always get the default priority,
it could could low-priority bufferevents to get too much priority.
4: With code like bufferevent_pair, it's easy to get into a
situation where two deferreds keep adding one another, preventing
the event loop from ever actually scanning for more events.
This commit fixes the above by giving deferreds a better notion of
priorities, and by limiting the number of deferreds that can be
added to the _current_ loop iteration's active queues. (Extra
deferreds are put into the active_later state.)
That isn't an all-purpose priority inversion solution, of course: for
that, you may need to mess around with max_dispatch_interval.
An event or event callback can now be in an additional state: "active
later". When an event is in this state, it will become active the
next time we run through the event loop. This lets us do what we
wanted to with deferred callbacks: make a type of active thing that
avoids infinite circular regress in a way that starves other events or
exhausts the stack. It improves on deferred callbacks by respecting
priorities, and by having a non-kludgy way to avoid event starvation.
This shouldn't have any visible effect, but it's necessary or
advisible for a few changes and cleanups I would like to make,
including:
* Replacing the deferred queue with a type that works more as if it
were an event.
* Introducing a useful "activate this on the next round through the
event loop" state for events and deferreds.
* Adding an "on until further notice" status for events, to allow a
saner win32-hybrid approach.
* Eventually, making all user callbacks first-class things with
event-like semantics.
