Getting somewhat closer to releasing a new version of #swad. I now improved the functionality to execute something on a different worker thread: Use an in-memory queue, providing a #lockfree version. This gives me a consistent reliable throughput of 3000 requests/s (with outliers up to 4500 r/s) at an average response time of 350 - 400 ms (with TLS enabled). For waking up worker threads, I implemented different backends as well: kqueue, eventfd and event-ports, the fallback is still a self-pipe.

So, #portability here really means implement lots of different flavors of the same thing.

Looking at these startup logs, you can see that #kqueue (#FreeBSD and other BSDs) is really a "jack of all trades", being used for "everything" if available (and that's pretty awesome, it means one single #syscall per event loop iteration in the generic case). #illumos' (#Solaris) #eventports come somewhat close (but need a lot more syscalls as there's no "batch registering" and certain event types need to be re-registered every time they fired), they just can't do signals, but illumos offers Linux-compatible signalfd. Looking at #Linux, there's a "special case fd" for everything. Plus #epoll also needs one syscall for each event to be registered. The "generic #POSIX" case without any of these interfaces is just added for completeness

Now that #swad 0.7 is released, it's time to prepare a new release of #poser, my own lib supporting #services on #POSIX systems, following a #reactor with #threadpool design.

During development of swad, I moved poser from using strictly only POSIX APIs (with the scalability limits of e.g. #select) to auto-detected support for #kqueue, #epoll, #eventports, #signalfd and #timerfd (so now it could, in theory(!), "compete" with e.g. libevent). I also fixed quite some hidden bugs, and added more base functionality, like a #dictionary using nested hashtables internally, or #async tasks mimicking the async/await pattern known from e.g, #csharp. I also deprecated two features, the periodic and global "service tick" (superseded by individual timers) and the "resolve hosts" property of a "connection" (superseded by a separate resolve class).

I'll have to decide on a few things, e.g. whether I'll remove the deprecated stuff immediately and bump the major version of the "posercore" lib. I guess I'll do just that. I'd also like to add all the web-specific stuff (http 1.0/1.1 server) that's currently part of the swad code as a "poserweb" lib. This would get a major version of 0, indicating a generally unstable API/ABI as of now....

And then, I'd have to decide where certain utility classes belong to. The rate limiter is probably useful for things other than web, so it should probably go to core. What about url encoding/decoding, for example?

Stay tuned, something will come here, maybe helping you to write a nice service in plain #C :

https://github.com/Zirias/poser

The next release of #swad will probably bring not a single new feature, but focus on improvements, especially regarding #performance. Support for using #kqueue (#FreeBSD et al) to handle #signals is a part of it (which is done and works). Still unsure whether I'll also add support for #Linux' #signalfd. Using kqueue also as a better backend for #timers is on the list.

Another hopefully quite relevant change is here:

https://github.com/Zirias/poser/commit/798f23547295f89fa0c751f0e707c3474b5c689c

In short, so far my #poser lib was always awaiting readiness notification (from kqueue, or #epoll on Linux, or select/poll for other platforms) before doing any read or write on a socket. This is the ideal approach for reads, because in the common case, a socket is NOT ready for reading ... our kernel must have received something from the remote end first. But for writes, it's not so ideal. The common case is that a socket IS ready to write (because there's space left in the kernel's send buffers). So, just try it, and only register for notifications if it ever fails, makes more sense. Avoids pointless waiting and pointless events, and e.g. with epoll, even unnecessary syscalls.

Still working on #swad, and currently very busy with improving quality, most of the actual work done inside my #poser library.

After finally supporting #kqueue and #epoll, I now integrated #xxhash to completely replace my previous stupid and naive hashing. I also added a more involved #dictionary class as an alternative to the already existing #hashtable. While the hashtable's size must be pre-configured and collissions are only ever resolved by storing linked lists, the new dictionary dynamically nests multiple hashtables (using different bits of a single hash value). I hope to achieve acceptable scaling while maintaining also acceptable memory overhead that way ...

#swad already uses both container classes as appropriate.

Next I'll probably revisit poser's #threadpool. I think I could replace #pthread condition variables by "simple" #semaphores, which should also reduce overhead ...

https://github.com/Zirias/swad

First change since #swad 0.2 will actually be a (huge?) improvement to my #poser lib. So far, it was hardwired to use the good old #POSIX #select call. This is perfectly fine for handling around up to 100 (or at least less than 1000, YMMV) clients.

Some #select implementations offer defining the upper limit for checked file descriptors. Added support for that.

POSIX also specifies #poll, which has very similar #scalability issues, but slightly different. Added support for this as well.

And then, I went on to add support for the #Linux-specific #epoll and #BSD-specific #kqueue (#FreeBSD, #NetBSD, #OpenBSD, ...) which are both designed to *solve* any scalability issues

A little thing that slightly annoyed me about kqueue was that there's no support for temporarily changing the signal mask, so I had to do the silly dance shown in the screenshot. OTOH, it offers changing event filters and getting events in a single call, which I might try to even further optimize ...

Version 4.8.0 of #syslog_ng is now available in #FreeBSD ports: https://wiki.freebsd.org/Ports/sysutils/syslog-ng

This version of syslog-ng has some FreeBSD-specific improvements: #opentelemetry support is now available, and more efficient directory monitoring thanks to a new, #kqueue-based implementation.

minor new on my #Xmoji (#X11 #emoji #keyboad): The #kqueue backend for watching the config file now works on #FreeBSD (as far as I can tell, hard to test every possible edge case).

I also want to add #inotify for #Linux. I just realized I'll need a Linux machine to properly do that. So, installing Debian in bhyve. Joined my samba domain, mounted my #kerberos-encrypted #NFS homedirs, couldn't login, no idea where to find diagnostic output with dreaded systemd ... but right now I found the solution to the problem:

# ln -s /usr/bin/zsh /usr/local/bin/zsh