relayd: fix DEBUG build feedback and OK tb@

usr.sbin/relayd: add support for PROXY protocol in TCP relays patch from Christoph Liebender OK: rsadowski@

Let relayd make use of libagentx. No functional change intended. OK tb@ Enthousiasm from Mischa Peters

Rewrite the agentx code of relayd. This new framework should allow us to add new objects easier if so desired and should handle a lot more corner-cases. This commit should also fix the following: - On most (all) tables it omits the *Entry elements, making it not map to OPENBSD-RELAYD-MIB.txt. - sstolen returns the size of the sockaddr_in{,6}, instead of the sin{,6}_addr resulting in garbage data to be put in the ip-field. - relaydSessionPortIn and relaydSessionPortOut are swapped - relaydSessions only uses relaydSessionIndex, while OPENBSD-RELAYD-MIB.txt says it should have 2 indices - miscellaneous minor things related to the AGENTX-protocol, like wonky index handeling and returning NOSUCHINSTANCE where NOSUCHOBJECT should be returned, etc. This commit does remove traps, but it's large enough as is and I intent on adding it soon(tm). It also deprecates the snmp keyword in favour of an agentx keyword. The snmp keyword is still available, but will be removed in the future. Tweaks and OK denis@ on the relayd parts Tweaks and OK claudio@ on the agentx parts "Get it in" deraadt@

Allow relayd to compile without reaching back into snmpd directory for agentx header. OK denis@

no need to generate y.tab.h if nothing uses it, set YFLAGS to nothing instead of CLEANFILES += y.tab.h okay millert@

Migrate relayd to use libtls for TLS. Still does the TLS privsep via the engine but at least we can use a sane API for new features. Going in now so it is possible to work with this in tree. General agreement at d2k17.

Add -Wcast-qual and cast away one false positive where we use a const u_char * variable for an iovec from within the libcrypto engine callback. OK millert@

Move local logging functions from log.c to new file util.c (that is also shared with relayctl). This allows us to unify common log.c with other daemons. It also clarifies the Copyright: log.c is by Henning, relayd's additions were from me. No functional or code changes, but it will make future updates easier.

LibreSSL now supports loading of CA certificates from memory, replace the internal and long-serving ssl_ctx_load_verify_memory() function with a call to the SSL_CTX_load_verify_mem() API function. The ssl_privsep.c file with hacks for using OpenSSL in privsep'ed processes can now go away; portable versions of smtpd and relayd should start depending on LibreSSL or they have to carry ssl_privsep.c in openbsd-compat to work with legacy OpenSSL. No functional change. Based on previous discussions with gilles@ bluhm@ and many others OK bluhm@ (as part of the libcrypto/libssl/libtls diff)

The OpenSSL engine passes a "const u_char *" to the callback but relayd's RSA privsep engine uses an iovec that expects a non-cast "void *". Cast it and disable the -Wcast-qual warning because I don't want to copy the data and I didn't find a better way to implement it. ok guenther@

Introduce privsep for private keys: - Move RSA private keys to a new separate process instead of copying them to the relays. A custom RSA engine is used by the SSL/TLS code of the relay processes to send RSA private key encryption/decryption (also used for sign/verify) requests to the new "ca" processes instead of operating on the private key directly. - Each relay process gets its own related ca process. Setting "prefork 5" in the config file will spawn 10 processes (5 relay, 5 ca). This diff also reduces the default number of relay processes from 5 to 3 which should be suitable in most installations without a very heavy load. - Don't keep text versions of the keys in memory, parse them once and keep the binary representation. This might still be the case in OpenSSL's internals but will be fixed in the library. This diff doesn't prevent something like "heartbleed" but adds an additional mitigation to prevent leakage of the private keys from the processes doing SSL/TLS. With feedback from many ok benno@

Adapt relayd to use AgentX protocol to send traps ok reyk@ benno@

Remove -Wbounded: it is now the compiler default.

Move the HTTP code into an extra file to make future changes easier to follow. No functional changes, only one function got renamed. ok benno@

Fix reload support in relayd(8) by reimplementing large parts of the daemon infrastructure. The previous design made it fairly hard to reload the complex data structures, especially relays and protocols. One of the reasons was that the privsep'd relayd processes had two ways of getting their configuration: 1) from memory after forking from the parent process and 2) and (partially) via imsgs after reload. The new implementation first forks the privsep'd children before the parents loads the configuration and sends it via imsgs to them; so it is only like 2) before. It is based on an approach that I first implemented for iked(8) and I also fixed many bugs in the code. Thanks to many testers including dlg@ sthen@ phessler@ ok pyr@ dlg@ sthen@

Reorganize the relayd code to use the proc.c privsep API/commodity functions that are based on work for iked and smtpd. This simplifies the setup of privsep processes and moves some redundant and repeated code to a single place - which is always good from a quality and security point of view. The relayd version of proc.c is different to the current version in iked because it uses 1:N communications between processes, eg. a single parent process is talking to many forked relay children while iked only needs 1:1 communications. ok sthen@ pyr@

Move imsg into libutil and add a man page. Minor bump for libutil. Previous versions of this diff and man page looked at by various people. "you should just commit" deraadt

add new 'router' functionality to dynamically add or remove routes based on health check results, using the existing table syntax. this allows to maintain multiple (uplink) gateways to implement link balancing or WAN link failover if no routing protocol or other keepalive method is available. works fine with or without net.inet.ip.multipath enabled. ok pyr@, jmc@ for manpages

Use OpenBSD's knuth shuffle algorithm of random values from bind to produce the DNS request ids instead of a simple per-request arc4random(). This ensure randomness but also satisfies the non-repeating property we need. ok deraadt@

Marry relayd with snmpd using new "send trap" option: Request to send a SNMP trap when the state of a host changes. relayd(8) will try to (re-)connect to snmpd(8) and request it to send a trap to the registered trap receivers, see snmpd.conf(5) for more information about the configuration. ok pyr@ thib@

hoststated gets renamed to relayd. easier to type, and actually says what the daemon does - it is a relayer that pays attention to the status of pools of hosts; not a status checkers that happens to do some relaying

allow to add labels to protocol actions, they will be printed in http error pages and can be used to refer to additional information. ok pyr@

Introduce two new functions to be able to load certificates while already chrooted and with privileges dropped. This is the very first step in being able to reload a layer 7 configuration. not ok reyk who's away but should be glad to see this in.

add support for relaying DNS traffic (with a little bit of packet header randomization). this adds an infrastructure to support UDP-based protocols. ok gilles@, tested by some