You have to trust the servers with your metadata, and that the servers have their inter-server communication locked down, but at least you can choose/operate servers.
Some clients are a bit flaky with their e2e encryption defaults or from a UI perspective it is easy to send an unencrypted message (in a new chat for example) before noticing that was how it was set.
There are a few XEPs the server needs which enable things like OMEMO, efficient mobile data/battery use, offline and multiple device deliverability, file transfers, etc. Audio/video calling has various requirements as I think xmpp only facilitates the setup of the call.
That rules it out for me then. I like to use XMPP+OMEMO with about 4-5 clients which I can continue a conversation with at any time. Main mobile, tablet, desktop, other desktop, and backup mobile which is usually switched off. (Even if a device has been missing for too long and run out of OMEMO keys, the keys sync up again once I send a message with it.)