Guide on using Rauthy as headless mode or with CLI

[type-checker]

Rauthy currently includes a default admin interface, which is great for getting started. However, for production use cases, we should provide a guide on how to run Rauthy in headless mode or via the CLI — that is, using just the server without the frontend.

[sebadob]

Rauthy has no CLI / headless mode.

It has the complete API available and you could manage it via only direct API calls (with an API key with proper access rights), but there is no CLI interface at all. If you would want to manage it via the terminal only, you would need to use curl or similar to do this, which is of course very inconvenient for daily use.

A complete CLI interface could be created, because all the types and stuff do exist already, but that would be a huge amount of work. I thought about a CLI tool for generating all necessary files for a deployment, which could take care of generating all configs, secrets, and so on, but never planned a full API CLI interface so far.

If you don't want the UI and instead a CLI interface only, have you taken a look at kanidm? It takes exactly that approach and afaik does not even come with an admin UI, but CLI only.

[type-checker]

I took some time to review both kanidm and rauthy again. Actually, you don't need to create a CLI since you've already set up the API endpoints. The issue, however, is that for most operations like creating OAuth Client, we first need to log in as an admin, and this currently requires a browser. Is it possible to log in as an admin using only the API, without the need for SPOW or a browser?

[sebadob]

You can operate it fully headless, if you really want, apart from a single thing, and that is configuring upstream auth providers. I don't see any problems with the setup. You can do anything you need, even inside CI pipelines and stuff like that, if you like.

Is there a specific reason why bootstrapping—and potentially the overall architecture—isn't fully headless by default, with the frontend positioned as an optional bonus layer?

Why should it? Why waste so many hours on trying to write a good CLI interface for very complex types, while the whole operation is just so much more convenient via a UI, where you have input validation, type checking, and all these things? Doing stuff via CLI is great, as long as you don't have complex types, where it gets super annoying to type everything out. For instance, when I modify K8s resources, I never use the patch CLI, because it's just so annoying to manually type out a complex JSON in the terminal. I always use edit to have a more convenient approach.

Additionally, the UI makes it possible for people to operate it on a daily basis who are not so technically advanced that they would even think about doing anything via CLI. Rauthy is for instance used in some places, where customers operate all user-related stuff themselves, without any issues or support requests at all.

On top of that, bootstrapping would need to be done via CLI in a similar way to get inital access securely and I don't see a problem with it. It's like a cloud-init or other template files.

Other stuff like events and auditing via CLI becomes overwhelming quickly as well. You can miss stuff so easily, when you are just inside a terminal. Apart from that, as said already, you can do almost anything you need.

And the last thing is, that it's just way more secure via UI, because of Webauthn and passkeys. No stored credentials inside a text file on your disk, but instead a hardware token which cannot be hacked at all. Yes, you can use Security Keys via CLI, but this does not work flawlessly on all platforms yet afaik.

[type-checker]

Thanks for the detailed explanation. I have one question regarding HIQLITE: is it designed for horizontal scalability, or is it limited to vertical scaling? I'm a bit confused because it seems like we're reading directly from the filesystem rather than using a centralized SQLite server. If that's the case, wouldn't horizontally scaling across multiple nodes cause issues with syncing transactional data? Could you clarify how that’s handled?

[sebadob]

All this information exists already, you just have to read it:

• https://github.com/sebadob/hiqlite/blob/main/README.md
• https://github.com/sebadob/hiqlite/blob/main/ARCHITECTURE.md

[type-checker]

I've read those shared docs before, and from what I understand, HIQLITE offers horizontal scalability for reads — with multiple nodes replicating data to enable fast local reads. However, for writes, everything is serialized through the Raft leader, meaning it still follows SQLite’s single-writer constraint as it doesn't run as a server. So essentially, write scalability is limited, and the system will primarily scale vertically.

Is my understanding correct?

Also, if this is how it works in a Kubernetes setup, I’m curious—why are we recommending a StatefulSet deployment? Wouldn’t a regular Deployment with a ReadWriteMany PVC and automated backups be simpler to manage? It seems like we could still automate the HQL Node ID in that setup. I’d like to understand the downsides of this approach and the reasoning behind preferring StatefulSet over a standard Deployment.

[sebadob]

(...) meaning it still follows SQLite’s single-writer constraint (...)

No matter what you do or how you twist and turn it, that will always be the limitation of SQLite. But don't be fooled by "only a single writer at a time" in terms of performance. hiqlite has been designed in a way that this actually became a strength because all other work than simply "speaking to the database", even serialization and error handling, are offloaded into different threads and will never block the writer, as explained in the linked readmes.

hiqlite will never(!) be a limitation when using Rauthy. The bottleneck will always be your resources for password hashing or simply too slow disks that cannot keep up with the traffic. In fact, I don't even have access to hardware big enough that I could use and check when / if Postgres even pulls ahead. Postgres multi-writer capabilities come with pretty big downsides. Locking and unlocking tables / rows is a lot more expensive and takes way more time than it does with SQLite. On top of it, the hqilite design makes it possible that the writer does not even need to care about locking. And it does not need to fsync all the time like postgres does, because the database can be rebuilt at any time from the Raft logs if it should crash and become corrupted.

The next hiqlite release will even have ~28% performance gains as well. When I tested some things on the weekend, I achieved up to 80k single inserts / second on a Ryzen 9950X with pretty quick memory and a Gen4 SSD. hiqlite can saturate your SSD until you become I/O bound, even with its single writer design. If you would be in the position where you would even barely reach the limits of the hiqlite performance, you would not even think about using it. You would most probably run a dedicated Postgres cluster and have a DBA who only takes care of your database as a full time job. A 1GBit/s connection just for network traffic would not be enough anymore at that point as well, as I was able to max out 1GBit/s between 60k and 70k inserts / s with only database replication traffic. This does not even include reads, as they are local only.

The rule is pretty simple. If you don't need HA, just use a single node. If you need HA, use 3. If you want to use Postgres because you have it running anyway, or you feel better with it, use Postgres.

Also, if this is how it works in a Kubernetes setup, I’m curious—why are we recommending a StatefulSet deployment? Wouldn’t a regular Deployment with a ReadWriteMany PVC and automated backups be simpler to manage?

That are totally different architectures and if you would do that, you would have a SPoF again. Apart from that, RWX volumes are typically insanely slow in comparison and have very low IOPS, as they need to manage multiple reads / writes and use internal locking. Most K8s storage providers don't even offter RWX volumes and if they do, they are most often NFS volumes or similar. I would never recommend anyone to use them, apart from maybe for stuff like backups or something like that.

And, if you have a volume attached, you use a StatefulSet by definition. That's it. That's what they are for.