with the demise of ESXi, I am looking for alternatives. Currently I have PfSense virtualized on four physical NICs, a bunch of virtual ones, and it works great. Does Proxmox do this with anything like the ease of ESXi? Any other ideas?
with the demise of ESXi, I am looking for alternatives. Currently I have PfSense virtualized on four physical NICs, a bunch of virtual ones, and it works great. Does Proxmox do this with anything like the ease of ESXi? Any other ideas?
And in about 2 years you’ll switch to LXD/Incus. :P
I’m currently off work with a broken shoulder, have you just given me a project?
Ahahaha that’s up to you. All best for your shoulder!
Incus looks cool. Have you virtualised a firewall on it? Is it as flexible as proxmox in terms of hardware passthrough options?
I find zero mentions online of opnsense on incus. 🤔
Yes it does run, but BSD-based VMs running on Linux have their details as usual. This might be what you’re looking for: https://discuss.linuxcontainers.org/t/run-freebsd-13-1-opnsense-22-7-pfsense-2-7-0-and-newer-under-lxd-vm/15799
Since you want to run a firewall/router you can ignore LXD’s networking configuration and use your opnsense to assign addresses and whatnot to your other containers. You can created whatever bridges / vlan-based interface on your base system and them assign them to profiles/containers/VMs. For eg. create a
cbr0network bridge usingsystemd-networkand then runlxc profile device add default eth0 nic nictype=bridged parent=cbr0 name=eth0this will usecbr0as the default bridge for all machines and LXD won’t provide any addressing or touch the network, it will just create aneth0interface on those machines attached to the bridge. Then your opnsense can be on the same bridge and do DHCP, routing etc. Obviously you can passthrough entire PCI devices to VMs and containers if required as well.When you’re searching around for help, instead of “Incus” you can search for “LXD” as it tend to give you better results. Not sure if you’re aware but LXD was the original project run by Canonical, recently it was forked into Incus (and maintained by the same people who created LXD at Canonical) to keep the project open under the Linux Containers initiative.
OPNsense running in the Incus live demo. Fun!
Enjoy your 30 min of Incus :P
With Incus only officially supported in Debian 13, and LXD on the way out, should I get going with LXD and migrate to Incus later? Or use the Zabbly repo and switch over to official Debian repos when they become available? What’s the recommended trajectory, would you say?
It depends on how fast you want updates. I’m sure you know how Debian works, so if you install LXD from Debian 12 repositories you’ll be on 5.0.2 LTS most likely for ever. If you install from Zabbly you’ll get the latest and greatest right now.
My companies’ machines are all running LXD from Debian repositories, except for two that run from Zabbly for testing and whatnot. At home I’m running from Debian repo. Migration from LXD 5.0.2 to a future version of Incus with Debian 13 won’t be a problem as Incus is just a fork and stgraber and other members of the Incus/LXC projects work very closely or also work in Debian.
Debian users will be fine one way or the other. I specifically asked stgraber about what’s going to happen in the future and this was his answer:
I hope this helps you decide.
Absolutely. Great intel; thank you!
Very informative, thank you.
I am generally very comfortable with Linux, but somehow this seems intimidating.
Although I guess I’m not using proxmox for anything other than managing VMs, network bridges and backups. Well, and for the feeling of using something that was set up by people who know what they’re doing and not hacked together by me until it worked…
And LXD/Incus can do that as well for you. Install it an by running
incus initit will ask you a few questions and get an automated setup with networking, storage etc. all running and ready for you to create VMs/Containers.What I was saying is that you can also ignore the default / automated setup and install things manually if you’ve other requirements.
Okay, I think I found a bit of a catch with Incus or LXD. I want a solution with a web UI, and while Incus has one, it seems to have access control either browser certificate based or with a central auth server. Neither are a good solution for me - I would much prefer regular user auth with the option to use an auth server at some point (but I don’t want to take all of this on all at once.)
I hope it’s okay that I keep coming back to you with these questions. You seem to be a strong Incus-evangelist. :)
I guess I could only expose the web UI on localhost and create an SSH tunnel in order to use it…? Not so good on mobile though, which is the strongest reason for a webui.
You aren’t wrong, the WebUI is stateless, it doesn’t know of any users nor it stores any other context information.
The certificates are required for the UI client to authenticate with the underlying LXD server itself. Much like the SSH authentication is boils down to creating a public/private key pair and the PK is added to your browser(s) and the public key to the server. I believe this is a good walkthrough of the process for anyone starting out.
At work we use Authelia and HAProxy to get around the need to distribute a certificate for each client / mange our logins with SSO and 2FA. At home I simply use Nginx as a reverse proxy to the WebUI with the
proxy_ssl_certificatepassing a certificate down to it. Here another configuration example of how to use Nginx to pass the certificate, you can then use Basic HTTP Auth to add a simple username/password to it.Thanks for your patience. I appreciate it and I’m learning a lot. 🙏
There’s a chance yet!
edit: That actually seems simple enough and should integrate nicely with the rest of my network. Cool!