In the past I have used Proxmox with ZFS raid on a basic mini PC. With ZFS raid it syncs everything except /boot. Proxmox has a tool called “proxmox-boot-tool-refresh” which will syncs /boot between drives. ZFS kernel module can be loaded in the initramfs so it will boot fine, even if missing a drive.

For this project I do not plan to use ZFS, but AFAIK software raid is now standard. Here is a popular video from Level1Techs talking about the flaws of hardware RAID: https://youtu.be/l55GfAwa8RI

Thank you for the detailed reply. You seem very knowledgeable. I will implement your suggestions as I redesign my network.

Thanks. Some of these entries maybe (20%) have IOMMU groups listed under “lspci_all”. But it is extremely awkward to search through. So maybe I will put a feature request in the forum to make IOMMU more searchable. But this is still likely the largest database of IOMMU groupings on the web, even if it is not easily searchable.

Thanks but these are only lists of CPUs and motherboards that support IOMMU, not the IOMMU groups. For me (and many others) the groupings are just as important as whether there is support at all.

The groupings are defined by the motherboard. In my experience, all motherboards that support IOMMU will put at least 1 PCIe slot in its own own group, which is good for Graphics Card passthrough. However, the grouping of other stuff like SATA controllers and NICs varies wildly between board, and that is what I am interested in.

Thank you, that is a very good point, I never thought of that. Just to confirm, best standard practice is for every connection, even as simple as a Nextcloud server accessing an NFS server, to go through the firewall?

Then I could just have one interface per host but use Proxmox host ID as the VLAN so they are all unique. Then, I would make a trunk on the guest OPNsense VM. In that way it is a router on a stick.

I was a bit hesitant to do firewall rules based off of IP addresses, as a compromised host could change its IP address. However, if each host is on its own VLAN, then I could add a firewall rule to only allow through the 1 “legitimate” IP per VLAN. The rules per subnet would still work though.

I feel like I may have to allow a couple CT/VMs to communicate without going through the firewall simply for performance reasons. Has that ever been a concern for you? None of the routing or switching would be hardware accelerated.

Thanks for the wise words. However I have some questions:

If you’re worried about someone malicious having access to your network connection, ssh is going to do a DNS lookup to map the hostname to an IP for the client.

Are you sure that this is true for Tor? .onion addresses never resolve to an IP address, even for the end user client. If I was on an untrusted network, both for the client and the server, the attacker could find out that I was using Tor, but not know literally anything more than that.

And attackers have aimed to exploit things like buffer overflows in IDSes before – this is a real thing.

I would expect an IDS to be an order of magnitude larger attack surface than Wireguard, and significantly less tested. Although I could also say that about SSH, and we had the recent backdoor. However, I think it is a lot more likely that a bug will cause a security method to be ineffective than actively turn it in to a method for exfiltration or remote access though. For example, with the recent SSH backdoor, if those servers had protected SSH behind Wireguard then they would have been safe even if SSH was compromised.