Hybrid Enough: OpenStack & libvirt


Since Enough now supports deployments of hosts with libvirt instead of OpenStack, one can imagine deploying a single host in the cloud and the rest of the hosts on physical machines. The host in the cloud runs the DNS and a reverse proxy behind a public IP for the example.com domain and is deployed with:

$ enough --domain example.com service create website

The services on physical machines (for instance a Nextcloud) are deployed with:

$ enough --domain lan.example.com service create --driver libvirt cloud

A VPN is added to example.com so that lan.example.com can become a client, otherwise there is no way for the reverse proxy running on example.com to reach the hosts in lan.example.com because they have no public IP.

The DNS at example.com delegates the lan.example.com zone to the DNS that runs in lan.example.com. It serves two purposes:

  • a user that connects to the VPN can conveniently ssh debian@cloud.lan.example.com despite the fact that it has a private IP on the physical machine
  • the postfix server running at postfix.lan.example.com can identify as lan.example.com which can be verified to be a real domain name instead of a fake one that would be denied by the receiving SMTP server

This is not too complicated but the implementation requires a significant number of manual steps that should be documented:

  • :white_check_mark: Creating the libvirt host and populating the ~/.enough/lan.example.com directory for it
  • :white_check_mark: Retrieving the VPN client credentials for lan.example.com
  • :white_check_mark: Writing a playbook to install the VPN client credentials on bind.lan.example.com, publish the route, configure nat with nft (in a way that is similar to what the VPN server does)
  • :white_check_mark: Writing a playbook to delegate lan.example.com from example.com
  • :white_check_mark: Writing a playbook to reverse proxy cloud.example.com to cloud.lan.example.com so that it is visible from the net
  • :white_check_mark: Add the admin ssh public key so it is installed by the authorized_keys playbook and they can conveniently ssh debian@bind.lan.example.com as well as ssh debian@bind.example.com
  • :white_check_mark: Set the public facing FQDN of cloud.lan.example.com to be cloud.example.com instead via an ansible variable
  • :white_check_mark: Set the postfix.lan.example.com mailname and HELO name to lan.example.com via an ansible variable
  • :white_check_mark: Install the root CA of lan.example.com in website.example.com so that the reverse proxy can use https://cloud.lan.example.com which does not have a Let’s Encrypt certificate
  • :white_check_mark: Setup the backups so that lan.example.com hosts are uploaded to OpenStack weekly

Special thanks to @pilou for the domain name naming idea: using lan.example.com instead of example.lan.

1 Like

After much hesitations the libvirt install command and the associated documentation merge request is ready and was successfully run against the new libvirt hypervisor installed earlier this week.

The hypervisor is running bullseye, which is a little early for Enough because it’s not released yet. And Enough has no support for anything but buster. However the release is a few months away and hypervisors should really be running bullseye if installed at this point in time to avoid a major upgrade real soon.

Added the libvirt install --vpn option, with a user guide, to connect an OpenStack based Enough to a libvirt based Enough.

It also adds a hardcoded route to the default libvirt CIDR in the VPN server and a CCD file that claims the lan client routes this CIDR. It will do nothing if there is no VPN (i.e. the IP will not be routed anywhere). Not hardcoding this route would require some kind of communication between the OpenStack based Enough and the libvirt based Enough even before they are connected with a VPN.

  • cons: the documentation and playbook to connect a libvirt Enough to an OpenStack enough only work for a single hypervisor
  • pro: simplicity

Added the openvpnclient playbook, with a user guide, to connect bind-host from libvirt to the OpenStack VPN.

Next step will be about setting up the reverse proxy in OpenStack for the the web services located on libvirt. It is straightforward and mostly documentation about how to configure nginx.

Added to the libvirt-hypervisor playbook to setup and document downloads of OpenStack backup for safekeeping.

1 Like

And with the addition of the last reverse proxy details there is enough documentation and code to bind OpenStack & libvirt together now :tada: The first service to benefit from this is … https://openedx.enough.community which permanently moved from OpenStack to libvirt today :tada: