Enough CLI proposal


#1

Bonjour,

When developers wants to contribute to Enough, they are expected to have python 2.7* installed and run the steps from the bootstrap file. The environment they already have is likely to interfere in ways that are difficult to predict.

I’ve had good experience embedding the development environment in a docker container and presenting it as a CLI. The developer needs to have docker & bash installed and the onboard steps would be to add one line to the .bashrc

  eval $(docker run --rm enough install)

and then:

  enough molecule test -s bind

That is assuming the OpenStack credentials are in the environment.

Going forward the CLI could also be useful to implement bootstraping of an entirely new Enough instance, using another domain. This phase currently requires many manual steps on OVH & Gandi and could be performed by providing credentials to the CLI which would then use the OVH & Gandi API.

enough bootstrap --ovh-user foo --ovh-password bar --gandi-user frob --gandi-password nitz

In other words the CLI could be a way to perform all operation on enough, a documented central point to call molecule, ansible-playbook etc.

Creating a piece of software is a slippery slope: it becomes a convenient way to express our darkest NIH symdroms :slight_smile: This is the primary reason why I did not propose to have a CLI when creating the first enough playbooks about a year ago, although it has been on my mind all along. I waited until a few patterns emerged that kept being done manually (bootstrap enough and bootstrap the dev environment).

I’m going to implement a CLI and create a merge request to give a concrete example of how it works. It will hopefully help the discussion.

Cheers


Bootstraping Enough and running a mail server
#2

For the record, the implementation started here https://lab.enough.community/main/infrastructure/issues/66


#3

Why are you coupling your CLI with ISPs? Using environment variables would save --commands for stuff that is vendor-free. Environment variables are also very Docker-friendly

OVH_USER=foo OVH_PASSWORD=bar GANDI_USER=frob GANDI_PASSWORD=nitz enough bootstrap --hosting --domain

Or to use Gandi for both hosting and domain:

GANDI_USER=frob GANDI_PASSWORD=nitz enough bootstrap --hosting --domain

(which would of course send a warning for password re-use, thus a better solution might be:)

HOSTING_CREDENTIALS=foo:bar DOMAIN_CREDENTIALS=frob:nitz enough bootstrap --hosting gandi --domain gandi

An advantage of using the last version is that passing the provider to the command would allow alternate ISPs to implement enough setups.

And now you have… Confusion :bike: :flushed:


#4

You are correct, this would be bad design. The example I provided lacks in context and deserves a separate discussion.

Problem

To bootstrap enough, it must be given OpenStack credentials and and the DNS for a domain name must be delegated to a host created with OpenStack. The enough.community infrastructure uses OVH and Gandi and these preliminary steps were done manually.

A CLI could automate all these steps if given credentials to an OpenStack provider and a registrar. The alternative would be document dozens of manual steps and it would be impractical.

User stories

  • As new organization, I want to be able to instantiate my own infrastructure based on enough on domain.net with a minimum of manual steps.

  • As a member of enough.community who disagrees with how it goes, I want to easily fork the entire infrastructure and go in a different direction

  • As an organization hosted on enough.community and happy about it, I want to easily self host on another domain to no longer depend on enough.community members

Implementation notes

OpenStack providers and DNS registrars are not normalized and automating their interactions requires code and documentation that is specific to each of them. The CLI should list all providers:

$ enough bootstrap --help
...
--dns-registrar (gandi|1on1|ovh)
--openstack-provider (ovh|entercloudsuite)
...

And the provider specific options should be prefixed with its name:

$ enough bootstrap --dns-registrar gandi --openstack-provider ovh --gandi-user foo --gandi-password bar --ovh-user frob --ovh-password nitz

which could also be set in a configuration file

---
dns-registrar:
   gandi:
      user: foo
      password: bar
openstack-provider:
   ovh:
      user: frob
      password: nitz

#5

Unfortunately gandi is not among the molecule supported providers and using Gandi for hosting would be more work.

As a first step it would be nice to have a driver for OpenStack provided by OVH (which is not the same unfortunately than OpenStack provided by RackSpace…) and a driver for DNS configuration for Gandi (I don’t think registrars ever agreed on a common API)


#6

Well yes: I’m not sure exactly where you’re going with this bug you got me curious :slight_smile: What I ultimately want to be able to do with the CLI is this:

  • backup the entire infrastructure I’m using and my data (not the data of other users) on my laptop
    enough backup --name mybackup --user foo --password bar https://enough.community
  • instantiate my backup locally enough bootstrap --local mybackup and see my web browser displaying the website with links to all services, as they were at the time of the backup (most likely using docker as a backend instead of OpenStack)
  • move from the enough infrastructure I’m using and my data onto another domain name that I own and control enough migrate --from enough.community --to newdomain.top --dns-registrar gandi ...
  • move from the enough infrastructure I’m using onto my hardware accessible via tor instead of the cloud enough migrate --from enough.community --to asdu23423hhga.onion --docker-cluster swarm --docker-api 1.2.3.4

I suppose you see where I’m going with this?


#7

Maybe some work that can be done in coordination with them? I need to show you a working document and discuss it in person. Let’s do this when you come to Brussels.

It reminds me of an under-estimated functionality of OpenVZ for what I’d call “mobile servers”: you could tell a virtual server to clone itself to a new IP address and switch over from a running instance to the next when it is deployed. A bit like Erlang nodes. I guess that’s what you’re aiming at: enable users to turn ISPs into a commodity, so that they can switch over to a new infrastructure in case of attack (including censorship) on both hosting and DNS.