Organization of the infrastructure repository


#1

A year ago the infrastructure repository was organized so roles are included in the molecule directory like so:

├── molecule
│   ├── authorized_keys
│   │   ├── molecule.yml
│   │   ├── playbook.yml
│   │   ├── roles
│   │   │   └── authorized_keys
│   │   │       ├── defaults
│   │   │       │   └── main.yml
│   │   │       ├── files
│   │   │       │   ├── ssh_keys
│   │   │       │   │   ├── dachary.pub
│   │   │       │   │   ├── fpoulain.pub
│   │   │       │   │   ├── pilou.pub
│   │   │       │   │   └── swarthon.pub
│   │   │       │   └── test_keys
│   │   │       │       ├── testkey
│   │   │       │       └── testkey.pub
│   │   │       └── tasks
│   │   │           ├── clouds.yml -> ../../../../../clouds.yml
│   │   │           └── main.yml
│   │   └── tests
│   │       └── test_all.py
│   ├── chat
│   │   ├── molecule.yml
│   │   ├── playbook.yml
│   │   ├── roles
│   │   │   └── mattermost
│   │   │       ├── defaults
│   │   │       │   └── main.yml
│   │   │       ├── handlers
│   │   │       │   └── main.yml
│   │   │       ├── tasks
│   │   │       │   ├── main.yml
│   │   │       │   └── mattermost.yml
│   │   │       └── templates
│   │   │           └── docker-compose-infrastructure.yml
│   │   └── tests
│   │       ├── test_icinga.py
│   │       └── test_mattermost.py

@pilou reminded me last week that it is contrary to the usual organization where the roles directory includes all roles and is at the same level as the molecule directory.

While this is good when implementing molecule tests for a single role, I think it is not suitable for the repository containing playbooks and roles dedicated to the Enough community infrastructure.

IMHO the pros of the current conventions are that each molecule directory:

  • is dedicated to playbooks and roles that contribute something similar. They are a context in which one can work without worrying about the other roles.
  • it is trivial to figure out what depends on what by looking at the ANSIBLE_ROLES_PATH variable in the molecule.yml file. Figuring dependencies would otherwise be complicated because ansible does not provide a way to explore the dependency graph. I’ve seen example of a roles directory with over fifty roles and figuring if a given role was still in use was difficult. As a number of roles were probably dead code but not trimmed. I believe this is a common pattern that makes maintenance difficult in the long run.
  • provides molecule/something/playbook.yml which is dedicated to testing every role under molecule/something and molecule/something/something-playbook.yml (imported in molecule/something/playbook.yml) which is included in the production playbook. The production playbook (enough-community-playbook.yml`) is cleanly divided and whenever something breaks in production it provides a self contained debug environment that is easier to debug and test than the whole playbook.

The cons are:

  • the organization looks weird to seasoned ansible developers
  • if a role dedicated to Enough matures and can be published on its own, the organization of the molecule repository that contains it will have to be restructured. It should just be a matter of copying the directory in which the role lives and pushing it to a separate repository.

This last point is what bugs me right now. I wonder if we could keep the benefits of the current organization but with something that people look familiar. Now that molecule has become mainstream, it would be beneficial.

What do you think?


#2

@pilou suggested to just have gitlab/molecule, postfix/molecule instead of the current molecule/gitlab, molecule/postfix etc. It’s not a big change but has the benefit of including the molecule directory within a higher context and it would not feel as weird.