You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

222 lines
7.4 KiB

This is the contribution guide to the ``
infrastructure which is based on Ansible and OpenStack. If you're a
seasoned Free Software contributor looking for a quick start, take a
look at the `list of bugs and features
otherwise keep reading.
* Repository and issue tracking:
* Forum:
* Instant messenging:
* License: `AGPLv3 <>`__
* :doc:`Who's who <team>`
Bugs and features list
Each service under the `` domain can be worked on
independently and have their own integration tests. There is no need
to understand how `Weblate` is deployed if you're improving
`Discourse`, for instance.
All contributors are `organized horizontally <>`__
* People with access to an exclusive resource must register themselves
in the :doc:`team directory <team>`
.. _getting_started:
Getting started
* ``git clone``
Running tests
* Install Docker.
The ``tests/`` script builds a docker image suitable for
running the tests, with all the required dependencies, based on a
Debian GNU/Linux buster. The following volumes are bind-mounted:
* ~/.enough
* ~/.ansible
* the root of the `infrastructure` repository
The working directory, in the container, is the root of the
`infrastructure` repository.
Unit and functional testing
There is no need to have OpenStack credentials to run these tests.
Integration testing
Unit and functional tests are welcome, integration tests are
.. note:: Ansible being declarative for the most part, unit tests are
only beneficial to verify loops and conditionals work as
expected. For instance by checking a file is created only if
**--tag something** is provided. An integration test is
necessary to check if the service is actually doing
anything useful. For instance the integration tests for
weblate request that the weblate server sends a mail and
verify it is relayed by the postfix server.
When modifying a role or a playbook in the directory `playbooks/ABC`
one is expected to add a test for the new behavior and verify it runs
* ``tests/ tox -e ABC``
When relevant, integration tests should be created as `icinga`
monitoring checks so they can be run on a regular basis in the
production environment to verify it keeps working.
Obtain an API token
Most integration tests need a publicly available DNS server. The provides a publicly available API to
delegate a domain to the designated DNS server. Members of the `group
enough <>`_
can sign-in, others can `request access
.. image:: api0.png
.. image:: api1.png
The **Token:** value displayed after signing in
must be set to the ``ENOUGH_API_TOKEN`` environment variable.
* ``ENOUGH_API_TOKEN=XXXXXXX tests/ tox -e bind``
Set the OpenStack credentials
Assuming you have your own OVH tenant or one was :ref:`provided to you
<infrastructure>`, the credentials must be set in environment
variables as follows:
* ``source``
* ``tests/ tox -e <service name>``
.. note::
If the command fails, because of a network failure or any other reason,
it is safe to run it again. It is idempotent and will re-use the environment
from the failed test.
The list of service names (i.e. tox test environments) is in the `tox.ini` file. It is possible
to skip some steps to speed up test debugging:
.. code::
$ tox -e bind -- --help playbooks
custom options:
--enough-no-create Do not run the create step
--enough-no-tests Do not run the tests step
--enough-no-destroy Do not run the destroy step
$ tests/ tox -e openvpn -- --enough-no-destroy playbooks/openvpn/tests
The domain name used for testing is in
where `bind` must be replaced by the name of the service. It is handy
for debugging (i.e. browsing the web interface of a service, ssh to a
machine that failed to run properly, etc.)
If a test fails, it will **not** destroy the resources provisioned
for the test, they must be destroyed explicitly with something like:
* ``tests/ tox -e openvpn -- --enough-no-create --enough-no-tests playbooks/openvpn/tests``
Uprade testing
The ```` script can be used to setup a scenario
based on a previous version of the repository:
$ 1.0.7 icinga
It essentially does the following:
* checkout the ``1.0.7`` tag into ``../infrastructure-versions/1.0.7/infrastructure``
* and run ``tox -e icinga`` from this directory
When ```` completes,
$ tox -e icinga
from the working directory will re-use the hosts created by the
```` run above and upgrade from ``1.0.7``.
Debugging tests
* ``tests/ tox -e py3 -- --log-cli-level=INFO -s -x tests/enough/common/``
Control-C won't work if you're trying to stop the tests, ``docker kill enough-tox`` should be used instead.
There should not be any leftover after a test involving OpenStack
fails, because the fixtures are supposed to thoroughly cleanup. But
bugs are to be expected in a test environment and it may be necessary
to manually remove leftovers, using the ``openstack`` command like so:
* ``tests/ env OS_CLIENT_CONFIG_FILE=~/.enough/dev/inventory/clouds.yml openstack --os-cloud production stack list``
* ``tests/ env OS_CLIENT_CONFIG_FILE=~/.enough/dev/clone-clouds.yml openstack --os-cloud production stack list``
Ansible repository layout
The `ansible repository
<>`_ groups playbooks
and roles in separate directories to reduce the number of files to
consider when working on improving a playbook or a service.
* ``playbooks/authorized_keys``: distribute SSH public keys
* ``playbooks/backup``: daily VMs snapshots
* ``playbooks/bind``: DNS server and client
* ``playbooks/letsencrypt-nginx``: nginx reverse proxy with letsencrypt integration
* ``playbooks/icinga``: resources monitoring
* ``playbooks/infrastructure``: VMs creation and firewalling
* ``playbooks/postfix``: outgoing mail relay for all VMs
* etc.
The other scenarii found in the `playbooks` directory are services such
as `weblate <>`_ or `discourse <>`_.
The toplevel directory contains the `playbook that applies to the production environment
<>`_. It
imports playbooks found in the `playbooks` directory.