Operations grimoire/Operations repository
Our configuration as code is stores in the operations (rOPS) repository.
Repository layout
| Path | Description |
|---|---|
| _modules | Custom execution modules |
| _states | Custom states modules |
| _tests | Unit tests for custom modules, scripts, tests for repo |
| hotfixes | Fixes you need to run to solve a problem on the servers |
| pillar | Configuration data structures to use in the states |
| roles | The states to deploy, divided in roles and then in units |
| utils | Helper scripts to maintain the repository |
| map.jinja | As we deploy on several OS and distros, mapping of packages names or directories |
| top.sls | topfile: what we deploy where? |
| PORTS | Documentation of the ports used in the configuration |
| UIDs | Documentation of the users used in the configuration |
| GIDs | Documentation of the groups used in the configuration |
Contributions howto
Repository source
You'll find the Operations repository at https://devcentral.nasqueron.org/source/operations/
If DevCentral isn't available, a mirror of the repository can be found at https://github.com/nasqueron/operations.
Workflow of contributions
We follow the general workflow is described at How to contribute code.
Before committing a change to the main branch, you can test a deployment on a server. In that case, merge in main immediately after the deployment as the repository is the source of truth for the server state.
Log what you do on #nasqueron-ops, so it will be included at https://infra.nasqueron.org/servers-log/
Code conventions
See also: Code conventions.
- Python code: run `black` before committing
- YAML: indent with two spaces, including list bullets
id:
method:
- somekey: value
- otherkey: value
IDE configuration
The states .sls file are Jinja2 templates producing YAML files, or "jinja2+yaml". If this case can't be handled by your IDE, it works best to use YAML syntax highlighting.
If you use PyCharm or IntelliJ IDEA Ultimate, you can configure this scheme:
- Allow YAML files to be Jinja2 templates
- Go to Settings > Languages & Frameworks > Template Languages
- Select Jinja2 as template language
- Add YAML in the list
- Open SLS as YAML
Linting
pre-commit
You can use pre-commit you can install with make to get a hook running Python code to lint the repository before a commit.
Beware if you use arc diff without a commit first: your message will be lost if pre-commit raises an error, always use git commit first.
On FreeBSD, there is an issue to build with clang 16+ the ruamel yaml clib library. You can build the FreeBSD port devel/py-ruamel.yaml.clib and copy the .whl to a stable path, then install it in your virtual environment.
On WindRiver, the .whl is available in /opt/python/py311, so you can:
- python3 -m venv /path/to/your/virtualenv
- source /path/to/your/virtualenv/bin/activate
- pip install /opt/python/py311/ruamel.yaml.clib-0.2.8-cp311-cp311-freebsd_14_0_release_p3_amd64.whl
Refactoring
Rename a lot of files
To move a directory and rename the Source file headers:
git mv roles/webserver-legacy/nginx/files roles/webserver-alkane/nginx/ cd roles/webserver-alkane/nginx/ find . -type f -name '*.conf' | xargs gsed -i s/webserver-legacy/webserver-alkane/g
