Dev zone/Vault

This page explains how to query vault from an application.

For infrastructure point of view, see Operations grimoire/Vault and Operations grimoire/Eglide/Vault.

Guides

Python with hvac

In Python, the hvac library is recommended.

Step 1 - Connect to Vault

Connection is straightforward and works out of the box even without parameters:

VAULT_CA_CERTIFICATE = "/usr/local/share/certs/nasqueron-vault-ca.crt"

def connect_to_vault():
    return hvac.Client(
        verify=VAULT_CA_CERTIFICATE,
    )

TLS connection needs to be validated with our root certificate. The exception is when we write operations code allowing us to renew the Vault certificates itself, where we use verify=False.

For a user script, that's it, the Vault address will be read from the VAULT_ADDR environment variable and a token should be configured by the user itself.

Comprehensive checks for VAULT_ADDR and tokens, with explicit error messages can be found in rOPS: utils/vault/wordpress-provision-secrets.py.

For an application, you can set URL from configuration (pass it with url=) and probably need to login too with credentials. AppRole is encouraged in that case.

# Those values are expected to be read from a configuration file
vault_certificate_path = "/usr/local/share/certs/nasqueron-vault-ca.crt"
vault_url = "https://172.27.27.7:8200"
vault_role_id = "00000000-0000-0000-0000-000000000000"
vault_secret_id = "00000000-0000-0000-0000-000000000000"

vault_client = hvac.Client(                                                                                                                                                                                          
    url=vault_url,
    verify=vault_certificate_path,
)                                                                                                                                                                                                                    

vault_client.auth.approle.login(                                                                                                                                                                                     
    role_id=vault_role_id,  
    secret_id=vault_secret_id,
)

Step 2 - Query a kv secret

Most operations with Vault uses the kv2 engine to get secrets as a collection of keys and values.

Secrets are normally located in apps/ for your application or ops/ for infrastructure credentials.

With hvac, best practice is to query them separately if you target only Nasqueron internal infrastructure:

def read_secret(vault_client, mount_point, prefix, key):                                                                                                                                                                           
    secret = vault_client.secrets.kv.read_secret_version(
        mount_point=mount_point,
        path=prefix + "/" + key,
    )                                                                                                                                                                                                                
    return secret["data"]["data"]


def read_ops_secret(vault_client, key):                                                                                                                                                                                            
    return read_secret(vault_client, "ops", "secrets", key)


def read_app_secret(vault_client, key):
    return read_secret(vault_client, "apps", "yourappname", key)

For applications to distribute, best is to write a full path starting with apps/ and ops/ secrets in a configuration. You can use the first part of the path as mount point:

def read_secret(vault_client, secret_path):
    tokens = secret_path.split("/")
    secret_mount = tokens[0]
    secret_path = "/".join(tokens[1:])

    secret = vault_client.secrets.kv.read_secret_version(
        mount_point=secret_mount,
        path=secret_path,
    )

    return secret["data"]["data"]

Best practices

Where to store credentials

Are you setting a credential only for your app as human? Store it to apps/your-app/somepath.

Is the credential to be deployed by Salt? Store it in ops/secrets/nasqueron/your-app/somepath. If you use AppRole, the secret id and role id to provision automatically to your configuration file will be stored there.

To avoid to manage two paths with the same credential and facilitate rotation, it's encouraged to reuse credentials from a database directly from an ops/secrets/dbserver/ path.

Troubleshoot

Access denied

The access can be denied when:

• you're not authenticated
• the resource doesn't exist, or the URL to it has an error
• the policy attached to your Vault credentials don't allow you to query the secret

Check if the URL is correct, try to identify by reading other troubleshoot sections in what case you are, and ask operations assistance to check if the policy is correct and has correctly been deployed.

Access denied when query a full path, without mounting point

Symptom: Permission denied on an URL starting by https://172.27.27.7:8200/v1/secret/data/

Cause: Your library use the default Vault secret engine mount point "secret" instead of "ops" or "apps"

Solution: separate explicitly "ops" or "apps" mount point from the rest of the secret path

____

Vault have secrets engines, each configured to a specific mounting point. We use two points for secrets, ops/, for secrets deployed by Salt, and apps/, for applications.

If you don't specify the mounting point, client libraries like hvac will try to use the default secret/ mount point:

   Traceback (most recent call last):
     [...]
     File "[...]/lib/python3.11/site-packages/hvac/utils.py", line 41, in raise_for_error                                                                                                  
       raise exceptions.VaultError.from_status(                                                                                                                                                                         
   hvac.exceptions.Forbidden: 1 error occurred:                                                              
       * permission denied                                                                                                                                                                                          
                                                                                                                                                                                                                  
   , on get https://172.27.27.7:8200/v1/secret/data/ops/secrets/dbserver/[...]

That URL has been built from the "secret" default mount point and your ops/secrets/dbserver/... path. Instead, URL to query should have been here /v1/ops/data/secrets/dbserver/...

Example of code above for hvac solve that issue.