Skip to main content

Configuration

The new Runops Agent removes the dependency on Github. It is faster, and can be 100% self-hosted (if desired). We packaged the Agent in a Docker image that runs anywhere. The Agent retrieves secrets from a Vault or Secret Manager of your choice, hosted by you.

Secrets#

Targets no longer store secrets. You can store Secrets in one of the Secret Manager tools:

  • AWS secrets manager
  • Hashcorp Vault
  • Google (GCP) secrets manager (future releases)

Each target requires a new secret record, named with the same name of the target. An example on how to create a secret to a mysql target named mysql-target-prod:

AWS secret-manager:#

# this is a working demo database
aws secretsmanager create-secret \
--name mysql-target-prod \
--description "mysql credentials for production" \
--secret-string '{"MYSQL_HOST":"34.69.12.143","MYSQL_USER":"demo-user","MYSQL_PASS":"GtK23d<ejkLy0ST2","MYSQL_DB":"demo"}'

It is highly recommended that a new IAM role and user are created to access this new resource. The secret-string param is a regular JSON.

Hashcorp vault secret manager#

For hashcorp, different secrets are required.

export VAULT_ADDR=http://127.0.0.1:8200
export VAULT_AUTH_METHOD=kubernetes-account-service
export VAULT_ROLE={{vault role}}
# optionally, if k8s auth is registered at a different address, then
export VAULT_AUTH_PATH=/v1/auth/kubernetes/login

The VAULT_AUTH_METHOD variable is how the runner is authenticating into Vault. Currently, only kubernetes-account-service is available, but other login methods will be available in the future.

Both k8s (account-service) and vault needs to be configured for this method to work. Please reach out for more details at: https://www.vaultproject.io/docs/auth/kubernetes

Deploy on kubernetes#

WARN: Pasting the below snippets in the terminal will automatically create the Kubernetes resources. Make sure you update the variables and set the context and namespace of your choice before running them.

Create k8s secrets:#

Replace with your values

# required
export TAGS={{"Any environment tag (i.e. dev, prod)"}}
export TOKEN={{"Request Runops for one"}}
export API_URL=https://api.runops.io
# if using aws secret-manager
export AWS_ACCESS_KEY_ID={{"limited user access key id"}}
export AWS_SECRET_ACCESS_KEY={{"limited user secret access key"}}
export AWS_REGION={{"aws region where secrets are stored"}}
# if using hashcorp vault
export VAULT_ADDR=http://127.0.0.1:8200
export VAULT_AUTH_METHOD=kubernetes-account-service
export VAULT_ROLE={{vault role}}

Create the Secret

Export the first 3 mandatory envs, and optionally the others related to customer provider

add -w 0 after base64 on Linux

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
name: runner-secrets
type: Opaque
data:
TENANT: $(echo -n ${TENANT} | base64)
TAGS: $(echo -n ${TAGS} | base64)
TOKEN: $(echo -n ${TOKEN} | base64)
API_URL: $(echo -n ${API_URL} | base64)
AWS_ACCESS_KEY_ID: $(echo -n ${AWS_ACCESS_KEY_ID} | base64)
AWS_SECRET_ACCESS_KEY: $(echo -n ${AWS_SECRET_ACCESS_KEY} | base64)
AWS_REGION: $(echo -n ${AWS_REGION} | base64)
VAULT_ADDR: $(echo -n ${VAULT_ADDR} | base64)
VAULT_AUTH_METHOD: $(echo -n ${VAULT_AUTH_METHOD} | base64)
VAULT_ROLE: $(echo -n ${VAULT_ROLE} | base64)
EOF

Create k8s Deployment:#

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
name: runner-deployment
labels:
app: runner
spec:
replicas: 1
selector:
matchLabels:
app: runner
template:
metadata:
labels:
app: runner
spec:
containers:
- name: runner
image: runops/runner:0.0.3
imagePullPolicy: Always
envFrom:
- secretRef:
name: runner-secrets
EOF

Target Configuration#

Create a new target (or update one) to use new runner. Tags should match the configuration for TAGS variable in the image deployment:

New Target#

#create
runops targets create \
--name "mysql-target-prod" \
--message "Mysql production Target" \
--runner_provider "runops" \
--secret_provider "aws" \
--secret_path "my-aws-secret-path" \
--tags "prod" \
--config "{\"PG_HOST\": \"postgres.host.remote\", \"PG_PORT\":5432}" \
--type mysql

secret_provider can be one of the following: [aws, hashcorp] secret_path is the name of the secret at aws (i.e. my-prod-mysql-secret), or the path at hashcorp (i.e. /databases/cred/my-postgres)

Existing Target#

#update
runops targets update \
--name "mysql-target-prod" \
--runner_provider "runops" \
--secret_provider "aws" \
--secret_path "my-aws-secret-path" \
--tags "prod" \
--config "{\"PG_HOST\": \"postgres.host.remote\", \"PG_PORT\":5432}" \
--type mysql

Now, you can create tasks normally for that target.

Supported target/task types#

Currently, this runner supports only:

  • mysql
  • mysql-csv
  • postgres
  • python
  • k8s
  • k8s-exec

Remaining types in future releases.

Github runners#

To continue using current gihub runner, simply omit runner-provider and secret-provider and it will default to github. On the other hand, a set of secrets must be supplied.

Custom markup