Skip to main content

Configuration

The Runops agent is a small, reliable, and cross-platform task runner that makes it easy to run tasks on your own infrastructure. Its main responsibilities are polling Runops for work, running tasks, and reporting back the status code and output log of the task. Here is what it looks like:

drawing

The picture looks complicated but you can get this done in 10-15 minutes going slow. We will break down each piece to make it easier. We need to setup 3 components inside your Cloud to setup the agent:

  1. Internal System
  2. Secrets Manager
  3. The agent

Internal System#

drawing

This is the easiest part. It consists of creating a credential in your internal system to be used by Runops. The procedure varies, it could be database, a Kubernetes cluster, or a AWS account. We'll use a Mysql database as the example for this guide, so I went ahead and created a user in a demo database so you can follow along, here it is:

{
"MYSQL_DB": "demo",
"MYSQL_PASS": "GtK23d<ejkLy0ST2",
"MYSQL_USER": "demo-user",
"MYSQL_HOST": "34.69.12.143"
}
note

Yes, this is the password to a working database. But don't worry, it's read-only and there is nothing sensitive in there.

Secrets Manager#

drawing

The Runops agent pulls credentials from the Secrets Manager of your choice, all within your infrastructure. Secrets never leave your infrastructure.

Each Target need a new secret record. In this guide we will use a Mysql database in the examples, but you can update the values to match the integration you need.

The easiest way to configure Secrets is to add them as Environment Variables to the Agent. Start with this one and layer the Secrets Manager integrations explained in the other tabs later.

All you need to do here is to add the contents of the config to an Environment Variable in the Agent:

export MY_TARGET=$(cat <<EOF
{
"MYSQL_DB": "demo",
"MYSQL_PASS": "GtK23d<ejkLy0ST2",
"MYSQL_USER": "demo-user",
"MYSQL_HOST": "34.69.12.143"
}
EOF
)

Agent Deployment#

drawing
note

Finish the Secrets Manager setup before proceeding.

Docker#

#Run the agent on Linux servers using Docker.
docker run \
--env TOKEN \
--env MY_TARGET \
runops/agent:latest

Kubernetes#

info

Pasting the snippets below in the terminal will automatically create the Kubernetes resources. Make sure you set the Kubernetes context and namespace of your choice before running them.

1. Create the Kubernetes Secret:#

# add `-w 0` after `base64` on Linux
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
name: runner-secrets
type: Opaque
data:
TOKEN: $(echo -n ${TOKEN} | base64)
MY_MY_TARGET: $(echo -n ${MY_TARGET} | base64)
EOF

2. Create the Kubernetes Deployment:#

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
name: runner-deployment
labels:
app: runner
spec:
replicas: 1
selector:
matchLabels:
app: runner
template:
metadata:
labels:
app: runner
spec:
containers:
- name: runner
image: runops/agent:latest
imagePullPolicy: Always
envFrom:
- secretRef:
name: runner-secrets
EOF

Create the Target#

Lastly we need to link the agent to Runops API.

#create
runops targets create \
--name 'demo-db' \
--type 'mysql' \
--secret_path 'MY_TARGET'

Congrats! Now the Tasks of this Target will use the new agent! We can run a SQL Task to try it out:

runops tasks create -t mysql-target-prod -s 'select 1'

Running Multiple Agents#

This is done via TAGS.

An agent will only run tasks from targets whose tags field match the TAGS environment variable of the agent.

If an agent is deployed with an environment variable like TAGS=prod, then all targets whose field tags match the value prod are executed by that agent. Multiple targets can have the same tags.

However, an agent MUST have a unique TAGS across the organisation. This means that only one agent can be deployed with TAGS=prod.

If a second agent tries to start with the same tags, the api will return an error and agent will exit.

If you already have an agent deployed with TAGS=prod, then another agent can be deployed with a different TAGS, for instance, TAGS=dev. Then, only targets whose tags field match the value dev will be executed by that agent.

  • TAGS can be any value that makes sense to the organisation.
  • One agent can be deployed without TAGS, and it will run only tasks from targets whose tags field is null.

Using Existing Secrets#

It's possible to map custom keys from your secret manager:

runops targets create \
--name 'demo-db' \
--type 'mysql'
--secret_provider 'aws' \
--runner_provider 'runops' \
--secret_path 'mysql-target-prod' \
--secret_mapping '{"MYSQL_HOST": "MYSQL_HOSTNAME"}'

When executing a task with this target, it will add the value of the secret key MYSQL_HOSTNAME to MYSQL_HOST, this allows a smoothly integration with existent keys in case of existent key in the secret manager.

⚠️ In case of malformed JSON configuration it will hang the execution of a task.

Enhanced Agent Security#

The agent is a system that has higher privileges in a private infrastructure, the created tasks are processed and executed through an authentication that maps an api key of an organization, however there's no way to guarantee that task is being executed by an authenticated and authorized user session. This requirement may be important to organizations in which requires this sort of enforcement.

The JWT tokens are kept in memory only during the execution of a task, they are never persisted.

Main Benefits#

  • Prevents the execution of tasks without authenticated the user session token
  • Enforced with the use of external auth providers (Okta, Auth0, ORY Hydra, etc)

Limitations#

  • Slack Tasks / REPL are disabled when this option is set
  • Automation of rotating keys is not yet supported

⚠️ Toggling between this feature (JWK_URL=) could be harmful of stale tasks being executed, perform this operation with careful.

Configuration#

Set the enviroment variable JWK_URL containing the public keys of your auth provider before starting the agent, the supported algorithms are RS256 and ECDSA256. To enforce authentication using our auth provider:

  • JWK_URL=https://runops.us.auth0.com/.well-known/jwks.json

Integrations#

The latest agent supports the integrations:

  • bash
  • hashicorp-vault
  • k8s
  • k8s-exec
  • k8s-apply
  • mongodb
  • mysql
  • mysql-csv
  • postgres
  • python
  • node
  • rails
  • rails-console
  • rails-console-ecs
  • sql-server

You can find the legacy Agent docs here.