Run Prometheus, Grafana, and Alertmanager

This guide runs Prometheus, Grafana, node_exporter, and Alertmanager with Docker Compose. The web interfaces bind to loopback and are reached through an SSH tunnel. node_exporter binds to host port 9100, so the host firewall must block public access to that port.

Prerequisites

• An Ubuntu 26.04 VPS with the Docker guide completed
• A real Alertmanager receiver, such as a private webhook or supported notification service
• Enough storage for the chosen retention period

The pinned versions below were reviewed on 23 September 2026. Check upstream release notes and update deliberately.

Create the project

sudo install -d -m 0755 -o "$USER" -g "$USER" /opt/monitoring/{prometheus/rules,alertmanager}
cd /opt/monitoring

Create .env, choose a long unique password, then run chmod 600 .env:

GRAFANA_ADMIN_PASSWORD=replace-with-a-long-password

Create compose.yaml:

services:
  prometheus:
    image: prom/prometheus:v3.14.0
    restart: unless-stopped
    command:
      - --config.file=/etc/prometheus/prometheus.yml
      - --storage.tsdb.path=/prometheus
      - --storage.tsdb.retention.time=15d
    ports: ["127.0.0.1:9090:9090"]
    extra_hosts:
      - host.docker.internal:host-gateway
    networks: [monitoring]
    volumes:
      - ./prometheus:/etc/prometheus:ro
      - prometheus-data:/prometheus

  alertmanager:
    image: prom/alertmanager:v0.34.1
    restart: unless-stopped
    ports: ["127.0.0.1:9093:9093"]
    networks: [monitoring]
    volumes:
      - ./alertmanager:/etc/alertmanager:ro
      - alertmanager-data:/alertmanager
    secrets:
      - alertmanager_webhook_url

  grafana:
    image: grafana/grafana:13.2.2
    restart: unless-stopped
    environment:
      GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_ADMIN_PASSWORD}
      GF_USERS_ALLOW_SIGN_UP: "false"
    ports: ["127.0.0.1:3000:3000"]
    networks: [monitoring]
    volumes:
      - grafana-data:/var/lib/grafana

  node-exporter:
    image: prom/node-exporter:v1.12.1
    restart: unless-stopped
    command: ["--path.rootfs=/host"]
    network_mode: host
    pid: host
    volumes:
      - /:/host:ro,rslave

networks:
  monitoring:

volumes:
  prometheus-data:
  alertmanager-data:
  grafana-data:

secrets:
  alertmanager_webhook_url:
    file: ./alertmanager/webhook-url

Configure Prometheus

Create prometheus/prometheus.yml:

global:
  scrape_interval: 30s
  evaluation_interval: 30s

alerting:
  alertmanagers:
    - static_configs:
        - targets: [alertmanager:9093]

rule_files:
  - /etc/prometheus/rules/*.yml

scrape_configs:
  - job_name: prometheus
    static_configs:
      - targets: [localhost:9090]
  - job_name: node
    static_configs:
      - targets: [host.docker.internal:9100]

Create prometheus/rules/node.yml:

groups:
  - name: node
    rules:
      - alert: NodeExporterDown
        expr: up{job="node"} == 0
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: node_exporter is unavailable

Configure Alertmanager

Create alertmanager/webhook-url with only the private receiver URL. The Alertmanager image runs as UID 65534 (nobody), so give that UID ownership while keeping the file private:

nano alertmanager/webhook-url
chmod 600 .env alertmanager/webhook-url
sudo chown 65534:65534 alertmanager/webhook-url

Docker Compose bind-mounts file-backed secrets, so the container must be able to read the host file. To change the URL later, edit it with sudoedit and restore UID 65534 ownership and mode 600. Create alertmanager/alertmanager.yml with url_file so the credential does not appear in the readable configuration:

route:
  receiver: operations
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 4h

receivers:
  - name: operations
    webhook_configs:
      - url_file: /run/secrets/alertmanager_webhook_url
        send_resolved: true

Prometheus and Alertmanager run as unprivileged users. Their bind-mounted directories must therefore be traversable and the YAML files readable. The directories created above use mode 755. Keep the configuration files at mode 644 because they contain no credentials:

chmod 644 compose.yaml prometheus/prometheus.yml prometheus/rules/node.yml alertmanager/alertmanager.yml

Start and validate

sudo docker compose config --quiet
sudo docker compose up -d
sudo docker compose ps
sudo docker compose exec prometheus promtool check config /etc/prometheus/prometheus.yml
sudo docker compose exec prometheus promtool check rules /etc/prometheus/rules/node.yml
sudo docker compose exec alertmanager amtool check-config /etc/alertmanager/alertmanager.yml
sudo docker compose exec alertmanager sh -c 'test -r /run/secrets/alertmanager_webhook_url'
curl --fail http://127.0.0.1:9090/-/ready
curl --fail 'http://127.0.0.1:9090/api/v1/query?query=up%7Bjob%3D%22node%22%7D'

The final query should return a value of 1. If it does not, inspect sudo docker compose logs node-exporter prometheus.

node_exporter uses the host network and PID namespace, as recommended upstream for host monitoring. It listens on all host interfaces at port 9100 by default. Prometheus reaches it through Docker’s host-gateway mapping. Check sudo ufw status numbered for any rule allowing public access to 9100, and test the port from another machine. Keep it closed to the internet.

Open Grafana safely

From your computer, create an SSH tunnel:

ssh -L 3000:127.0.0.1:3000 USER@SERVER_IP

Open http://127.0.0.1:3000, sign in as admin with the password from .env, and add http://prometheus:9090 as a Prometheus data source. Build or import a dashboard for CPU, memory, filesystem, and network metrics.

Test the alert path

Temporarily stop node_exporter:

sudo docker compose stop node-exporter

After two evaluation minutes, confirm NodeExporterDown is firing in Prometheus, appears in Alertmanager, and reaches the configured receiver. Start node_exporter again and confirm the alert resolves:

sudo docker compose start node-exporter

Operate the stack

• Alert on monitoring-disk usage before it fills.
• Back up the Compose files, rule files, receiver configuration, and Grafana volume.
• Test restoring Grafana configuration and dashboards.
• Review new image versions and release notes before updating.
• Keep ports 3000, 9090, 9093, and 9100 closed publicly unless you add an authenticated reverse proxy and a deliberate access policy.

See self-hosted monitoring on a VPS for plan sizing and operating tradeoffs.

Sources: Prometheus configuration, Alertmanager configuration, node_exporter guide, and Grafana Prometheus data source.