DocsGuides
Monitoring your servers
What Flatlined collects from every server, how to collect your app's own Prometheus metrics, and how to read it all in Grafana.
What every server gets
Every server is watched from the moment it starts. You don't install or configure anything:
- CPU, memory and network traffic, on the server's Metrics tab.
- Logs: everything the container writes to stdout and stderr, on its Logs tab. Files it writes inside the container aren't collected, so log to stdout.
- Health: crash loops, restarts, out-of-memory kills and failing health checks, on the server's overview. Whoever started a change gets an email if it fails, and owners and admins get one when a server stays unhealthy for 15 minutes.
If that tells you what you need, you're done. Read on to collect numbers your app measures itself, such as requests served, queue length or players online.
Your app's own metrics
Flatlined can collect metrics your app publishes in the Prometheus format, the plain-text format most /metrics endpoints serve. Flatlined fetches them from your server on a schedule, so your app doesn't send anything.
1. Check the app publishes them
Many images already do, or will with a setting: Grafana, Traefik, Caddy, MinIO, Spring Boot with Micrometer, and most Go services. For your own code, add the Prometheus client library for your language and serve its endpoint.
Try it on your own machine before you deploy:
docker run --rm -d -p 9100:9100 --name check your-imagecurl -s localhost:9100/metrics | headcurl -s localhost:9100/metrics | grep -vc '^#'docker rm -f checkThe last number is how many samples the app publishes. Keep it under 5,000 (see Limits).
The app has to listen on all interfaces (0.0.0.0), not only localhost, because Flatlined fetches from outside the container. The metrics port doesn't need to be published: it's opened only to Flatlined's collector.
2. Turn collection on
Any of these works:
-
The dashboard: on the server's Config tab, set Metrics to Collect, enter the port and path, and save.
-
flatlined.yaml: add
metricsto the service'srun. Onlyportis required.flatlined.yaml run: port: 8080 metrics: port: 9100 path: /metrics # the default intervalSeconds: 60 # 30 or 60, the defaultEach deploy sets what the file says, so leaving
metricsout of the file turns collection off. -
An agent over MCP:
update_server_configwithmetrics.
Saving restarts the server if it's running.
3. See them
A minute or two later, the server's Metrics tab shows an App Metrics card. Pick a metric to chart its past hour. Counters, whose names end in _total, _count, _sum or _bucket, are shown per second.
The top of the card says how the last collection went: when it ran and how many samples it found, or why it failed. Agents can read the same with the MCP tool get_metrics_collection.
For dashboards, longer ranges, comparing servers or alerts, use Grafana.
Apps that don't publish metrics
Postgres, Redis, nginx and most game servers don't publish Prometheus metrics. The usual answer is an exporter: a small program that reads the app's statistics and publishes them in the Prometheus format. A Flatlined server runs one container, so the exporter runs as a server of its own in the same project and reaches the app by its name.
Here's Postgres with postgres_exporter. First add a project secret named POSTGRES_PASSWORD (the project's settings, under Secrets), then deploy:
version: 1services: db: image: postgres:18 run: port: 5432 secrets: [POSTGRES_PASSWORD] volumes: - path: /var/lib/postgresql size: 10 db-metrics: image: prometheuscommunity/postgres-exporter:v0.21.1 run: port: 9187 vcpu: 512 env: DATA_SOURCE_URI: db:5432/postgres?sslmode=disable DATA_SOURCE_USER: postgres secrets: - env: DATA_SOURCE_PASS from: POSTGRES_PASSWORD metrics: port: 9187The exporter's series are labeled with the exporter's server, db-metrics, not with db, so give it a name that says what it watches.
Other apps work the same way with their exporter. For Redis, run oliver006/redis_exporter on port 9121 with REDIS_ADDR: redis://<your redis service>:6379. Prometheus keeps a list of exporters for many more.
Apps that push metrics
Some programs push their metrics instead of serving them, such as batch jobs that use a Pushgateway. Run the Pushgateway as a server and collect from it:
pushgateway: image: prom/pushgateway:v1.11.5 run: port: 9091 vcpu: 512 metrics: port: 9091Your other servers push to http://pushgateway:9091. Flatlined doesn't receive OpenTelemetry or StatsD yet. For those, run a collector the same way, such as the OpenTelemetry Collector with its Prometheus exporter, and collect from it.
Reading them in Grafana
There are two ways:
- One click: open Grafana in your team's sidebar and create it there. It runs as an ordinary server, in the project you pick, at the smallest size, billed like any other, with your team's metrics and logs connected and dashboards for your servers. Its Metrics Drilldown browses everything collected, your apps' metrics included.
- Your own Grafana: the same Grafana page shows the addresses, and how to add them as Prometheus and Loki data sources with a Read only token.
Your apps' metrics are under job="user-workloads". Every series is labeled with server_id, the server it came from, and with namespace, pod and instance, where it ran. To show servers by name, join flatlined_server_info, which has server and project:
sum by (server) ( rate(http_requests_total{job="user-workloads"}[5m]) * on (server_id) group_left (server) flatlined_server_info)Metrics can be queried up to 31 days back, and logs 7 days. Only your team's members can read them; a project's guests can't.
Limits
- One endpoint per server: one port and one path.
- Collected every 30 or 60 seconds. Stopped servers aren't collected.
- 5,000 samples per collection. Over that, the whole collection is dropped, not just the extra.
- 10,000 new series a day. Past that, new series are dropped until the next day. Series already being collected carry on.
- Flatlined's labels win. If your app publishes its own
server_id,namespace,pod,joborinstancelabel, it's kept with anexported_prefix, asexported_job. - Some names are reserved. Metrics whose names start with
flatlined_,container_,machine_,kube_,kubelet_,argocd_,loki_,longhorn_,scrape_orvm_, andup, are dropped. If you run your own Loki or VictoriaMetrics, its own metrics won't be collected.
When nothing shows up
Look at the top of the App Metrics card first. It shows what went wrong at the last collection, with the error in full under Details:
- The server hasn't been found yet. Wait a minute after saving. Only running servers are collected.
- Nothing answered on the metrics port. Check the port, and that the app listens on
0.0.0.0. An app bound tolocalhostanswers on your machine but not to Flatlined. - The metrics path answered with an error. Check the path. Run the check in step 1 against the same image and port.
- Over the limit of 5,000 samples. Publish fewer series: drop labels with many values, such as user IDs or full URLs.
- Took more than 10 seconds to answer. Make the endpoint answer from memory, not by doing work on each request.
- Dropped because the name is reserved. Rename the metrics. See Limits.
- More than 10,000 new series today. Labels whose values keep changing make new series. Keep label values to a fixed set.
If the card says the last collection worked but a metric is missing, check that its name isn't reserved and that the app publishes it on every request, not only after something happens.
Still stuck? Contact support.