监控维度
主机维度
主机 CPU
主机内存
主机本地镜像情况
主机上容器运行情况
镜像维度
镜像的基本信息
镜像与容器的基本信息
镜像构建的历史信息 (层级依赖信息)
容器维度
容器基本信息
容器的运行状态信息
容器用量信息
docker 监控命令
- docker ps
- docker images
- docker stats
- # 注意: docker stats 只有选择 libcontainer 作为执行驱动时才可以使用
- # docker stats 时常会有一些限制, 使用 stats API 会显示更多的信息
- echo -e "GET /containers/tools/stats HTTP/1.0\r\n" | nc -U /var/run/docker.sock
- docker inspac
- docker top
- docker port
- cAdvisor
google cAdvisor https://github.com/google/cadvisor 的一些特性:
拥有远程管理的丰富 API 支持 API docs
有 web ui 管理界面
是 google 除支持 k8s 外的另一个项目
监控信息输出到 influxdb 数据库进行存储和读取, 支持很多其他的插件
支持将容器的统计信息一 prometheus 标准指标形式输出并存储在 / metrics HTTP 服务端点
- sudo docker run --volume=/:/rootfs:ro --volume=/var/run:/var/run:rw --volume=/sys:/sys:ro --volume=/var/lib/docker/:/var/lib/docker:ro --volume=/dev/disk/:/dev/disk:ro --publish=8080:8080 --detach=true --name=cadvisor google/cadvisor:latest
- DataDog
SoundCloud 的 Promtheus
Promtheus 是一个开源服务监控系统与时间序列数据库, 结合使用, exporter 是基于 prometheus 开放的 Http 接口二次开发用来抓取应用程序的指标数据
- Global setting
- network configure
- docker network create --driver bridge --subnet 10.0.0.0/24 --gateway 10.0.0.1 monitor
- Grafana configure
- Grafana Template https://grafana.com/dashboards
- # pull grafana image
- docker pull grafana/grafana
grafana 目录
配置文件 / etc/grafana/
sqlite3 database file /var/lib/grafana
grafana 在 docker 中的环境变量
GF_SERVER_ROOT_URL=http://grafana.server.name
指定 grafana 的访问路径
GF_SECURITY_ADMIN_PASSWORD=secret
指定 grafana 的登录密码
GF_INSTALL_PLUGINS=grafana-clock-panel,grafana-simple-JSON-datasource
指定安装插件的变量
构建持久性存储
docker run -d -v /var/lib/grafana --name grafana-storage busybox:latest
启动容器
- # 启动 Grafana 容器
- docker run -d -p 3000:3000 --name grafana --volumes-from grafana-storage -e "GF_INSTALL_PLUGINS=grafana-clock-panel,grafana-simple-json-datasource" -e "GF_SERVER_ROOT_URL=http://10.0.0.10:3000" -e "GF_SECURITY_ADMIN_PASSWORD=marion" --network monitor --ip 10.0.0.10 --restart always grafana/grafana
- # 查看配置文件以及数据目录的挂载位置
- docker inspect grafana
- cAdvisor
- sudo docker run --volume=/:/rootfs:ro --volume=/var/run:/var/run:rw --volume=/sys:/sys:ro --volume=/var/lib/docker/:/var/lib/docker:ro --volume=/dev/disk/:/dev/disk:ro --detach=true --name=cadvisor --network monitor --ip 10.0.0.11 google/cadvisor:latest
- Deploy by docker-compose YAML file
- docker-compose.YAML
- prometheus:
- image: prom/prometheus:latest
- container_name: monitoring_prometheus
- restart: unless-stopped
- volumes:
- - ./data/prometheus/config:/etc/prometheus/
- - ./data/prometheus/data:/prometheus
- command:
- - '-config.file=/etc/prometheus/prometheus.yml'
- - '-storage.local.path=/prometheus'
- - '-alertmanager.url=http://alertmanager:9093'
- expose:
- - 9090
- ports:
- - 9090:9090
- links:
- - cadvisor:cadvisor
- - node-exporter:node-exporter
- node-exporter:
- image: prom/node-exporter:latest
- container_name: monitoring_node_exporter
- restart: unless-stopped
- expose:
- - 9100
- cadvisor:,
- image: google/cadvisor:latest
- container_name: monitoring_cadvisor
- restart: unless-stopped
- volumes:
- - /:/rootfs:ro
- - /var/run:/var/run:rw
- - /sys:/sys:ro
- - /var/lib/docker/:/var/lib/docker:ro
- expose:
- - 8080
- grafana:
- image: grafana/grafana:latest
- container_name: monitoring_grafana
- restart: unless-stopped
- links:
- - prometheus:prometheus
- volumes:
- - ./data/grafana:/var/lib/grafana
- environment:
- - GF_SECURITY_ADMIN_PASSWORD=MYPASSWORT
- - GF_USERS_ALLOW_SIGN_UP=false
- - GF_SERVER_DOMAIN=myrul.com
- - GF_SMTP_ENABLED=true
- - GF_SMTP_HOST=smtp.gmail.com:587
- - GF_SMTP_USER=myadrress@gmail.com
- - GF_SMTP_PASSWORD=mypassword
- - GF_SMTP_FROM_ADDRESS=myaddress@gmail.com
prometheus.YAML
- # my global config
- global:
- scrape_interval: 120s # By default, scrape targets every 15 seconds.
- evaluation_interval: 120s # By default, scrape targets every 15 seconds.
- # scrape_timeout is set to the global default (10s).
- # Attach these labels to any time series or alerts when communicating with
- # external systems (federation, remote storage, Alertmanager).
- external_labels:
- monitor: 'my-project'
- # Load and evaluate rules in this file every 'evaluation_interval' seconds.
- rule_files:
- # - "alert.rules"
- # - "first.rules"
- # - "second.rules"
- # A scrape configuration containing exactly one endpoint to scrape:
- # Here it's Prometheus itself.
- scrape_configs:
- # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
- - job_name: 'prometheus'
- # Override the global default and scrape targets from this job every 5 seconds.
- scrape_interval: 120s
- # metrics_path defaults to '/metrics'
- # scheme defaults to 'http'.
- static_configs:
- - targets: ['localhost:9090','cadvisor:8080','node-exporter:9100', 'nginx-exporter:9113']
- command
- docker-compose up -d
- sysdig
- sysdig offical https://www.sysdig.org/
- docker pull sysdig/sysdig
- docker run -i -t --name sysdig --privileged -v /var/run/docker.sock:/host/var/run/docker.sock -v /dev:/host/dev -v /proc:/host/proc:ro -v /boot:/host/boot:ro -v /lib/modules:/host/lib/modules:ro -v /usr:/host/usr:ro sysdig/sysdig
- docker container exec -it sysdig bash
- csysdig
- Weave Scope
- weave scope https://github.com/weaveworks/scope
常用的容器监控工具
Docker Runtime Metrics Guide
来源: http://www.bubuko.com/infodetail-2876474.html