一, Prometheus 支持的多种服务发现机制 (常用如下)
static_configs: 静态服务发现
file_sd_configs: 文件服务发现
dns_sd_configs: DNS 服务发现
kubernetes_sd_configs: Kubernetes 服务发现
consul_sd_configs: Consul 服务发现
二, Prometheus 主要配置 prometheus.YAML 中的 scrape_configs 以及 consul_sd_configs 如下
- scrape_configs:
- # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
- - job_name: 'prometheus'
- # metrics_path defaults to '/metrics'
- # scheme defaults to 'http'.
- static_configs:
- - targets: ['localhost:9090']
- - job_name: 'consul'
- consul_sd_configs:
- - server: 'localhost:8500'
- relabel_configs:
- - source_labels: [__meta_consul_tags]
- regex: .*,prome,.*
- action: keep
- - source_labels: [__meta_consul_service]
- target_label: job
通过重新加载 prometheus.YAML 配置添加
./prometheus --config.file="prometheus.yml" --web.listen-address="0.0.0.0:9090" --Web.enable-admin-API --Web.enable-lifecycle &
--Web.enable-admin-API 运行通过 Web 方式管理 prometheus(删除清空数据等操作)
--Web.enable-lifecycle 运行通过 Web 方式重新加载 prometheus 配置 (相当于 reload)
三, k8s 环境实现
将配置制作成 secret 挂在到 prometheus 中
1, 编写 prometheus-additional-consul.YAML
- scrape_configs:
- # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
- - job_name: 'prometheus'
- # metrics_path defaults to '/metrics'
- # scheme defaults to 'http'.
- static_configs:
- - targets: ['localhost:9090']
- - job_name: 'consul'
- consul_sd_configs:
- - server: 'localhost:8500'
- relabel_configs:
- - source_labels: [__meta_consul_tags]
- regex: .*,prome,.*
- action: keep
- - source_labels: [__meta_consul_service]
- target_label: job
2, 制作 secret
kubectl create secret generic additional-consul-configs --from-file=prometheus-additional-consul.YAML -n monitoring
3, 添加配置到 prometheus 的 pod 中, 修改 prometheus-prometheus.YAML, 并更新资源
- apiVersion: monitoring.coreos.com/v1
- kind: Prometheus
- metadata:
- labels:
- prometheus: k8s
- name: k8s
- namespace: monitoring
- spec:
- alerting:
- alertmanagers:
- - name: alertmanager-main
- namespace: monitoring
- port: Web
- baseImage: quay.io/prometheus/prometheus
- nodeSelector:
- kubernetes.io/os: Linux
- podMonitorNamespaceSelector: {}
- podMonitorSelector: {}
- replicas: 2
- resources:
- requests:
- memory: 400Mi
- ruleSelector:
- matchLabels:
- prometheus: k8s
- role: alert-rules
- securityContext:
- fsGroup: 2000
- runAsNonRoot: true
- runAsUser: 1000
- serviceAccountName: prometheus-k8s
- serviceMonitorNamespaceSelector: {}
- serviceMonitorSelector: {}
- version: v2.11.0
- additionalScrapeConfigs:
- name: additional-consul-configs
- key: prometheus-additional-consul.YAML
4, 注意事项: 权限不够 (配置更新, 但是没有对应的监控任务生成)
修改名为 prometheus-k8s 的 ClusterRole 权限, 并更新资源
- apiVersion: rbac.authorization.k8s.io/v1
- kind: ClusterRole
- metadata:
- name: prometheus-k8s
- rules:
- - apiGroups:
- - ""
- resources:
- - nodes
- - services
- - endpoints
- - pods
- - nodes/proxy
- verbs:
- - get
- - list
- - watch
- - apiGroups:
- - ""
- resources:
- - configmaps
- - nodes/metrics
- verbs:
- - get
- - nonResourceURLs:
- - /metrics
- verbs:
- - get
来源: http://www.bubuko.com/infodetail-3341494.html