Kubernetes拓扑可视化监控之Goldpinger

586 字
3 分钟
Kubernetes拓扑可视化监控之Goldpinger

集群节点数量过多,经常会遇到某个节点访问出现问题,却不能及时发现,goldpinger就是为了解决此问题诞生的,与其他服务相比,goldpinger更加轻量 此文档通过argo cd管理goldpinger 尝试过Weave Scope,发现使用资源过多,参考[[Kubernetes拓扑可视化监控之Scope]]

参考文档: goldpinger

一, 部署#

1 准备配置文件#

如果使用argo需要存放到git上,参考 [[../DevOps/ArgoCD#2 argocd-server 配置与使用|ArgoCD]]

Terminal window
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: goldpinger-serviceaccount
namespace: goldpinger
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: goldpinger
namespace: goldpinger
annotations:
link.argocd.argoproj.io/external-link: http://10.1.167.173:30080 #argocd显示跳转的链接
labels:
app: goldpinger
spec:
updateStrategy:
type: RollingUpdate
selector:
matchLabels:
app: goldpinger
template:
metadata:
labels:
app: goldpinger
spec:
serviceAccount: "goldpinger-serviceaccount"
securityContext:
runAsNonRoot: true
runAsUser: 1000
fsGroup: 2000
containers:
- name: goldpinger
env:
- name: HOST
value: "0.0.0.0"
- name: PORT
value: "8080"
- name: HOSTNAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: HOSTS_TO_RESOLVE
value: "www.baidu.com ccops.cc kubernetes.default" # 要检测解析域名
- name: HTTP_TARGETS
value: http://ccops.cc # 要检测访问域名
- name: TCP_TARGETS
value: 8.8.8.8:53 # 要检测使用的dns服务器
image: "docker.io/bloomberg/goldpinger:v3.7.0"
imagePullPolicy: Always
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
resources:
limits:
memory: 80Mi
requests:
cpu: 1m
memory: 40Mi
ports:
- containerPort: 8080
name: http
readinessProbe:
httpGet:
path: /healthz
port: 8080
initialDelaySeconds: 20
periodSeconds: 5
livenessProbe:
httpGet:
path: /healthz
port: 8080
initialDelaySeconds: 20
periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
name: goldpinger
namespace: goldpinger
annotations:
link.argocd.argoproj.io/external-link: http://10.1.167.173:30080
labels:
app: goldpinger
spec:
type: NodePort
ports:
- port: 8080
name: http
nodePort: 30080 # 暴露的端口
selector:
app: goldpinger
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: goldpinger-clusterrole
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- list # 只给查看pod的权限
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: goldpinger-clusterrolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: goldpinger-clusterrole
subjects:
- kind: ServiceAccount
name: goldpinger-serviceaccount
namespace: goldpinger

2 argocd同步#

这里直接查看状态了

image.png
image.png

3 查看goldpinger#

点击这里就会跳转到annotations 里配的地址

image.png
image.png
可以看到已经有拓扑了
image.png
image.png

监控与告警#

1 prometheus采集#

- job_name: 'goldpinger'
metrics_path: '/metrics'
static_configs:
- targets: ["10.1.1.1:30080"]

2 告警规则#

Terminal window
rules]# cat goldpinger.yml
groups:
- name: goldpinger
rules:
- alert: goldpinger_nodes_unhealthy
expr: sum(goldpinger_nodes_health_total{status="unhealthy"})
BY (instance, goldpinger_instance) > 0
for: 5m
annotations:
description: |
Goldpinger instance {{ $labels.goldpinger_instance }} has been reporting unhealthy nodes for at least 5 minutes.
summary: 'Instance {{ $labels.instance }} down'
- alert: goldpinger_http_unhealthy
expr: sum(goldpinger_http_errors_total) BY (instance, goldpinger_instance, host) > 0
for: 1m
annotations:
description: |
Goldpinger instance {{ $labels.goldpinger_instance }} unable to access {{ $labels.host }} for at least 1 minutes.
summary: '{{ $labels.goldpinger_instance }} unable to access {{ $labels.host }}'
- alert: goldpinger_tcp_unhealthy
expr: sum(goldpinger_tcp_errors_total) BY (instance, goldpinger_instance, host) > 1
for: 1m
annotations:
description: |
Goldpinger instance {{ $labels.goldpinger_instance }} unable to access {{ $labels.host }} for at least 1 minutes.
summary: '{{ $labels.goldpinger_instance }} unable to access {{ $labels.host }}'

3 grafana#

导入此模板

3.1 查看#

image.png
image.png

文章分享

如果这篇文章对你有帮助,欢迎分享给更多人!

Kubernetes拓扑可视化监控之Goldpinger
https://ccops.cc/posts/5e10a5d8/
作者
蹦蹦站长
发布于
2023-03-31
许可协议
CC BY-NC-SA 4.0

评论区

Profile Image of the Author
蹦蹦站长
一个养兔子的运维
公告
感谢你的来访!希望在这里能找到对你有用的内容!
音乐
封面

音乐

暂未播放

0:00 0:00
暂无歌词
分类
标签
站点统计
文章
81
分类
6
标签
45
总字数
88,456
运行时长
0
最后活动
0 天前
站点信息
构建平台
Local
博客版本
Firefly v6.12.1
文章许可
CC BY-NC-SA 4.0

文章目录