運維監控報表太麻煩？用 Grafana-Reporter 一鍵搞定！

作者：劉俊夏 2024-12-30 09:24:19

Grafana Image Renderer插件可以與其他工具和系統集成，例如將生成的圖像嵌入到報告生成工具、文檔管理系統或自定義的應用程序中。這樣，用戶就能夠方便地分享、保存和展示Grafana儀表盤的快照，而無需訪問Grafana本身。

引言

為什么要部署這樣的一個組件呢？前幾天的早上，我們這邊提出了一個需求，能否把 Grafana 的 Dashboard 生成為 PDF 文件呢，然后再發送到相應的郵件或者聊天軟件，這樣做的好處是，比較好玩吧，新鮮，好看，哈哈，開玩笑的。

既然提出了這樣的需求，那肯定是咱有這個實力的，哈哈，自賣自夸。但總之，這個需求還是挺有意思的。我喜歡挑戰，喜歡新技術。

這邊折騰了會，就發現了 Grafan-reporter 這樣一款組件。

介紹

Github地址：https://github.com/IzakMarais/reporter，對中文不支持，dashboard 中含有中文字符會報錯。

首先介紹一下Grafana Image Renderer，它是一個用于將Grafana儀表盤轉換為靜態圖像的插件。它提供了在Grafana中生成PNG、JPEG和SVG格式圖像的功能，這些圖像可以用于導出儀表盤的快照、生成報告、在文檔中嵌入等用途。通過Grafana Image Renderer插件，用戶可以通過簡單的HTTP請求將特定的Grafana儀表盤轉換為靜態圖像。用戶可以指定要轉換的儀表盤、圖像的寬度和高度等參數，并通過HTTP響應獲取圖像的二進制數據。該插件在處理圖像時還支持渲染時過濾器的應用，例如應用時間范圍、自定義變量、主題等，以確保生成的圖像準確地顯示所需的數據和樣式。

Grafana Reporter 是一個基于Grafana的插件，用于生成和導出報告。它允許用戶將Grafana中的監控數據和儀表盤內容導出為報告，以便與團隊成員、管理層或其他人共享。Grafana Reporter支持導出為PDF、PNG、SVG和CSV等格式，可以定制報告的樣式、布局和內容。用戶可以設定報告的時間范圍、圖表和指標，并在報告中包含注釋、摘要和其他信息。Grafana Reporter使用戶能夠以可視化和易讀的方式展示和分享他們的監控數據。　

部署

本環境采用 Kubernetes 安裝，因為我喜歡用 Kubernetes，我看網上的文章幾乎沒有使用 Kubernetes 安裝的。你也可以使用 Docker 或者 Docker-compose，又或者源碼編譯安裝。

這邊的監控采用的是 kube-prometheus-stack 的 Helm chart 安裝，這邊的重點不在安裝 Prometheus，所以就直接跳過了。

因為有些信息比較敏感，這邊的域名都做了修改。

部署 Grafana-reporter

這邊使用 StatefulSet 部署，他這邊需要指定下我們的 Grafana 的地址，還有我們的 Localtime 也需要修改為本地的。

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: grafana-reporter
  namespace: monitoring
  labels:
    app: grafana-reporter
spec:
  replicas: 1
  selector:
    matchLabels:
      app: grafana-reporter
  serviceName: grafana-reporter
  template:
    metadata:
      labels:
        app: grafana-reporter
    spec:
      hostNetwork: true 
      containers:
        - name: grafana-reporter
          image: izakmarais/grafana-reporter
          imagePullPolicy: Always
          ports:
            - containerPort: 8686
          args:
            - "-proto=https://"
            - "-ip=grafana.example.com"
            - "-ssl-check=true" # 如果 Grafana 使用了自簽名證書,我這邊因為有證書，所以就true了
          volumeMounts:
            - name: localtime
              mountPath: /etc/localtime
              readOnly: true
      volumes:
        - name: localtime
          hostPath:
            path: /etc/localtime
            type: File

我們還需要創建一個 Service：

apiVersion: v1
kind: Service
metadata:
  name: grafana-reporter
  namespace: monitoring
  labels:
    app: grafana-reporter
spec:
  selector:
    app: grafana-reporter
  ports:
    - protocol: TCP
      port: 8686        
      targetPort: 8686  
  type: ClusterIP

再來一個 Ingress：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: id-test-reporter
  namespace: monitoring
spec:
  ingressClassName: nginx
  rules:
  - host: reporter.example.com
    http:
      paths:
      - backend:
          service:
            name: grafana-reporter
            port:
              number: 8686
        path: /
        pathType: ImplementationSpecific
  tls:
  - hosts:
    - reporter.example.com
    secretName: hiwin-test-ph-com-tls

Apply 所有；

kubectl apply -f .

我們來看下它的頁面：

圖片

這就算成功了。

部署 Grafana-image-render

這邊使用 StatefulSet 控制器部署：

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: grafana-image-renderer
  namespace: monitoring
  labels:
    app: grafana-image-renderer
spec:
  serviceName: grafana-image-renderer-headless
  replicas: 1
  selector:
    matchLabels:
      app: grafana-image-renderer
  template:
    metadata:
      labels:
        app: grafana-image-renderer
    spec:
      containers:
        - name: renderer
          image: grafana/grafana-image-renderer:latest
          ports:
            - containerPort: 8081
          env:
            - name: RENDERER_CALLBACK_URL
              value: "https://grafana.example.com" # Grafana 服務地址

service：

apiVersion: v1
kind: Service
metadata:
  name: grafana-image-renderer
  namespace: monitoring
spec:
  selector:
    app: grafana-image-renderer
  ports:
    - protocol: TCP
      port: 8081
      targetPort: 8081
  type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
  name: grafana-image-renderer-headless
  namespace: monitoring
spec:
  selector:
    app: grafana-image-renderer
  ports:
    - protocol: TCP
      port: 8081
      targetPort: 8081
  clusterIP: None

Ingress：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: grafana-image-renderer-ingress
  namespace: monitoring
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: "50m" # 調整請求體大小限制
    nginx.ingress.kubernetes.io/proxy-read-timeout: "300"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "300"
    nginx.ingress.kubernetes.io/limit-connections: "50"

spec:
  ingressClassName: nginx
  rules:
    - host: render.example.com # 替換為您自己的域名
      http:
        paths:
          - path: /
            pathType: ImplementationSpecific
            backend:
              service:
                name: grafana-image-renderer-headless   # 這邊使用的是 Headless 服務
                port:
                  number: 8081
  tls:
    - hosts:
        - render.example.com
      secretName: example-com-tls

Apply 部署：

kubectl apply -f .

測試一些是否有問題：

圖片

可以看到，沒有問題。

在一開始的部署中，我是不知道還需要部署 Grafana-image-render，所以當時因為這個，一直不能生成 PDF 文件。行了，往前看。

配置 Grafana

接下來我們配置下 Grafana，我們需要為 Grafana 添加下變量：

apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
    app.kubernetes.io/instance: kube-prometheus-stack
    app.kubernetes.io/name: grafana
    app.kubernetes.io/version: 11.3.1
    helm.sh/chart: grafana-8.6.3
  name: kube-prometheus-stack-grafana
  namespace: monitoring
spec:
  persistentVolumeClaimRetentionPolicy:
    whenDeleted: Retain
    whenScaled: Retain
  podManagementPolicy: OrderedReady
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/instance: kube-prometheus-stack
      app.kubernetes.io/name: grafana
  serviceName: kube-prometheus-stack-grafana-headless
  template:
    metadata:
      annotations:
        checksum/config: e1e314c7fdc64d2ecfb41c8c9c1d43e5dbf8d4fc87a11b2373bfbba95ff86a24
        checksum/dashboards-json-config: 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b
        checksum/sc-dashboard-provider-config: ce3cfbda1358535f70c551752a34e1d92271c95272f1891dfef8d65a403063e4
        checksum/secret: f89c841c5ea6371a733417993185fefcf0760914325c94b37519ea2e919222a5
        kubectl.kubernetes.io/default-container: grafana
      labels:
        app.kubernetes.io/instance: kube-prometheus-stack
        app.kubernetes.io/name: grafana
        app.kubernetes.io/version: 11.3.1
        helm.sh/chart: grafana-8.6.3
    spec:
      automountServiceAccountToken: true
      containers:
      - env:
        - name: METHOD
          value: WATCH
        - name: LABEL
          value: grafana_dashboard
        - name: LABEL_VALUE
          value: "1"
        - name: FOLDER
          value: /tmp/dashboards
        - name: RESOURCE
          value: both
        - name: NAMESPACE
          value: ALL
        - name: REQ_USERNAME
          valueFrom:
            secretKeyRef:
              key: admin-user
              name: kube-prometheus-stack-grafana
        - name: REQ_PASSWORD
          valueFrom:
            secretKeyRef:
              key: admin-password
              name: kube-prometheus-stack-grafana
        - name: REQ_URL
          value: http://localhost:3000/api/admin/provisioning/dashboards/reload
        - name: REQ_METHOD
          value: POST
        image: quay.io/kiwigrid/k8s-sidecar:1.28.0
        imagePullPolicy: IfNotPresent
        name: grafana-sc-dashboard
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          seccompProfile:
            type: RuntimeDefault
        volumeMounts:
        - mountPath: /tmp/dashboards
          name: sc-dashboard-volume
          
      - env:
        - name: METHOD
          value: WATCH
        - name: LABEL
          value: grafana_datasource
        - name: LABEL_VALUE
          value: "1"
        - name: FOLDER
          value: /etc/grafana/provisioning/datasources
        - name: RESOURCE
          value: both
        - name: REQ_USERNAME
          valueFrom:
            secretKeyRef:
              key: admin-user
              name: kube-prometheus-stack-grafana
        - name: REQ_PASSWORD
          valueFrom:
            secretKeyRef:
              key: admin-password
              name: kube-prometheus-stack-grafana
        - name: REQ_URL
          value: http://localhost:3000/api/admin/provisioning/datasources/reload
        - name: REQ_METHOD
          value: POST
        image: quay.io/kiwigrid/k8s-sidecar:1.28.0
        imagePullPolicy: IfNotPresent
        name: grafana-sc-datasources
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          seccompProfile:
            type: RuntimeDefault
        volumeMounts:
        - mountPath: /etc/grafana/provisioning/datasources
          name: sc-datasources-volume
          
      - env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: status.podIP
        - name: GF_SECURITY_ADMIN_USER
          valueFrom:
            secretKeyRef:
              key: admin-user
              name: kube-prometheus-stack-grafana
        - name: GF_SECURITY_ADMIN_PASSWORD
          valueFrom:
            secretKeyRef:
              key: admin-password
              name: kube-prometheus-stack-grafana
        - name: GF_PATHS_DATA
          value: /var/lib/grafana/
        - name: GF_PATHS_LOGS
          value: /var/log/grafana
        - name: GF_PATHS_PLUGINS
          value: /var/lib/grafana/plugins
        - name: GF_PATHS_PROVISIONING
          value: /etc/grafana/provisioning
        - name: GF_RENDERING_SERVER_URL      # Grafana 將向該地址發送渲染請求，并由 Image Renderer 服務生成圖片（例如 PNG、JPEG）
          value: https://render.example.com
        - name: GF_RENDERING_CALLBACK_URL     # 當 Image Renderer 完成渲染任務后，它會將生成的圖片或渲染結果回傳給 Grafana
          value: https://grafana.example.com
        - name: GF_LOG_FILTERS
          value: rendering:debug
        image: docker.io/grafana/grafana:11.3.1
        imagePullPolicy: IfNotPresent
        livenessProbe:
          httpGet:
            path: /api/health
            port: 3000
            scheme: HTTP
          initialDelaySeconds: 60
          periodSeconds: 10
          timeoutSeconds: 30
        name: grafana
        ports:
        - containerPort: 3000
          name: grafana
        - containerPort: 9094
          name: gossip-tcp
        - containerPort: 9094
          name: gossip-udp
        readinessProbe:
          httpGet:
            path: /api/health
            port: 3000
            scheme: HTTP
          periodSeconds: 10
          timeoutSeconds: 1
        volumeMounts:
        - mountPath: /etc/grafana/grafana.ini
          name: config
          subPath: grafana.ini
        - mountPath: /var/lib/grafana
          name: storage
        - mountPath: /tmp/dashboards
          name: sc-dashboard-volume
        - mountPath: /etc/grafana/provisioning/dashboards/sc-dashboardproviders.yaml
          name: sc-dashboard-provider
          subPath: provider.yaml
        - mountPath: /etc/grafana/provisioning/datasources
          name: sc-datasources-volume
      initContainers:
      - command:
        - chown
        - -R
        - 472:472
        - /var/lib/grafana
        image: docker.io/library/busybox:1.31.1
        imagePullPolicy: IfNotPresent
        name: init-chown-data
        securityContext:
          capabilities:
            add:
            - CHOWN
          runAsUser: 0
        volumeMounts:
        - mountPath: /var/lib/grafana
          name: storage
      restartPolicy: Always
      securityContext:
        fsGroup: 472
        runAsUser: 472
      serviceAccountName: kube-prometheus-stack-grafana
      terminationGracePeriodSeconds: 30
      volumes:
      - configMap:
          name: kube-prometheus-stack-grafana
        name: config
      - emptyDir: {}
        name: sc-dashboard-volume
      - configMap:
          name: kube-prometheus-stack-grafana-config-dashboards
        name: sc-dashboard-provider
      - emptyDir: {}
        name: sc-datasources-volume
  updateStrategy:
    rollingUpdate:
      partition: 0
    type: RollingUpdate
  volumeClaimTemplates:
  - metadata:
      name: storage
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 20Gi
      storageClassName: alicloud-nas-subpath

等待更新完成后，可以測試一些是否有問題。

測試

如何測試呢，不要著急，跟緊我的步驟。

首先我們需要先在 Grafana 中創建一個 SA (serviceaccount)，用于授權和訪問。

圖片

這邊起個名字，再選擇一個 Admin 的 Role ：

圖片

創建完成之后，再創建一個 Token，保存好：

圖片

接下來我們還需要獲取到 Grafana Dashboard 的 ID，這邊選擇一個你想要生成 PDF 文件的 Dashboard：

然后進入，注意如何選擇：

https://grafana.example.com/d/efa86fd1d0c121a26444b636a3f509a8/kubernetes-compute-resources-cluster?orgId=1&from=now-1h&to=now&timeznotallow=utc&var-datasource=default&var-cluster=&refresh=10s

域名后面的 d/ 到下一個 / 之間的就是我們需要的 ID

圖片

接下來我們開始真正的測試：

完成格式為：

https://reporter.example.com/api/v5/report/alertmanager-overview?apitoken=glsa_M4a256mzUizdmdGsSNK8SZW7h9QfvhGr_6616e37c

我們來解析下：

https://reporter.example.com/

? 這個是你的 Grafana-reporter 的域名

api/v5/report/

? 這個是固定的格式，不能改

alertmanager-overview

? 這個也就是你的 Grafana Dashboard 的 ID

?apitoken=glsa_M4a256mzUizdmdGsSNK8SZW7h9QfvhGr_6616e37c

? 這個就是你的 Grafana SA 生成的 Token

接下來我們訪問下，看下是否有問題。

可以看到失敗了，為什么呢？我們來排查下：

提前說明下，我這邊 kubectl 的命令都配置了 Alias。如果你也需要，請查看這篇文章。

進入容器

$ kgpo -nmonitoring
NAME                                                        READY   STATUS    RESTARTS   AGE
alertmanager-kube-prometheus-stack-alertmanager-0           2/2     Running   0          19d
grafana-image-renderer-0                                    1/1     Running   0          156m
grafana-image-renderer-1                                    1/1     Running   0          156m
grafana-image-renderer-2                                    1/1     Running   0          156m
grafana-reporter-0                                          1/1     Running   0          92m
kube-prometheus-stack-grafana-0                             3/3     Running   0          34m
kube-prometheus-stack-kube-state-metrics-5b58cf95c8-blv79   1/1     Running   0          19d
kube-prometheus-stack-operator-b48974f48-m59d2              1/1     Running   0          19d
prometheus-kube-prometheus-stack-prometheus-0               2/2     Running   0          19d

$ kex grafana-reporter-0 -nmonitoring -- bash
root@grafana-reporter-0:/# cd /tmp/   # 這個目錄是它默認存放生成的 PDF 文件的位置
root@grafana-reporter-0:/tmp# ls
e4b21f3b-b031-4fa2-9443-a37f1095261e
root@grafana-reporter-0:/tmp# cd e4b21f3b-b031-4fa2-9443-a37f1095261e/
root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e# ls
images  report.aux  report.log  report.tex
root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e# cd images/
root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# ls
image2.png  image3.png  image5.png  image6.png

我們可以使用 file 這個命令查看一些這個文件的類型

$ root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# file image2.png 
bash: file: command not found

那就安裝下這個命令；

$ root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# apt update
Ign:1 http://deb.debian.org/debian stretch InRelease
Ign:2 http://security.debian.org/debian-security stretch/updates InRelease
Ign:3 http://deb.debian.org/debian stretch-updates InRelease
Err:4 http://security.debian.org/debian-security stretch/updates Release
  404  Not Found [IP: 151.101.2.132 80]
Err:5 http://deb.debian.org/debian stretch Release
  404  Not Found [IP: 199.232.46.132 80]
Err:6 http://deb.debian.org/debian stretch-updates Release
  404  Not Found [IP: 199.232.46.132 80]
Reading package lists... Done
E: The repository 'http://security.debian.org/debian-security stretch/updates Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'http://deb.debian.org/debian stretch Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'http://deb.debian.org/debian stretch-updates Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

報錯了，那就修改下。

問題是由于容器中的 apt 使用了一個過期的 Debian 軟件源 (stretch)，這些源已經不再維護或者被替換，因此無法正確獲取包列表

$ root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# echo "deb http://archive.debian.org/debian stretch main contrib non-free" > /etc/apt/sources.list
$ root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# echo "deb http://archive.debian.org/debian-security stretch/updates main contrib non-free" >> /etc/apt/sources.list

# 安裝
$ root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# apt update
Ign:1 http://deb.debian.org/debian stretch InRelease
Ign:2 http://security.debian.org/debian-security stretch/updates InRelease
Ign:3 http://deb.debian.org/debian stretch-updates InRelease
Err:4 http://security.debian.org/debian-security stretch/updates Release
  404  Not Found [IP: 151.101.2.132 80]
Err:5 http://deb.debian.org/debian stretch Release
  404  Not Found [IP: 199.232.46.132 80]
Err:6 http://deb.debian.org/debian stretch-updates Release
  404  Not Found [IP: 199.232.46.132 80]
Reading package lists... Done
E: The repository 'http://security.debian.org/debian-security stretch/updates Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'http://deb.debian.org/debian stretch Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'http://deb.debian.org/debian stretch-updates Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

$ root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# echo "deb http://archive.debian.org/debian stretch main contrib non-free" > /etc/apt/sources.list
$ root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# echo "deb http://archive.debian.org/debian-security stretch/updates main contrib non-free" >> /etc/apt/sources.list
$ root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# apt update
Ign:1 http://archive.debian.org/debian stretch InRelease
Get:2 http://archive.debian.org/debian-security stretch/updates InRelease [59.1 kB]
Get:3 http://archive.debian.org/debian stretch Release [118 kB]
Get:4 http://archive.debian.org/debian stretch Release.gpg [3177 B]
Get:5 http://archive.debian.org/debian-security stretch/updates/main amd64 Packages [782 kB]
Get:6 http://archive.debian.org/debian-security stretch/updates/non-free amd64 Packages [14.1 kB]
Get:7 http://archive.debian.org/debian-security stretch/updates/contrib amd64 Packages [1760 B]
Get:8 http://archive.debian.org/debian stretch/main amd64 Packages [7080 kB]
Get:9 http://archive.debian.org/debian stretch/contrib amd64 Packages [50.7 kB]
Get:10 http://archive.debian.org/debian stretch/non-free amd64 Packages [78.3 kB]
Fetched 8186 kB in 1s (6008 kB/s)                         
Reading package lists... Done
Building dependency tree       
Reading state information... Done
31 packages can be upgraded. Run 'apt list --upgradable' to see them.

$ root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# apt install -y file
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libmagic-mgc libmagic1
The following NEW packages will be installed:
  file libmagic-mgc libmagic1
0 upgraded, 3 newly installed, 0 to remove and 31 not upgraded.
Need to get 397 kB of archives.
After this operation, 5265 kB of additional disk space will be used.
Get:1 http://archive.debian.org/debian stretch/main amd64 libmagic-mgc amd64 1:5.30-1+deb9u3 [222 kB]
Get:2 http://archive.debian.org/debian stretch/main amd64 libmagic1 amd64 1:5.30-1+deb9u3 [111 kB]
Get:3 http://archive.debian.org/debian stretch/main amd64 file amd64 1:5.30-1+deb9u3 [64.2 kB]
Fetched 397 kB in 0s (4579 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libmagic-mgc.
(Reading database ... 6831 files and directories currently installed.)
Preparing to unpack .../libmagic-mgc_1%3a5.30-1+deb9u3_amd64.deb ...
Unpacking libmagic-mgc (1:5.30-1+deb9u3) ...
Selecting previously unselected package libmagic1:amd64.
Preparing to unpack .../libmagic1_1%3a5.30-1+deb9u3_amd64.deb ...
Unpacking libmagic1:amd64 (1:5.30-1+deb9u3) ...
Selecting previously unselected package file.
Preparing to unpack .../file_1%3a5.30-1+deb9u3_amd64.deb ...
Unpacking file (1:5.30-1+deb9u3) ...
Setting up libmagic-mgc (1:5.30-1+deb9u3) ...
Setting up libmagic1:amd64 (1:5.30-1+deb9u3) ...
Processing triggers for libc-bin (2.24-11+deb9u3) ...
Setting up file (1:5.30-1+deb9u3) ...

# 看一下這個文件
root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# ls
image2.png  image3.png  image5.png  image6.png
root@grafana-reporter-0:/tmp/e4b21f3b-b031-4fa2-9443-a37f1095261e/images# file image2.png 
image2.png: ASCII text, with no line terminators

可以看到，這是文本，不是 png 圖片，所以問題是 grafana 沒有成功調用到 Grafana-image-render，去生成相應的 PNG 圖片。

我們這邊再去看下我們的 Grafana 的 logs

# 截取一部分 logs，看起來沒有什么特別大的問題：
logger=rendering renderer=http t=2024-12-26T10:41:49.307925807Z level=debug msg="calling remote rendering service" url="https://render.example.com?deviceScaleFactor=1.000000&domain=grafana.example.com&encoding=png&height=500&renderKey=lFiW6KBBhrA8vY2BQ9OOXdPSZUU2FL5b&timeout=60&timeznotallow=&url=https%3A%2F%2Fgrafana.example.com%2Fd-solo%2Falertmanager-overview%2F_%3Ffrom%3Dnow-1h%26height%3D500%26panelId%3D2%26theme%3Dlight%26to%3Dnow%26width%3D1000%26render%3D1&width=1000"
logger=context userId=1 orgId=1 uname=admin t=2024-12-26T10:41:53.196481964Z level=info msg="Request Completed" method=GET path=/api/live/ws status=-1 remote_addr=127.0.0.1 time_ms=5 duratinotallow=5.670954ms size=0 referer= handler=/api/live/ws status_source=server
logger=live t=2024-12-26T10:41:53.389851883Z level=info msg="Initialized channel handler" channel=grafana/dashboard/uid/efa86fd1d0c121a26444b636a3f509a8 address=grafana/dashboard/uid/efa86fd1d0c121a26444b636a3f509a8
logger=rendering renderer=http t=2024-12-26T10:42:18.992113007Z level=info msg=Rendering path="d-solo/alertmanager-overview/_?from=now-1h&height=500&panelId=6&theme=light&to=now&width=1000" userID=2
logger=rendering renderer=http t=2024-12-26T10:42:18.995074149Z level=info msg=Rendering path="d-solo/alertmanager-overview/_?from=now-1h&height=500&panelId=2&theme=light&to=now&width=1000" userID=2
logger=rendering renderer=http t=2024-12-26T10:42:19.010142812Z level=debug msg="calling remote rendering service" url="https://render.example.com?deviceScaleFactor=1.000000&domain=grafana.example.com&encoding=png&height=500&renderKey=ElH0jx7AYodbbkv2lEzvV7Qf1OkeiRac&timeout=60&timeznotallow=&url=https%3A%2F%2Fgrafana.example.com%2Fd-solo%2Falertmanager-overview%2F_%3Ffrom%3Dnow-1h%26height%3D500%26panelId%3D6%26theme%3Dlight%26to%3Dnow%26width%3D1000%26render%3D1&width=1000"
logger=rendering renderer=http t=2024-12-26T10:42:19.011440398Z level=info msg=Rendering path="d-solo/alertmanager-overview/_?from=now-1h&height=500&panelId=5&theme=light&to=now&width=1000" userID=2
logger=rendering renderer=http t=2024-12-26T10:42:19.024147345Z level=debug msg="calling remote rendering service" url="https://render.example.com?deviceScaleFactor=1.000000&domain=grafana.example.com&encoding=png&height=500&renderKey=FGMPcxnruw1ZbPXEeH4v0BBrEeUSKvjR&timeout=60&timeznotallow=&url=https%3A%2F%2Fgrafana.example.com%2Fd-solo%2Falertmanager-overview%2F_%3Ffrom%3Dnow-1h%26height%3D500%26panelId%3D5%26theme%3Dlight%26to%3Dnow%26width%3D1000%26render%3D1&width=1000"
logger=rendering renderer=http t=2024-12-26T10:42:19.031615719Z level=info msg=Rendering path="d-solo/alertmanager-overview/_?from=now-1h&height=500&panelId=3&theme=light&to=now&width=1000" userID=2
logger=rendering renderer=http t=2024-12-26T10:42:19.044274424Z level=debug msg="calling remote rendering service" url="https://render.example.com?deviceScaleFactor=1.000000&domain=grafana.example.com&encoding=png&height=500&renderKey=a47WWfW9PiO75DoNSHKwfdMwPSeTIiNo&timeout=60&timeznotallow=&url=https%3A%2F%2Fgrafana.example.com%2Fd-solo%2Falertmanager-overview%2F_%3Ffrom%3Dnow-1h%26height%3D500%26panelId%3D3%26theme%3Dlight%26to%3Dnow%26width%3D1000%26render%3D1&width=1000"
logger=rendering renderer=http t=2024-12-26T10:42:19.105601469Z level=debug msg="calling remote rendering service" url="https://render.example.com?deviceScaleFactor=1.000000&domain=grafana.example.com&encoding=png&height=500&renderKey=KnO2tIXJg9EvhcbhfDjbadYxAqChU0eN&timeout=60&timeznotallow=&url=https%3A%2F%2Fgrafana.example.com%2Fd-solo%2Falertmanager-overview%2F_%3Ffrom%3Dnow-1h%26height%3D500%26panelId%3D2%26theme%3Dlight%26to%3Dnow%26width%3D1000%26render%3D1&width=1000"

我們來排查下，看一下我們的 Grafana 的 YAML 文件里面的 image-render 的地址，其它多余的就不看了，問題重要在這里，去 Google 了一會，發現我可能少一個路徑 /render，這個是默認的，那我們就加上去，看下它是否 OK：

k edit sts kube-prometheus-stack-grafana -nmonitoring
····
        - name: GF_RENDERING_SERVER_URL
          value: https://render.example.com/render
        - name: GF_RENDERING_CALLBACK_URL 
          value: https://grafana.example.com
····

訪問下：

圖片

可以看到，我們成功了，如果你這邊還有問題，可以查看下 Grafana 和 Grafana-reporter 的 logs。

責任編輯：武曉燕來源：云原生運維圈

運維監控插件