Prometheus Exporter: json-exporter

甚麼是JSON Exporter

JSON Exporter是一個Prometheus的exporter（指標收集器），它能夠從提供JSON格式的API中收集指標數據，並將這些數據轉換為Prometheus所支持的格式，以便Prometheus進行分析和視覺化。

JSON Exporter的運行方式是通過設置JSON配置文件，其中包括API端點和相關的指標數據，然後使用Prometheus的配置文件將JSON Exporter添加到Prometheus的targets列表中。

JSON Exporter可以收集各種不同類型的指標數據，例如計數器（counters）、量規（gauges）和直方圖（histograms）等，並可以根據需要對數據進行轉換和聚合。

如何設置Exporter

在官方的Writing exporters文件中，有下面這一段

Each exporter should monitor exactly one instance application, preferably sitting right beside it on the same machine. That means for every HAProxy you run, you run a haproxy_exporter process. For every machine with a Mesos worker, you run the Mesos exporter on it, and another one for the master, if a machine has both.

The theory behind this is that for direct instrumentation this is what you’d be doing, and we’re trying to get as close to that as we can in other layouts. This means that all service discovery is done in Prometheus, not in exporters. This also has the benefit that Prometheus has the target information it needs to allow users probe your service with the blackbox exporter.

這段的意思是說，Exporter盡量和主要的Container放在同一個POD裡面，如下圖:

這樣做主要的原因是可以避免單點失敗，且更符合微服務架構的理念。

實作概要

下面是輸出的JSON檔案的範例

{
  "code": 0,
  "server": "vid-69t27o3",
  "streams": [
    {
      "id": "vid-0diw412",
      "name": "livestream",
      "vhost": "vid-y000397",
      "app": "live",
      "tcUrl": "rtmp://172.16.46.86:1935/live",
      "url": "/live/livestream",
      "live_ms": 1681903514993,
      "clients": 4,
      "frames": 0,
      "send_bytes": 45370,
      "recv_bytes": 34930,
      "kbps": {
        "recv_30s": 0,
        "send_30s": 0
      },
      "publish": {
        "active": false
      },
      "video": null,
      "audio": null
    }
  ]
}

下面是json-exporter的config.yml

modules:
  default:
    metrics:
    - name: server
      path: "{ .server}"

    - name: stream_clients
      type: object
      help: Example of sub-level value scrapes from a json
      path: '{.streams[?(@.name!="")]}'
      labels:
        name: '{.name}' 
      values:
        clients: '{.clients}' 
        send_bytes: '{.send_bytes}'
        recv_bytes: '{.recv_bytes}'
        frames: '{.frames}'
        publish: '{.publish.active}'

    headers:
      X-Dummy: my-test-header

設定要監控的POD的YAML

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/instance: srs-core1
  name: srs-core1
  namespace: stu-srs
spec:
  template:
    metadata:
	  creationTimestamp: null
      labels:
        app.kubernetes.io/instance: srs-core1
	containers:
      ....Other container here
      - image: dev-registry.xycloud.org/ldr/streaming/json-exporter
        imagePullPolicy: IfNotPresent
        name: json-exporter
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
		args: ["--config.file", "/config.yml"]
        volumeMounts:
        - mountPath: /config.yml
          name: json-exporter
          subPath: config.yml
	volumes:  
      - configMap:
          defaultMode: 420
          name: json-exporter
        name: json-exporter

進入該POD的SHELL裡面查看Json-exporter所吐的資料

curl "http://localhost:7979/probe?target=http://127.0.0.1:1985/api/v1/streams/"

可以把資料串到Prometheus了

請參考以下文章

Claire's Blog