摘要：
普罗米修斯提供了远程写和远程读数据存储方法，可以帮助我们长时间存储数据并方便查询。板条箱b提供了相应的适配器，可以直接进行调整。
prometheus 提供了remote_write 以及remote_read 的数据存储方式，可以帮助我们进行数据的长时间存储、方便查询
cratedb 提供了对应的adapter，可以直接进行适配。
以下演示一个简单的cratedb 集群以及通过write 以及read 存储通过grok exporter 暴露的日记prometheus metrics
环境准备
- 数据请求流程
inputlog->grok exporter -> prometheus->cratedb adpater->cratedb cluster - docker-compose 文件
version: "3"services: grafana: image: grafana/grafana ports: - "3000:3000" prometheus: image: prom/prometheus volumes: - "./prometheus.yml:/etc/prometheus/prometheus.yml" ports: - "9090:9090" cratedb-adapter: image: crate/crate_adapter command: -config.file /opt/config/config.yml ports: - "9268:9268" volumes: - "./cratedb-adapter:/opt/config" grok: image: dalongrong/grok-exporter volumes: - "./grok/example:/opt/example" - "./grok/grok.yaml:/grok/config.yml" ports: - "9144:9144" crate1: image: crate volumes: - "./cratedb/data1:/data" - "./cratedb/1.yaml:/crate/config/crate.yml" ports: - "4200:4200" - "4300:4300" - "5432:5432" crate2: image: crate volumes: - "./cratedb/data2:/data" - "./cratedb/2.yaml:/crate/config/crate.yml" ports: - "4201:4200" - "4301:4300" - "5433:5432" crate3: image: crate volumes: - "./cratedb/data3:/data" - "./cratedb/3.yaml:/crate/config/crate.yml" ports: - "4202:4200" - "4302:4300" - "5434:5432"- prometheus 配置
通过静态配置的方式添加target,统计配置了remote_write 以及remote_read
scrape_configs: - job_name: grok metrics_path: /metrics scrape_interval: 10s scrape_timeout: 10s static_configs: - targets: ['grok:9144'] - job_name: cratedb-adapter metrics_path: /metrics scrape_interval: 10s scrape_timeout: 10s static_configs: - targets: ['cratedb-adapter:9268']remote_write: - url: http://cratedb-adapter:9268/writeremote_read: - url: http://cratedb-adapter:9268/read- cratedb 集群配置
当前使用的是社区版本，对于集群模式，最大支持的是3个节点的，但是一般场景也够用了
- node1 配置
cluster.name: crateclusternode.name: crate1node.master: truenode.data: truehttp.port: 4200psql.port: 5432transport.tcp.port: 4300discovery.seed_hosts: ["crate1"]cluster.initial_master_nodes: ["crate1"]http.cors.enabled: truehttp.cors.allow-origin: "*"gateway.expected_nodes: 3gateway.recover_after_nodes: 2gateway.recover_after_time: 5mnetwork.host: _local_,_site_path.logs: /data/logpath.data: /data/datablobs.path: /data/blobs - node2 配置
cluster.name: crateclusternode.name: crate2node.master: falsenode.data: truehttp.port: 4200psql.port: 5432transport.tcp.port: 4300discovery.seed_hosts: ["crate1"]cluster.initial_master_nodes: ["crate1"]http.cors.enabled: truehttp.cors.allow-origin: "*"gateway.expected_nodes: 3gateway.recover_after_nodes: 2gateway.recover_after_time: 5mnetwork.host: _local_,_site_path.logs: /data/logpath.data: /data/datablobs.path: /data/blobs - node3 配置cluster.name: crateclusternode.name: crate3node.master: falsenode.data: truehttp.port: 4200psql.port: 5432transport.tcp.port: 4300discovery.seed_hosts: ["crate1"]cluster.initial_master_nodes: ["crate1"]http.cors.enabled: truehttp.cors.allow-origin: "*"network.host: _local_,_site_gateway.expected_nodes: 3gateway.recover_after_nodes: 2gateway.recover_after_time: 5mpath.logs: /data/logpath.data: /data/datablobs.path: /data/blobs- grok exporter配置
定义的日志匹配模式
global: config_version: 2input: type: file path: /opt/example/examples.log readall: truegrok: patterns_dir: ./patternsmetrics: - type: counter name: grok_example_lines_total help: Counter metric example with labels. match: '%{DATE} %{TIME} %{USER:user} %{NUMBER}' labels: user: '{{.user}}'server: port: 9144- cratedb adapter 配置
暴露write 以及read 服务,因为使用集群模式，所以我 配置了多个节点
crate_endpoints:- host: "crate1" # Host to connect to (default: "localhost"). port: 5432 # Port to connect to (default: 5432). user: "crate" # Username to use (default: "crate") password: "" # Password to use (default: ""). schema: "" # Schema to use (default: ""). max_connections: 5 # The maximum number of concurrent connections (default: 5). enable_tls: false # Whether to connect using TLS (default: false). allow_insecure_tls: false # Whether to allow insecure / invalid TLS certificates (default: false).- host: "crate2" # Host to connect to (default: "localhost"). port: 5432 # Port to connect to (default: 5432). user: "crate" # Username to use (default: "crate") password: "" # Password to use (default: ""). schema: "" # Schema to use (default: ""). max_connections: 5 # The maximum number of concurrent connections (default: 5). enable_tls: false # Whether to connect using TLS (default: false). allow_insecure_tls: false # Whether to allow insecure / invalid TLS certificates (default: false).- host: "crate3" # Host to connect to (default: "localhost"). port: 5432 # Port to connect to (default: 5432). user: "crate" # Username to use (default: "crate") password: "" # Password to use (default: ""). schema: "" # Schema to use (default: ""). max_connections: 5 # The maximum number of concurrent connections (default: 5). enable_tls: false # Whether to connect using TLS (default: false). allow_insecure_tls: false # Whether to allow insecure / invalid TLS certificates (default: false).- metrics 的table
使用cratedb 我们需要先定义table，table 的schema 定义官方提供了模版
CREATE TABLE "metrics" ( "timestamp" TIMESTAMP, "labels_hash" STRING, "labels" OBJECT(DYNAMIC), "value" DOUBLE, "valueRaw" LONG, "day__generated" TIMESTAMP GENERATED ALWAYS AS date_trunc('day', "timestamp"), PRIMARY KEY ("timestamp", "labels_hash", "day__generated") ) PARTITIONED BY ("day__generated");启动&&测试
- 启动集群
docker-compose up -d- 效果
- 通过admin ui 创建table
- 写入数据统计
- 数据查询效果
说明
使用crate adapter 进行mtrics 数据的持久化存储也是一个不错的选择，以上演示没有包含关于grafana与prometheus 的集成，可以参考github
的完整配置自己添加下，这样就相对完整了,同时因为暴露了pg 协议的数据，我们可以直接通过grafanna 进行数据查看，展示。
参考资料
https://github.com/crate/crate_adapter
https://github.com/crate/crate
https://crate.io/docs/crate/reference/en/latest/config/cluster.html
https://github.com/fstab/grok_exporter
https://github.com/rongfengliang/prometheus-cratedb-cluster-docker-compose