Skip to content

Latest commit

 

History

History
81 lines (70 loc) · 2.86 KB

README.md

File metadata and controls

81 lines (70 loc) · 2.86 KB
Raw

timescaleproxy

A proxy for writing Telegraf JSON HTTP outputs into a TimescaleDB. Written in Go. Feedback is welcome. Bear in mind this is my first Go-project.

As the Telegraf plugin for TimescaleDB is pending pending for being included in the Telegraf codebase. I saw the need for a workaround using a HTTP-proxy that recieves metrics in JSON and writes them to the TimescaleDB.

It's inspired by the plugin but instead of a general PostgreSQL output it outputs only to TimescaleDB.

Security is non-existant at the moment so this should only be used far away from the internetz. I'd recommend either behind a secured nginx reverse proxy or over a wireguard interface.

Install

git clone https://github.com/dafvid/timescaleproxy.git
cd timescaleproxy
go install
go build

Config

./timescaleproxy -printconf > config.json

Db

  • Db.Host: PostgreSQL server host
  • Db.Port: PostgreSQL port
  • Db.Schema: Timescale database schema
  • Db.Database: Timescale database name
  • Db.User: Timescale database user
  • Db.Password: Timescale database user password
  • Db.MaxConns: MaxConns setting for pgxpool
  • Db.MinConns: MinConns setting for pgxpool

Listen

  • Listen.adress: Timescale proxy listening adress
  • Listen.port: Timescale proxy listening port

Misc

  • TimestampUnit: matches the setting json_timestamp_units in section [[outputs.http]] in telegraf.conf
  • DefaultDropPolicy: Interval setting for add_retention_policy(). Clear to disable retention.
  • LogLevel: Log level. Clear to disable logs.

Run

./timescaleproxy -c config.json

Telegraf config

[[outputs.http]]
  url = "http://url.to.server:8432/"
  data_format = "json"
  json_timestamp_units = "1ms"
  [outputs.http.headers]
    Content-Type = "application/json; charset=utf-8"

Updates

2022-10-07

  • Added Config documentation

2022-08-10

  • Added FreeBSD RC-script to run the proxy as a daemon

2022-06-22

  • Changed the timestamp column name to ts

2020-07-31:

  • Export and read JSON-config
  • Create tables and tags in DB from first Metric
  • Writes JSON-metrics to TimescaleDB

TODO

  • Per measurement config for retention
  • config
  • create tables from first measurement
  • write measurement to db
  • tags as FK
  • schema config
  • handle influx line protocol (less portable maybe)
  • measurement config (column type for field)
  • sanitize strings
  • default retention policy
  • error handling
  • update table if measurement changes (unlikely)