AppInsights on-premises

What's this?

This repository contains a service that is API compatible with Azure AppInsights but that stores all telemetry in a local persistence layer. There are two main intended use-cases for this service:

1. Enable developers to use AppInsights client SDKs but keep all data on-premises, for example to enable building applications that work in an Azure cloud environment but also in disconnected deployment scenarios.

2. Enable developers to write integration tests for their telemetry layer and write assertions against telemetry generated during their test runs, e.g. assert that no exceptions were logged, assert that a certain number of custom events were generated, etc.

The service has been tested with the following AppInsights client SDKs:

• Python
• NodeJS
• Javascript

The service currently supports the following persistence layers:

• Relational storage in PostgreSQL
• Object storage systems via Apache Libcloud including Azure Storage, Azurite, Azure IoT Edge Storage, etc.

Setup

Development setup

To run the service locally, execute the following commands:

# select the data storage backend
export BACKEND=postgres

# run the database and appinsights server
make build start

# send sample telemetry to the appinsights server
make tests

Production setup

For production deployments, the service can be run via Docker. Separate Docker images are published for each of the persistence layers:

# PostgreSQL persistence layer
docker run \
  -p 8000:8000 \
  -e APPINSIGHTS_INSTRUMENTATIONKEY="553161ed-0c6b-41a8-973e-77a411391be5" \
  -e DATABASE_URL="postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:5432/${POSTGRES_DB}" \
  cwolff/appinsights-on-premises:0.3.0-postgres

# Apache Libcloud persistence layer targeting Azurite
docker run \
  -p 8000:8000 \
  -e APPINSIGHTS_INSTRUMENTATIONKEY="553161ed-0c6b-41a8-973e-77a411391be5" \
  -e DATABASE_URL="libcloud://${AZURITE_ACCOUNT}:${AZURITE_SECRET}@azure_blobs?endpoint=${AZURITE_HOST}:10000&ssl=False" \
  docker pull cwolff/appinsights-on-premises:0.3.0-libcloud

As illustrated in the examples above, the Docker containers must be configured with two required environment variables:

1. DATABASE_URL points the service to the desired persistence layer.
2. APPINSIGHTS_INSTRUMENTATIONKEY is a comma separated list of the GUID client identifiers that are permitted to publish telemetry to the service. Telemetry sent from clients who don't present one of these identifiers is silently dropped by the service.

Integration with client code

To integrate an application that uses the AppInsights client SDK with the service, the only necessary change is to point the client SDK to the service's telemetry endpoint. For example, when using the Python AppInsights SDK:

from applicationinsights import TelemetryClient
from applicationinsights.channel import AsynchronousQueue, AsynchronousSender, TelemetryChannel

# define the endpoint of the service and an instrumentation key registered with the service
endpoint = 'http://localhost:8000/'
ikey = '553161ed-0c6b-41a8-973e-77a411391be5'

# point the telemetry client to the custom endpoint
client = TelemetryClient(ikey, TelemetryChannel(queue=AsynchronousQueue(AsynchronousSender(endpoint))))

# now use the telemetry client as normal, e.g.:
client.track_event('my_event', {'some_property': 'a value'})

To inspect the telemetry generated by the client code, use a tool appropriate to the persistence layer that the service is configured to use, e.g. DBeaver to view telemetry stored in PostgreSQL or Azure Storage Explorer to view telemetry stored in Azurite.

[ ~ Dependencies scanned by PyUp.io ~ ]