PDF-STORAGE

PDF storage is designed to be a universally usable PDF generator from HTML templates.

Key aspects:

• The consumer of the API doesn't have to care how PDFs, templates and so on are persisted - the storage will handle that part.
• A simple API which takes HTML and templating data and responds with a final PDF file URL.
• Supports merging PDFs as one.
• Can handle very large amounts of generated PDFs and prioritize files which require a faster output (files which are currently opened if the queue is long) before everything else.
• Supports AWS, Azure and Google services and data stores.

For further details, see the API description document.

Testing locally in docker

Easiest way to run and test application is to start in with docker

docker run -it -p 5000:5000 -e ASPNETCORE_ENVIRONMENT=Development ptcos/pdf-storage

Navigate to http://localhost:5000/doc

Running locally for development

Install .NET core SDK.

dotnet run --environment=Development

The default development setup mocks all external dependencies.

Navigate to http://localhost:5000/doc/.

The following headers must be included in API calls:

Authorization: ApiKey apikeyfortesting
Content-Type: application/json-patch+json

Local development with mocks enabled

The default Development environment setup enables all mocks in appsettings.Development.json. This way the service should start and function correctly without any external dependencies.

{
  "DbType": "inMemory",
  "MqType": "inMemory",
  "PdfStorageType": "inMemory"
}

These can be overwritten with environment variables: PdfStorageType = "inMemory" etc.

On Linux (Debian) set development, install pdftk and chromium.

sudo apt-get -y install pdftk chromium

On Windows, Chrome is required.

Run a local development database

PostgreSQL:

docker run --name pdf-storage-postgress -e POSTGRES_PASSWORD=passwordfortesting -it -p 5432:5432 postgres

dotnet run --environment=development ConnectionString='User ID=postgres;Password=passwordfortesting;Host=localhost;Port=5432;Database=pdfstorage;Pooling=true;' DbType=postreSql

SqlServer:

docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=testpassword1#!' -p 1433:1433 --name sql1 -d mcr.microsoft.com/mssql/server:2017-latest

dotnet run --environment=development ConnectionString='Server=localhost,1433;Database=pdf-storage;User=sa;Password=testpassword1#!' DbType=sqlServer

Hangfire dashboard

Dashboard allows traffic only from localhost on production builds.

For this reason, port forwarding to the running container is required (in Kubernetes).

kubectl -n pdf-storage port-forward pdf-storage-master-4075827073-5h77r 5000

Navigate (via forwarded port) to http://localhost:5000/hangfire.

Other technologies

The localhost requirement applies, but access methods may vary.

PDF stores

Google bucket

PDF storage supports Google bucket for saving PDF binaries.

Mount a valid service account file, configure its path and Google configurations in appconfig or environment variables.

{
  "PdfStorageType": "googleBucket",
  "GoogleCloud": {
    "GoogleBucketName": "pdf-storage-master",
    "GoogleAuthFile": "/path/to/key/google.key.json"
  }
}

Example (not valid) service account file, see Google service accounts for further information.

{
  "type": "service_account",
  "project_id": "some-project",
  "private_key_id": "1234",
  "private_key": "-----BEGIN PRIVATE KEY-----\nKEY_SHOULD_BE_HERE-----END PRIVATE KEY-----\n",
  "client_email": "[email protected]",
  "client_id": "1234",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://accounts.google.com/o/oauth2/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/pdf-storage-master%40some-project.some-project.com"
}

AWS S3

Configure application to use AWS S3 store.

{
  "PdfStorageType": "awsS3",
  "AwsS3": {
    "AwsS3BucketName": "pdf-storage-master",
    "AccessKey": "thisisaccesskey",
    "SecretKey": "ThisIsSecretKey",
    "AwsServiceURL": "http://localhost:9000",
    "AwsRegion": "EUCentral1"
  }
}

Azure storage

PDF storage supports Azure storage accounts as storage.

{
  "PdfStorageType": "azureStorage",
  "AzureStorage": {
    "StorageConnectionString": "DefaultEndpointsProtocol=https;AccountName=[your_account];AccountKey=[your_key];EndpointSuffix=core.windows.net",
    "ContainerName": "pdf-storage"
  }
}

Local storage

Use local disk as storage.

{
  "PdfStorageType": "local",
  "LocalStorage": {
    "Folder": "/tmp"
  }
}

BaseUrl and PathBase configuration parameters

BaseUrl parameter needs to be set to that domain+path where this is hosted. Eg. if it is hosted at https://pdf-storage.example.com then BaseUrl needs to be set as "https://pdf-storage.example.com"

If you want to deploy this application to another than root path, eg. https://example.com/pdf-storage then PathBase needs to be set to "/pdf-storage" and BaseUrl "https://example.com/pdf-storage"

Migrations

Tooling requires dotnet-ef available, so run:

dotnet tool install -g dotnet-ef

There is a special script for migrations since multiple database engines are supported.

./AddOrRemoveMigrations.ps1 -MigrationName "DescriptionForMigration"

Removing the latest migration

./AddOrRemoveMigrations.ps1 -Operation Remove -MigrationName "DescriptionForMigration"

Insights

If application insights is enabled, additional metrics will be added with notation pdf-storage-.

Examples:

pdf-storage-hangfire-enqueued
pdf-storage-hangfire-scheduled
pdf-storage-hangfire-failed
pdf-storage-hangfire-processing
pdf-storage-hangfire-servers

License

The MIT License (MIT)