Velcro for Elasticsearch

Create Elasticsearch indices, mappings and documents -- without any code!

📌 velcro strap currently deletes and re-creates indices and is not meant for production data management.

CLI

There is one subcommand: velcro strap, which will read index mappings from velcro.yaml and populate indices with data.

Getting started

Install globally

npm i -g @eighty4/velcro
velcro strap

Or npx it

npx @eighty4/velcro strap

Or add an npm script and npm run velcro it

{
  "scripts": {
    "velcro": "velcro strap"
  },
  "devDependencies": {
    "@eighty4/velcro": "0.0.6"
  }
}

Configuring Elasticsearch connection

The velcro cli uses the following args to configure its connection to Elasticsearch:

Location of velcro.yaml config

velcro strap --config-file path/to/velcro.yaml

Elasticsearch node address

velcro strap --node-address https://us-central-1.big-searchable-startup.com

Username and password authentication

velcro strap --use-basic-auth and set VELCRO_ES_USER and VELCRO_ES_PASSWORD environment variables

Bearer token authentication

velcro strap --use-token-auth and set VELCRO_ES_TOKEN environment variable

Api key authentication

velcro strap --use-api-key-auth and set VELCRO_ES_API_KEY environment variable

Skip TLS verification

velcro strap --skip-tls-verify

strap command

velcro strap will create all index mappings and documents configured in velcro.yaml

---
indices:
  my-index-name:
    properties:
      a_field_name: keyword
      another_field: text
  my-other-index:
    properties:
      created_when: date

Indexing documents with velcro strap

Documents to be bootstrapped are configured with the documents key.

---
indices:
  my-index-name:
    properties:
      a_field_name: keyword
      another_field: text

documents:
  all:
    my-index-name:
      - _id: foobar
        doc:
          a_field_name: MY_VERY_OWN_ID
          another_field: this doc is created with a specified id
      - a_field_name: GENERATED_DOC_ID
        another_field: this doc's id will be generated by elasticsearch
      - a_field_name: GENERATED_DOC_ID
        another_field: this structure will also create a doc with an elasticsearch generated id

Documents under documents.all will be created any time velcro strap is run.

Environment-specific documents

Creating documents for a specific environment, such as test, can be done by running velcro strap --env test and nesting documents under documents.test in velcro.yaml:

documents:
  test:
    my-index-name:
      - a_field_name: CREATED
        another_field: velcro strap --env test will create this doc
  prod:
    my-index-name:
      - a_field_name: SKIPPED
        another_field: velcro strap --env test will omit this doc
  all:
    my-index-name:
      - a_field_name: CREATED
        another_field: velcro strap will always create docs nested under all

Specifying a doc's _id

A document may include an id by specifying it with the _id key and nesting the document's fields under the doc key. The field id_source comments on the document's id origin.

---
indices:
  my-index-name:
    properties:
      id_source: keyword

documents:
  all:
    my-index-name:
      - _id: foobar
        doc:
          id_source: MY_VERY_OWN_ID
      - id_source: GENERATED_DOC_ID

Testing

createVelcroTestStrap

A test strap manages indices and mappings for execution of a test and can be created with createVelcroTestStrap. createVelcroTestStrap will generate unique names for each test's indices that can be removed from Elasticsearch at the end of a test with cleanup().

createVelcroTestStrap uses a config object to specify indices and docs, and it also reads velcro.yaml so indices can be referenced by name (this will error without my-index-name declared in a velcro.yaml):

import {createVelcroTestStrap} from 'velcro'

const velcro = await createVelcroTestStrap({
    indices: ['my-index-name'],
    documents: [
        {
            a_field_name: "value",
            another_field: "text",
        }
    ]
})

// do some testing

await velcro.cleanup()

Indices can also be configured from test code:

const velcro = await createVelcroTestStrap({
    indices: [{
        name: 'my-index-name',
        mappings: {
            a_field_name: 'text',
            another_field: 'text',
        }
    }],
    documents: [
        {
            a_field_name: "value",
            another_field: "text",
        }
    ]
})

📌 createVelcroTestStrap({configPath}) can be used to read a velcro.yaml from a relative path.

📌 Documents will not be created from velcro.yaml for tests.

VelcroTestStrap

createVelcroTestStrap returns VelcroTestStrap which includes APIs to assist with Elasticsearch application testing.

managedIndexName(indexName: string): string

Returns the generated index name for the given index mapping.

documentIds(indexName: string): Array<string>

Returns the index's created documents' ids.

documentId(indexName: string, index: number): string

Returns the doc id of a given index at the specified array index from the createVelcroTestStrap({documents}) array.

deleteDocuments()

Delete all managed documents.

deleteIndices()

Delete all managed indices.

cleanup()

Delete all managed indices and close Elasticsearch client.