Skip to content
This repository has been archived by the owner on Sep 28, 2022. It is now read-only.

Batch ingest #253

Merged
merged 26 commits into from
Oct 11, 2019
Merged
Changes from 1 commit
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
a3d15a2
WIP: proof of concept for batch based ingest
jaffee Aug 20, 2019
62bc87f
support for string record IDs, parity with old ingest
jaffee Aug 21, 2019
edf6cb8
implement int field support
jaffee Aug 26, 2019
3146a31
add many config options to picsv, test, benchmark
jaffee Aug 27, 2019
802f882
support for empty values in importbatch
jaffee Aug 27, 2019
9634954
documentation for batch importer
jaffee Aug 27, 2019
2fa57bd
gofmt -s importbatch.go
jaffee Aug 27, 2019
eed88d3
add locks on translator cache, shardNodes invalidation
jaffee Aug 27, 2019
4cc60a1
gofmt -s
jaffee Aug 28, 2019
4d7a11e
move Translator out of client to ImportBatch
jaffee Aug 28, 2019
c33123a
interfacify Translator, add errors and batch addition
jaffee Aug 28, 2019
6d88b17
export client KeyTranslation and ImportRoaring methods
jaffee Aug 28, 2019
994cf36
move batch stuff to subpackage
jaffee Aug 28, 2019
a930004
update circle CI to Go 1.12/13-rc
jaffee Aug 28, 2019
8e2f334
use shardnode cluster change detector w/ cleanup
jaffee Aug 28, 2019
c2b019c
remove picsv command (moving to pdk)
jaffee Aug 31, 2019
b811c3b
start adding time quantum support to batch import
jaffee Sep 9, 2019
e40d84a
complete per-record timestamp support
jaffee Sep 9, 2019
2dfc4b7
test and fix clearValues bug
jaffee Sep 10, 2019
bfe8680
handle byte slice batch record IDs
jaffee Oct 4, 2019
9b81c1a
fix and comment for importvalues
jaffee Oct 4, 2019
4129aee
support for string slice values in batch ingest
jaffee Oct 8, 2019
6791c14
fix importbatch bug, wrap some errs
jaffee Oct 8, 2019
1d062fd
support multiple of the same field in batch
jaffee Oct 9, 2019
58c2816
map from int instead of slice
jaffee Oct 9, 2019
d00044f
fix flaky test due to unpredictable translation order
jaffee Oct 11, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update circle CI to Go 1.12/13-rc
jaffee committed Aug 28, 2019
commit a93000430cbb5b26a9d7ee22333f1c4cd2b843a3
14 changes: 7 additions & 7 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -2,7 +2,7 @@ version: 2
defaults: &defaults
working_directory: /go/src/github.com/pilosa/go-pilosa
docker:
- image: circleci/golang:1.11
- image: circleci/golang:1.12
environment:
GO111MODULE: "on"
fast-checkout: &fast-checkout
@@ -30,18 +30,18 @@ jobs:
- *fast-checkout
- run: make install-gometalinter
- run: make gometalinter
test-golang-1.12-rc: &base-test
test-golang-1.13-rc: &base-test
<<: *defaults
steps:
- *fast-checkout
- run: make test-all
docker:
- image: circleci/golang:1.12-rc
- image: circleci/golang:1.13-rc
- image: pilosa/pilosa:master
test-golang-1.11:
test-golang-1.12:
<<: *base-test
docker:
- image: circleci/golang:1.11
- image: circleci/golang:1.12
- image: pilosa/pilosa:master
workflows:
version: 2
@@ -51,9 +51,9 @@ workflows:
- linter:
requires:
- build
- test-golang-1.12-rc:
- test-golang-1.13-rc:
requires:
- build
- test-golang-1.11:
- test-golang-1.12:
requires:
- build