Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

regex stanza bugfix #365

Merged
merged 11 commits into from
Apr 6, 2020
Merged

regex stanza bugfix #365

merged 11 commits into from
Apr 6, 2020

Conversation

jmeixensperger
Copy link
Contributor

@jmeixensperger jmeixensperger commented Mar 20, 2020

Addresses #364

@li-wu
Copy link
Contributor

li-wu commented Mar 23, 2020

Seems 12 test cases failed by this PR. Could you take a further check, Jack? Thanks.

@jmeixensperger
Copy link
Contributor Author

@li-wu @GordonWang I'm having issues getting the tests to run locally. Let's make sure they are working during the KT tonight.

li-wu
li-wu previously approved these changes Mar 26, 2020
@echo 'Start container with splunk'
docker-compose -f tests/large/provision/docker-compose.yml up &

sleep 120
@echo 'Provision splunk container'
docker-compose -f tests/large/provision/docker-compose.yml exec -T splunk sh -c 'cd /opt/splunk;./provision.sh;/opt/splunk/bin/splunk enable listen 9997 -auth admin:changeme;/opt/splunk/bin/splunk add index test_0;/opt/splunk/bin/splunk add index test_1;/opt/splunk/bin/splunk restart'
docker exec --user splunk provision_splunk_1 sh -c 'cd /opt/splunk;./provision.sh;./add_httpevent_collector.sh;/opt/splunk/bin/splunk enable listen 9997 -auth admin:changeme;/opt/splunk/bin/splunk add index test_0;/opt/splunk/bin/splunk add index test_1;/opt/splunk/bin/splunk restart'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use --no-cache option here for docker-comopse instead of introducing more steps.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That only works for the build command, so we'd still have to build the image either way (I assume this is the extra step you are referring to).

GordonWang
GordonWang previously approved these changes Apr 2, 2020
@jmeixensperger jmeixensperger dismissed stale reviews from GordonWang and li-wu via 1f49565 April 2, 2020 19:51
@li-wu li-wu self-requested a review April 3, 2020 03:41
@@ -2,7 +2,7 @@ version: "3.3"
services:
splunk:
hostname: eventgensplunk
build: .
image: provision_splunk:latest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since docker-compose is not used, I suppose docker-compose.yml is not used any more?

@@ -51,12 +51,15 @@ test_helper:
@echo 'Installing docker-compose'
bash install_docker_compose.sh

@echo 'Build a docker image'
docker build -t provision_splunk:latest -f tests/large/provision/Dockerfile tests/large/provision

@echo 'Start container with splunk'
docker-compose -f tests/large/provision/docker-compose.yml up &
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@li-wu the docker-compose is still used here

Copy link
Contributor

@GordonWang GordonWang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work

@li-wu li-wu merged commit e91d4b8 into develop Apr 6, 2020
jmeixensperger added a commit that referenced this pull request Apr 6, 2020
* Fix structlog dependency for app (#280)

* zipfile fix (#284)

* Fix bug 286 random token replacement (#287)

* Fix bug 286 random token replacement

* Change perdayvolume generator logic to get random token value replacement

* Versioning scheme (#278)

* [global] perDayVolume (#288)

* exclude global from perDayVolume assignment

* Address comment

* Fix security vulnerability issue (#289)

* Fix custom plugin stale docs (#290)

* Server fix (#293)

* Flag added

* server fix for count and env clean

* Fix bug 285 (#297)

* Add syslogAddHeader config directive (#296)

* Add syslog header to event in syslog mode

* timezone setting bugfix #249

* Using multiprocess pool to address the OOM issue (#301)

* Using multiprocess pool to address the OOM issue

* Fix test case fail

* Remove workerQueue unfinished tasks (#302)

* Bumped version to 6.5.2

* controller fix (#304)

* controller fix

* variable assignment fix (#306)

* add healthcheck endpoint and ping it every half an hour (#308)

* Fix security vulnerability issue reported by GitHub (#309)

* make controller also hit redis on healthcheck (#311)

* Fixed typo (#313)

* update redis version to use new functions (#318)

* Update tutorial related docs (#315)

* Revert oom fix change (#320)

* make circle ci fail when test case fails (#317)

* Add disable logging option to fix oom (#323)

* Add disable logging option to fix oom

* Fix modular input bug

* Fixing api server (#322)

* fixed multiple attributes

* debug

* Py3 upgrade (#291)

* init commit for python 3 migration

* migrate to py3 for test

* Fix import issue

* Fix test cases

* Fix jinja test case

* Fix multiprocess issue

* Fix a few warnings

* Fix test case fail

* Fix test case

* Fix cluster test cases

* Fix set_conf error

* Fix shebang

* Remove logutils lib from source

* Fix splunkstream import error

* Remove logutils from requirements.txt

* Fix build spl error

* Change code block indent

* Fix doc anchor issue

* Fix app running on Splunk 8.0 issue

* Fix xrange issue introduced by develop branch

* Fix merge bug

* Fix bug

* remove logutils

* Fix makefile bug

* Fix configparser bug

* Change filter function

* Bump version to 7.0.0

* Fix dateutil package bug

* cleanup memory (#326)

* cleanup memory

* addressing comment

* Fixed standalone healthcheck (#328)

* Add python2 libs required by perf test case (#331)

* make splunk_eventgen as a module (#332)

* Fix oom caused by ujson (#336)

* Fix oom caused by ujson

* Fix test env

* add release automation script (#335)

* add release automation script

* Fix process leak and start/stop 500 issue (#344)

* Fix process leak and start/stop 500 issue

* Fix test case fail

* Fix high system load issue (#345)

* Adding an option (#340)

* Adding an option

* Adding ujson2 for python3 as well

* minor improvements

* Scp plugin (#343)

* intermediate code for scp_plugin

* code clean up

* Added doc

* revert windbag

* Cap

* added test

* revert (#347)

* changing name (#349)

* Change (#350)

* change token type (#351)

* Add multithread support (#353)

* Add multithread support

* Fix typo

* Revert coverage version to 4.5.4 (#355)

* Fix CI failure cased by jinja new version and update the log config to reduce log in app (#358)

* Add missing documentation for negative floats, as introduced in PR#127 (#360)

* Bump nokogiri from 1.10.4 to 1.10.8 in /docs (#359)

Bumps [nokogiri](https://github.com/sparklemotion/nokogiri) from 1.10.4 to 1.10.8.
- [Release notes](https://github.com/sparklemotion/nokogiri/releases)
- [Changelog](https://github.com/sparklemotion/nokogiri/blob/master/CHANGELOG.md)
- [Commits](sparklemotion/nokogiri@v1.10.4...v1.10.8)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: Li Wu <[email protected]>

* Fix jinja plugin bug and test addon sample (#361)

* regex stanza bugfix (#365)

* prelim changes

* fix regex length matching

* fix wildcard sample + csv cases

* small csv fix, add test case

* revert accidental change

* Upgraded test instance

* add httpevent collector

* forgot to update stanza name

* add escape for csv matching + test case

* add sample, remove stanza

* push not working

Co-authored-by: tonyl <[email protected]>

* Improvement/release script (#368)

* add PR to master

* update title in sent PR

Co-authored-by: Li Wu <[email protected]>

* Correct python2/3 issue (#363)

* msg must be a bytes like object

* msg must be a bytes like object

Co-authored-by: Li Wu <[email protected]>

* update eventgen version to 7.1.0

Co-authored-by: Li Wu <[email protected]>
Co-authored-by: Tony Lee <[email protected]>
Co-authored-by: Patrik Nordlén <[email protected]>
Co-authored-by: Guodong Wang <[email protected]>
Co-authored-by: David Wang <[email protected]>
Co-authored-by: Guodong Wang <[email protected]>
Co-authored-by: Erwin Vrolijk <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Faircloth <[email protected]>
jmeixensperger added a commit that referenced this pull request Apr 15, 2020
* Fix structlog dependency for app (#280)

* zipfile fix (#284)

* Fix bug 286 random token replacement (#287)

* Fix bug 286 random token replacement

* Change perdayvolume generator logic to get random token value replacement

* Versioning scheme (#278)

* [global] perDayVolume (#288)

* exclude global from perDayVolume assignment

* Address comment

* Fix security vulnerability issue (#289)

* Fix custom plugin stale docs (#290)

* Server fix (#293)

* Flag added

* server fix for count and env clean

* Fix bug 285 (#297)

* Add syslogAddHeader config directive (#296)

* Add syslog header to event in syslog mode

* timezone setting bugfix #249

* Using multiprocess pool to address the OOM issue (#301)

* Using multiprocess pool to address the OOM issue

* Fix test case fail

* Remove workerQueue unfinished tasks (#302)

* Bumped version to 6.5.2

* controller fix (#304)

* controller fix

* variable assignment fix (#306)

* add healthcheck endpoint and ping it every half an hour (#308)

* Fix security vulnerability issue reported by GitHub (#309)

* make controller also hit redis on healthcheck (#311)

* Fixed typo (#313)

* update redis version to use new functions (#318)

* Update tutorial related docs (#315)

* Revert oom fix change (#320)

* make circle ci fail when test case fails (#317)

* Add disable logging option to fix oom (#323)

* Add disable logging option to fix oom

* Fix modular input bug

* Fixing api server (#322)

* fixed multiple attributes

* debug

* Py3 upgrade (#291)

* init commit for python 3 migration

* migrate to py3 for test

* Fix import issue

* Fix test cases

* Fix jinja test case

* Fix multiprocess issue

* Fix a few warnings

* Fix test case fail

* Fix test case

* Fix cluster test cases

* Fix set_conf error

* Fix shebang

* Remove logutils lib from source

* Fix splunkstream import error

* Remove logutils from requirements.txt

* Fix build spl error

* Change code block indent

* Fix doc anchor issue

* Fix app running on Splunk 8.0 issue

* Fix xrange issue introduced by develop branch

* Fix merge bug

* Fix bug

* remove logutils

* Fix makefile bug

* Fix configparser bug

* Change filter function

* Bump version to 7.0.0

* Fix dateutil package bug

* cleanup memory (#326)

* cleanup memory

* addressing comment

* Fixed standalone healthcheck (#328)

* Add python2 libs required by perf test case (#331)

* make splunk_eventgen as a module (#332)

* Fix oom caused by ujson (#336)

* Fix oom caused by ujson

* Fix test env

* add release automation script (#335)

* add release automation script

* Fix process leak and start/stop 500 issue (#344)

* Fix process leak and start/stop 500 issue

* Fix test case fail

* Fix high system load issue (#345)

* Adding an option (#340)

* Adding an option

* Adding ujson2 for python3 as well

* minor improvements

* Scp plugin (#343)

* intermediate code for scp_plugin

* code clean up

* Added doc

* revert windbag

* Cap

* added test

* revert (#347)

* changing name (#349)

* Change (#350)

* change token type (#351)

* Add multithread support (#353)

* Add multithread support

* Fix typo

* Revert coverage version to 4.5.4 (#355)

* Fix CI failure cased by jinja new version and update the log config to reduce log in app (#358)

* Add missing documentation for negative floats, as introduced in PR#127 (#360)

* Bump nokogiri from 1.10.4 to 1.10.8 in /docs (#359)

Bumps [nokogiri](https://github.com/sparklemotion/nokogiri) from 1.10.4 to 1.10.8.
- [Release notes](https://github.com/sparklemotion/nokogiri/releases)
- [Changelog](https://github.com/sparklemotion/nokogiri/blob/master/CHANGELOG.md)
- [Commits](sparklemotion/nokogiri@v1.10.4...v1.10.8)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: Li Wu <[email protected]>

* Fix jinja plugin bug and test addon sample (#361)

* regex stanza bugfix (#365)

* prelim changes

* fix regex length matching

* fix wildcard sample + csv cases

* small csv fix, add test case

* revert accidental change

* Upgraded test instance

* add httpevent collector

* forgot to update stanza name

* add escape for csv matching + test case

* add sample, remove stanza

* push not working

Co-authored-by: tonyl <[email protected]>

* Improvement/release script (#368)

* add PR to master

* update title in sent PR

Co-authored-by: Li Wu <[email protected]>

* Correct python2/3 issue (#363)

* msg must be a bytes like object

* msg must be a bytes like object

Co-authored-by: Li Wu <[email protected]>

* Release eventgen 7.1.0. Merge to develop branch. (#372)

* Updating Master (#312)

* Fix structlog dependency for app (#280)

* zipfile fix (#284)

* Fix bug 286 random token replacement (#287)

* Fix bug 286 random token replacement

* Change perdayvolume generator logic to get random token value replacement

* Versioning scheme (#278)

* [global] perDayVolume (#288)

* exclude global from perDayVolume assignment

* Address comment

* Fix security vulnerability issue (#289)

* Fix custom plugin stale docs (#290)

* Server fix (#293)

* Flag added

* server fix for count and env clean

* Fix bug 285 (#297)

* Add syslogAddHeader config directive (#296)

* Add syslog header to event in syslog mode

* timezone setting bugfix #249

* Using multiprocess pool to address the OOM issue (#301)

* Using multiprocess pool to address the OOM issue

* Fix test case fail

* Remove workerQueue unfinished tasks (#302)

* Bumped version to 6.5.2

* controller fix (#304)

* controller fix

* variable assignment fix (#306)

* add healthcheck endpoint and ping it every half an hour (#308)

* Fix security vulnerability issue reported by GitHub (#309)

* make controller also hit redis on healthcheck (#311)

* Update Master (#314)

* Fix structlog dependency for app (#280)

* zipfile fix (#284)

* Fix bug 286 random token replacement (#287)

* Fix bug 286 random token replacement

* Change perdayvolume generator logic to get random token value replacement

* Versioning scheme (#278)

* [global] perDayVolume (#288)

* exclude global from perDayVolume assignment

* Address comment

* Fix security vulnerability issue (#289)

* Fix custom plugin stale docs (#290)

* Server fix (#293)

* Flag added

* server fix for count and env clean

* Fix bug 285 (#297)

* Add syslogAddHeader config directive (#296)

* Add syslog header to event in syslog mode

* timezone setting bugfix #249

* Using multiprocess pool to address the OOM issue (#301)

* Using multiprocess pool to address the OOM issue

* Fix test case fail

* Remove workerQueue unfinished tasks (#302)

* Bumped version to 6.5.2

* controller fix (#304)

* controller fix

* variable assignment fix (#306)

* add healthcheck endpoint and ping it every half an hour (#308)

* Fix security vulnerability issue reported by GitHub (#309)

* make controller also hit redis on healthcheck (#311)

* Release 7.0.0 (#324)

* Fix structlog dependency for app (#280)

* zipfile fix (#284)

* Fix bug 286 random token replacement (#287)

* Fix bug 286 random token replacement

* Change perdayvolume generator logic to get random token value replacement

* Versioning scheme (#278)

* [global] perDayVolume (#288)

* exclude global from perDayVolume assignment

* Address comment

* Fix security vulnerability issue (#289)

* Fix custom plugin stale docs (#290)

* Server fix (#293)

* Flag added

* server fix for count and env clean

* Fix bug 285 (#297)

* Add syslogAddHeader config directive (#296)

* Add syslog header to event in syslog mode

* timezone setting bugfix #249

* Using multiprocess pool to address the OOM issue (#301)

* Using multiprocess pool to address the OOM issue

* Fix test case fail

* Remove workerQueue unfinished tasks (#302)

* Bumped version to 6.5.2

* controller fix (#304)

* controller fix

* variable assignment fix (#306)

* add healthcheck endpoint and ping it every half an hour (#308)

* Fix security vulnerability issue reported by GitHub (#309)

* make controller also hit redis on healthcheck (#311)

* Fixed typo (#313)

* update redis version to use new functions (#318)

* Update tutorial related docs (#315)

* Revert oom fix change (#320)

* make circle ci fail when test case fails (#317)

* Add disable logging option to fix oom (#323)

* Add disable logging option to fix oom

* Fix modular input bug

* Fixing api server (#322)

* fixed multiple attributes

* debug

* Py3 upgrade (#291)

* init commit for python 3 migration

* migrate to py3 for test

* Fix import issue

* Fix test cases

* Fix jinja test case

* Fix multiprocess issue

* Fix a few warnings

* Fix test case fail

* Fix test case

* Fix cluster test cases

* Fix set_conf error

* Fix shebang

* Remove logutils lib from source

* Fix splunkstream import error

* Remove logutils from requirements.txt

* Fix build spl error

* Change code block indent

* Fix doc anchor issue

* Fix app running on Splunk 8.0 issue

* Fix xrange issue introduced by develop branch

* Fix merge bug

* Fix bug

* remove logutils

* Fix makefile bug

* Fix configparser bug

* Change filter function

* Bump version to 7.0.0

* Release 7.0.0 (#325)

* Fix structlog dependency for app (#280)

* zipfile fix (#284)

* Fix bug 286 random token replacement (#287)

* Fix bug 286 random token replacement

* Change perdayvolume generator logic to get random token value replacement

* Versioning scheme (#278)

* [global] perDayVolume (#288)

* exclude global from perDayVolume assignment

* Address comment

* Fix security vulnerability issue (#289)

* Fix custom plugin stale docs (#290)

* Server fix (#293)

* Flag added

* server fix for count and env clean

* Fix bug 285 (#297)

* Add syslogAddHeader config directive (#296)

* Add syslog header to event in syslog mode

* timezone setting bugfix #249

* Using multiprocess pool to address the OOM issue (#301)

* Using multiprocess pool to address the OOM issue

* Fix test case fail

* Remove workerQueue unfinished tasks (#302)

* Bumped version to 6.5.2

* controller fix (#304)

* controller fix

* variable assignment fix (#306)

* add healthcheck endpoint and ping it every half an hour (#308)

* Fix security vulnerability issue reported by GitHub (#309)

* make controller also hit redis on healthcheck (#311)

* Fixed typo (#313)

* update redis version to use new functions (#318)

* Update tutorial related docs (#315)

* Revert oom fix change (#320)

* make circle ci fail when test case fails (#317)

* Add disable logging option to fix oom (#323)

* Add disable logging option to fix oom

* Fix modular input bug

* Fixing api server (#322)

* fixed multiple attributes

* debug

* Py3 upgrade (#291)

* init commit for python 3 migration

* migrate to py3 for test

* Fix import issue

* Fix test cases

* Fix jinja test case

* Fix multiprocess issue

* Fix a few warnings

* Fix test case fail

* Fix test case

* Fix cluster test cases

* Fix set_conf error

* Fix shebang

* Remove logutils lib from source

* Fix splunkstream import error

* Remove logutils from requirements.txt

* Fix build spl error

* Change code block indent

* Fix doc anchor issue

* Fix app running on Splunk 8.0 issue

* Fix xrange issue introduced by develop branch

* Fix merge bug

* Fix bug

* remove logutils

* Fix makefile bug

* Fix configparser bug

* Change filter function

* Bump version to 7.0.0

* Fix dateutil package bug

* update eventgen version to 7.1.0

Co-authored-by: Tony Lee <[email protected]>
Co-authored-by: Li Wu <[email protected]>

Co-authored-by: Li Wu <[email protected]>
Co-authored-by: Tony Lee <[email protected]>
Co-authored-by: Jack Meixensperger <[email protected]>
Co-authored-by: Patrik Nordlén <[email protected]>
Co-authored-by: David Wang <[email protected]>
Co-authored-by: Erwin Vrolijk <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Faircloth <[email protected]>
@li-wu li-wu deleted the bugfix/regex-stanza branch May 7, 2020 07:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants