Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update run_cass_and_test.sh script to setup cassandra schemas #5628

Conversation

taylanisikdemir
Copy link
Member

@taylanisikdemir taylanisikdemir commented Jan 25, 2024

What changed?
run_cass_and_test.sh is creating a cassandra container and then running tests with CASSANDRA=1 env var.
This wouldn't work if the cassandra db is not setup properly. e.g. creating keyspaces.
It was working for me locally because cassandra container was using previously populated data folder. Fresh installations or docker cleanup cause the script to not work.

So updating the script to also run cadence:master-auto-setup image which does db prep on startup.

Note: I looked into using docker/start.sh instead of running the cadence container but didn't choose that path. It requires cqlsh to be installed on local machine. I have correct version of cqlsh but it complains about some python packages missing. Better to just rely on already working setup on cadence:master-auto-setup.

Follow up note: Some tests invoke cqlsh locally so not having a proper local cqlsh setup causes those to fail. Will come back to that issue because it's annoying. Might also ditch this script and fix the docker-compose based test invocation setup.

@coveralls
Copy link

coveralls commented Jan 25, 2024

Pull Request Test Coverage Report for Build 018d4834-cc72-4295-8bbf-e495c1920cda

  • 0 of 1 (100.0%) changed or added relevant line in 1 file are covered.
  • 59 unchanged lines in 10 files lost coverage.
  • Overall coverage increased (+0.05%) to 62.702%

Files with Coverage Reduction New Missed Lines %
common/task/weighted_round_robin_task_scheduler.go 1 88.06%
common/task/parallel_task_processor.go 2 93.06%
service/history/execution/mutable_state_builder.go 2 68.64%
service/matching/matcher.go 2 90.72%
tools/cli/admin_db_decode_thrift.go 2 70.13%
service/history/queue/timer_gate.go 3 95.83%
service/history/task/fetcher.go 4 85.05%
service/history/task/transfer_standby_task_executor.go 6 86.6%
service/history/execution/mutable_state_task_refresher.go 17 56.65%
service/history/task/task_util.go 20 70.57%
Totals Coverage Status
Change from base Build 018d4778-79f6-4879-81e3-2321ec8f121a: 0.05%
Covered Lines: 92273
Relevant Lines: 147161

💛 - Coveralls

@davidporter-id-au
Copy link
Member

Just be careful that this script doesn't propagate error codes properly (it needs set -e), it'll pass even if there's an error. if it was for anything other local development that'd probably not be ok

Copy link
Member

@Shaddoll Shaddoll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know that you can run tests locally with these docker compose?
https://github.com/uber/cadence/tree/master/docker/buildkite

@taylanisikdemir
Copy link
Member Author

Do you know that you can run tests locally with these docker compose? https://github.com/uber/cadence/tree/master/docker/buildkite

@Shaddoll hmm missed that README file. tried it and fails during setup unzipping protoc. Are you using these commands successfully?

 docker-compose -f docker/buildkite/docker-compose-local.yml run integration-test-cassandra

@taylanisikdemir
Copy link
Member Author

Just be careful that this script doesn't propagate error codes properly (it needs set -e), it'll pass even if there's an error. if it was for anything other local development that'd probably not be ok

set -eo pipefail is already there.

@taylanisikdemir taylanisikdemir force-pushed the taylan/improve_run_cass_test_script branch from 07cfd8c to 87f39bc Compare January 26, 2024 23:57
@taylanisikdemir taylanisikdemir enabled auto-merge (squash) January 27, 2024 00:00
@taylanisikdemir taylanisikdemir changed the title Fix run_cass_and_test.sh script Update run_cass_and_test.sh script to setup cassandra schemas Jan 27, 2024
@taylanisikdemir taylanisikdemir enabled auto-merge (squash) January 27, 2024 00:01
@taylanisikdemir taylanisikdemir merged commit aca05e4 into cadence-workflow:master Jan 27, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants