Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance rippled unit tests speed #3432

Closed
carlhua opened this issue Jun 3, 2020 · 2 comments
Closed

Enhance rippled unit tests speed #3432

carlhua opened this issue Jun 3, 2020 · 2 comments
Labels
CI Continuous Integration Functionality Food for Thought Ideas, moonshots, etc. Stuff to think about broadly. Not necessarily directly actionable. Good First Issue Great issue for a new contributor Potential Bounty Idea RIPD Export Exported from legacy JIRA issue tracking Tech Debt Non-urgent improvements

Comments

@carlhua
Copy link
Contributor

carlhua commented Jun 3, 2020

Summary

The existing rippled unit tests take substantial amount of time to complete, consider enhancing the speed of rippled. One of the issues here is that the signalStop call that causes ApplicationImp to signal and waits for its children to stop before it, itself, stops.

Exported from RIPD-1384
Originally reported for version: 1.5.0

@carlhua carlhua added Tech Debt Non-urgent improvements RIPD Export Exported from legacy JIRA issue tracking labels Jun 3, 2020
@intelliot intelliot added Good First Issue Great issue for a new contributor CI Continuous Integration Functionality Food for Thought Ideas, moonshots, etc. Stuff to think about broadly. Not necessarily directly actionable. Potential Bounty Idea labels Apr 6, 2023
@intelliot
Copy link
Collaborator

  • currently parallelized by suite [separate processes] but individual test functions are not, even though they are mostly independent and could be run in parallel
    • would be an easy way to substantially speed up unit test runs
    • but needs to be looked into - might not work and would require restructuring
  • bounty would be good for this
    • bounty hunter could imagine better solutions
  • another option: split test classes into multiple classes and make them separate suites
    • have done this for TxQ - there are 2 TxQ test classes
    • issue: when changing a subsystem you want to run that subsystem's tests - but it would then be easy to forget to run all the relevant tests
      • workaround: make it clear which tests are relevant? wildcards in test names?
  • some tests run repeatedly in order to exercise amendments
    • same test running repeatedly under slightly different conditions
    • some of these could be removed, e.g. for ones that have been enabled for a while
    • could go along with retiring amendments that have been enabled more than 2 yrs?
  • no one loves our unit test frameworks and there are some good ones out there
    • but would be a very heavy lift to replace it
  • move some out to libxrpl and write new executables
    • could integrate with other unit test frameworks that have useful features
    • parallelize test processes

related: why flaky tests?

  • hypothesis: tem malformed - returned by env when it gets an ill-formed response from RPC; may be due to a timeout on the RPC
    • would make sense that underpowered CI machine would have timeouts on RPC responses
    • can we make the timeout longer?
      • maybe in asio code?

@intelliot intelliot changed the title Enhance rippled unit tests speed (Version: 1.5.0) Enhance rippled unit tests speed May 1, 2023
@intelliot
Copy link
Collaborator

#4556

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous Integration Functionality Food for Thought Ideas, moonshots, etc. Stuff to think about broadly. Not necessarily directly actionable. Good First Issue Great issue for a new contributor Potential Bounty Idea RIPD Export Exported from legacy JIRA issue tracking Tech Debt Non-urgent improvements
Projects
None yet
Development

No branches or pull requests

2 participants