Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added a new 'Run all' button to interactive toolbar on the web interface #981

Merged
merged 14 commits into from
Aug 24, 2023
+448 −14
Merged

Added a new 'Run all' button to interactive toolbar on the web interface #981

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
c7a9ad0
Added a new 'Run all' button to Toolbar on the web interface
rnemes Aug 10, 2023
227d910
Added newsfragment
rnemes Aug 10, 2023
caa2b7b
Update doc/newsfragments/2404_new.run_all_button.rst
rnemes Aug 10, 2023
abc5495
Minor changes
rnemes Aug 11, 2023
6dd0acc
Merge branch 'main' into RunAllInteractive
rnemes Aug 15, 2023
f4cf239
Plan-level Run button now supports filtering
rnemes Aug 23, 2023
691fe4d
Merge branch 'RunAllInteractive' of https://github.com/rnemes/testpla…
rnemes Aug 23, 2023
3c11876
Resolved conflict
rnemes Aug 23, 2023
565b87b
Merge branch 'main' into RunAllInteractive
rnemes Aug 23, 2023
1fe3225
Updated unit test
rnemes Aug 23, 2023
fe22198
Updated unit test
rnemes Aug 23, 2023
52c2fcb
Updated unit test
rnemes Aug 23, 2023
8e22ac1
Updated unit test
rnemes Aug 23, 2023
63f7103
Addressed reviewer request
rnemes Aug 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/newsfragments/2404_new.run_all_button.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Introduced a new plan-level Run button to the toolbar on the interactive GUI.
23 changes: 17 additions & 6 deletions testplan/runnable/interactive/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -227,24 +227,35 @@ def reset_test(self, test_uid, await_results=True):
self._update_reports([(self.test(test_uid).dry_run().report, [])])

def run_all_tests(
self, await_results: bool = True
self,
shallow_report: Optional[Dict] = None,
await_results: bool = True,
) -> Union[TestReport, Awaitable]:
"""
Runs all tests.

:param shallow_report: shallow report entry, optional
:param await_results: Whether to block until tests are finished,
defaults to True.
:return: If await_results is True, returns a testplan report.
Otherwise, returns a future which will yield a testplan report when
ready.
"""
if not await_results:
return self._run_async(self.run_all_tests)

self.logger.debug("Interactive mode: Run all tests")
return self._run_async(
self.run_all_tests, shallow_report=shallow_report
)

for test_uid in self.all_tests():
self.run_test(test_uid)
if shallow_report:
self.logger.debug("Interactive mode: Run filtered tests")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Loglines are a welcome addition, but I do not see it for the others. Also I wonder if it is INFO level already or perhaps USER_INFO.

for multitest in shallow_report["entries"]:
self.run_test(
test_uid=multitest["name"], shallow_report=multitest
)
else:
self.logger.debug("Interactive mode: Run all tests")
for test_uid in self.all_tests():
self.run_test(test_uid=test_uid)

def run_test(
self,
Expand Down
15 changes: 13 additions & 2 deletions testplan/runnable/interactive/http.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,8 +153,19 @@ def put(self):
new_runtime_status,
):
_check_execution_order(ihandler.report)
ihandler.report.runtime_status = RuntimeStatus.WAITING
ihandler.run_all_tests(await_results=False)
filtered = "entries" in shallow_report
if filtered:
entries = _extract_entries(shallow_report)
ihandler.report.set_runtime_status_filtered(
RuntimeStatus.WAITING,
entries,
)
else:
ihandler.report.runtime_status = RuntimeStatus.WAITING
ihandler.run_all_tests(
shallow_report=shallow_report if filtered else None,
await_results=False,
)

return _serialize_report_entry(ihandler.report)

Expand Down
43 changes: 40 additions & 3 deletions testplan/web_ui/testing/src/Report/InteractiveReport.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import {
ReloadButton,
ResetButton,
AbortButton,
RunAllButton,
SaveButton,
} from "../Toolbar/InteractiveButtons";
import NavBreadcrumbs from "../Nav/NavBreadcrumbs";
Expand Down Expand Up @@ -63,6 +64,7 @@ class InteractiveReportComponent extends BaseReport {
this.resetAssertionStatus = this.resetAssertionStatus.bind(this);
this.resetReport = this.resetReport.bind(this);
this.abortTestplan = this.abortTestplan.bind(this);
this.runAll = this.runAll.bind(this);
this.reloadCode = this.reloadCode.bind(this);
this.envCtrlCallback = this.envCtrlCallback.bind(this);
this.handleClick = this.handleClick.bind(this);
Expand All @@ -72,6 +74,7 @@ class InteractiveReportComponent extends BaseReport {
navWidth: `${INTERACTIVE_COL_WIDTH}em`,
resetting: false,
reloading: false,
running: false,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I looked at this new status and wonder two things:

  • Can we do without it?
  • If not, then we need to block certain actions when it is True. Right now I think we can trigger it twice without problem, send a reload during it, reset, etc... See for instance, how the reloadCode in InteractiveReport is handling it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we do without it?

We only need that for logging the state of the progress. However, we could use another variable eg.: runtime_status which could replace resetting, reloading, aborting as well, but in that case need to refactor the related functionalities.

If not, then we need to block certain actions when it is True. Right now I think we can trigger it twice without problem, send a reload during it, reset, etc... See for instance, how the reloadCode in InteractiveReport is handling it.

That makes sense. I will update the code.

aborting: false,
assertionStatus: defaultAssertionStatus,
};
Expand Down Expand Up @@ -122,9 +125,12 @@ class InteractiveReportComponent extends BaseReport {
response.data.runtime_status === "finished" ||
response.data.runtime_status === "not_run"
) {
if (this.state.resetting){
if (this.state.resetting) {
this.setState({ resetting: false });
}
if (this.state.running) {
this.setState({ running: false });
}
}
if (
!this.state.report ||
Expand Down Expand Up @@ -503,11 +509,33 @@ class InteractiveReportComponent extends BaseReport {
return shallowEntry;
}

/**
* Send request of start all tests to server.
*/
runAll() {
if (
this.state.resetting || this.state.reloading ||
this.state.aborting || this.state.running
) {
return;
} else {
const updatedReportEntry = {
...this.shallowReportEntry(this.state.filteredReport.report),
runtime_status: "running",
};
this.putUpdatedReportEntry(updatedReportEntry);
this.setState({ running: true });
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once the fixes for the active filter issues are in, we probably need to see if it is working with a filter expression. If not, we will need to extend support, I do not think that piece of the API is handling the filtering.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it doesn't handle filtering, it starts everything.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned offline, this will be needed.


/**
* Reset the report state to "resetting" and request the change to server.
*/
resetReport() {
if (this.state.resetting || this.state.reloading || this.state.aborting) {
if (
this.state.resetting || this.state.reloading ||
this.state.aborting || this.state.running
) {
return;
} else {
const updatedReportEntry = {
Expand All @@ -523,7 +551,10 @@ class InteractiveReportComponent extends BaseReport {
* Send request of reloading report to server.
*/
reloadCode() {
if (this.state.resetting || this.state.reloading || this.state.aborting) {
if (
this.state.resetting || this.state.reloading ||
this.state.aborting || this.state.running
) {
return;
}
let currentTime = new Date();
Expand Down Expand Up @@ -670,6 +701,12 @@ class InteractiveReportComponent extends BaseReport {
updateEmptyDisplayFunc={noop}
updateTagsDisplayFunc={noop}
extraButtons={[
<RunAllButton
key="runall-button"
running={this.state.running}
runAllCbk={this.runAll}
filter={this.state.filteredReport.filter.text}
/>,
<ReloadButton
key="reload-button"
reloading={this.state.reloading}
Expand Down
89 changes: 89 additions & 0 deletions testplan/web_ui/testing/src/Report/__tests__/InteractiveReport.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -683,4 +683,93 @@ describe("InteractiveReport", () => {
});
});
});

it("Run all tests", (done) => {
const interactiveReport = renderInteractiveReport();

const report = initialReport();
const multitest = report.entries[0];
expect(multitest.category).toBe("multitest");
multitest.env_status = "STARTED";

const testcase = report.entries[0].entries[0].entries[0];
expect(testcase.category).toBe("testcase");

interactiveReport.setState({
filteredReport: {
report: report,
filter: {text: null}
},
});
interactiveReport.update();
interactiveReport.instance().runAll();
moxios.wait(() => {
const request = moxios.requests.mostRecent();
expect(request.url).toBe("/api/v1/interactive/report");
expect(request.config.method).toBe("put");
const putData = JSON.parse(request.config.data);

request
.respondWith({
status: 200,
response: putData,
})
.then(() => {
moxios.wait(() => {
const request = moxios.requests.mostRecent();
expect(request.url).toBe("/api/v1/interactive/report");
expect(request.config.method).toBe("put");
const putData = JSON.parse(request.config.data);
expect(putData.runtime_status).toBe("running");
expect(putData.entries).not.toBeDefined();
done();
});
});
});
});

it("Run filtered tests", (done) => {
const interactiveReport = renderInteractiveReport();

const report = initialReport();
const multitest = report.entries[0];
expect(multitest.category).toBe("multitest");
multitest.env_status = "STARTED";

const testcase = report.entries[0].entries[0].entries[0];
expect(testcase.category).toBe("testcase");

interactiveReport.setState({
filteredReport: {
report: report,
filter: {text: "something"}
},
});
interactiveReport.update();
interactiveReport.instance().runAll();
moxios.wait(() => {
const request = moxios.requests.mostRecent();
expect(request.url).toBe("/api/v1/interactive/report");
expect(request.config.method).toBe("put");
const putData = JSON.parse(request.config.data);

request
.respondWith({
status: 200,
response: putData,
})
.then(() => {
moxios.wait(() => {
const request = moxios.requests.mostRecent();
expect(request.url).toBe("/api/v1/interactive/report");
expect(request.config.method).toBe("put");
const putData = JSON.parse(request.config.data);
expect(putData.runtime_status).toBe("running");
expect(putData.entries).toHaveLength(1);
expect(putData.entries[0].name).toBe("MultiTestName");
done();
});
});
});
});
});
35 changes: 35 additions & 0 deletions testplan/web_ui/testing/src/Report/__tests__/__snapshots__/InteractiveReport.test.js.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ exports[`InteractiveReport Handles environment being started 1`] = `
expandStatus="default"
extraButtons={
Array [
<RunAllButton
filter={null}
runAllCbk={[Function]}
running={false}
/>,
<ReloadButton
reloadCbk={[Function]}
reloading={false}
Expand Down Expand Up @@ -92,6 +97,11 @@ exports[`InteractiveReport Parially refreshes the report on update. 1`] = `
expandStatus="default"
extraButtons={
Array [
<RunAllButton
filter={null}
runAllCbk={[Function]}
running={false}
/>,
<ReloadButton
reloadCbk={[Function]}
reloading={false}
Expand Down Expand Up @@ -174,6 +184,11 @@ exports[`InteractiveReport Updates testcase state 1`] = `
expandStatus="default"
extraButtons={
Array [
<RunAllButton
filter={null}
runAllCbk={[Function]}
running={false}
/>,
<ReloadButton
reloadCbk={[Function]}
reloading={false}
Expand Down Expand Up @@ -653,6 +668,11 @@ exports[`InteractiveReport handles individual parametrizations being run 1`] = `
expandStatus="default"
extraButtons={
Array [
<RunAllButton
filter={null}
runAllCbk={[Function]}
running={false}
/>,
<ReloadButton
reloadCbk={[Function]}
reloading={false}
Expand Down Expand Up @@ -735,6 +755,11 @@ exports[`InteractiveReport handles individual test suites being run 1`] = `
expandStatus="default"
extraButtons={
Array [
<RunAllButton
filter={null}
runAllCbk={[Function]}
running={false}
/>,
<ReloadButton
reloadCbk={[Function]}
reloading={false}
Expand Down Expand Up @@ -817,6 +842,11 @@ exports[`InteractiveReport handles individual testcases being run 1`] = `
expandStatus="default"
extraButtons={
Array [
<RunAllButton
filter={null}
runAllCbk={[Function]}
running={false}
/>,
<ReloadButton
reloadCbk={[Function]}
reloading={false}
Expand Down Expand Up @@ -899,6 +929,11 @@ exports[`InteractiveReport handles tests being run 1`] = `
expandStatus="default"
extraButtons={
Array [
<RunAllButton
filter={null}
runAllCbk={[Function]}
running={false}
/>,
<ReloadButton
reloadCbk={[Function]}
reloading={false}
Expand Down
Loading