feat: change Pit output format to XML, close #616 #661

andrewbwogi · 2018-11-30T11:20:31Z

The suggested tasks in #616 have been implemented. It remains to be decided what additional information in the PitXMLResult should be used in the final DSpot report. If the method signature of the mutated classes will be used, it could be presented in a more intuitive form. Currently it would be in Java bytecode format. @danglotb, what additional information do you think should be used, and is the current format of method signatures ok?

coveralls · 2018-11-30T11:34:18Z

Pull Request Test Coverage Report for Build 1722

189 of 239 (79.08%) changed or added relevant lines in 10 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.2%) to 84.638%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
dspot/src/main/java/eu/stamp_project/utils/pit/PitCSVResultParser.java	29	30	96.67%
dspot/src/main/java/eu/stamp_project/dspot/selector/PitMutantScoreSelector.java	19	22	86.36%
dspot/src/main/java/eu/stamp_project/utils/pit/AbstractPitResult.java	23	27	85.19%
dspot/src/main/java/eu/stamp_project/utils/pit/PitXMLResultParser.java	55	59	93.22%
dspot/src/main/java/eu/stamp_project/utils/pit/AbstractParser.java	15	20	75.0%
dspot/src/main/java/eu/stamp_project/utils/pit/PitXMLResult.java	24	31	77.42%
dspot/src/main/java/eu/stamp_project/utils/options/JSAPOptions.java	20	31	64.52%
dspot/src/main/java/eu/stamp_project/utils/pit/PitCSVResult.java	2	17	11.76%

Files with Coverage Reduction	New Missed Lines	%
dspot/src/main/java/eu/stamp_project/test_framework/AssertJTestFramework.java	1	42.86%

Totals
Change from base Build 1721:	-0.2%
Covered Lines:	4617
Relevant Lines:	5455

💛 - Coveralls

danglotb · 2018-11-30T13:01:52Z

Hi @andrewbwogi

Thank you very much for the pull request!

It looks great, I'm gonna review the code and keep you updated! :-)

The failing is due to the checkstyle. You should use braces for if statement.

danglotb · 2018-11-30T13:19:22Z

The additional information seems okay for me. The method descriptor should stay as it is, since it a standard format to describe java methods signatures.

danglotb · 2018-11-30T13:20:17Z

dspot/src/main/java/eu/stamp_project/dspot/selector/PitMutantScoreSelector.java


    public PitMutantScoreSelector() {
        this.testThatKilledMutants = new HashMap<>();
+        parser = new PitXMLResultParser();


Here, the parser is hardcoded. Could we use a command line option or a property to allow the user to use the format he wants, XML or CSV?

danglotb · 2018-11-30T13:23:29Z

dspot/src/main/java/eu/stamp_project/utils/pit/AbstractParser.java

+import java.io.IOException;
+import java.util.List;
+
+abstract public class AbstractParser {


We could use a generic here:

abstract public class AbstractParser<T extends AbstractPitResult>

danglotb · 2018-11-30T13:24:00Z

dspot/src/main/java/eu/stamp_project/utils/pit/AbstractParser.java

+        return new File(directoryReportPit.getPath() + PATH_TO_MUTATIONS_RESULT);
+    }
+
+    public List<AbstractPitResult> parseAndDelete(String pathToDirectoryResults) {


And reuse the generic here:

public List<T> parseAndDelete(String pathToDirectoryResults) {

danglotb · 2018-11-30T13:24:12Z

dspot/src/main/java/eu/stamp_project/utils/pit/AbstractParser.java

+        return results;
+    }
+
+    abstract public List<AbstractPitResult> parse(File fileResults);


dspot/src/main/java/eu/stamp_project/utils/pit/PitCSVResult.java

danglotb · 2018-11-30T13:30:10Z

dspot/src/main/java/eu/stamp_project/dspot/selector/PitMutantScoreSelector.java

    }

    public PitMutantScoreSelector(String pathToOriginalResultOfPit) {
        this();
-        initOriginalPitResult(PitResultParser.parse(new File(pathToOriginalResultOfPit)));
+        initOriginalPitResult(parser.parse(new File(pathToOriginalResultOfPit)));


We may have a problem here. In fact, the pathToOriginalResultOfPit come from the command line.

I suggest that: we update the command line options (m|--path-pit-result) in order to:
say that we support also the XML format and according to the given format, we use the correct parser.

Today, one may want to you the CSV format for the initial mutation score but want to use the XML as current output.

danglotb · 2018-11-30T13:31:38Z

@oscarlvp could you check if you need something more?

danglotb · 2018-12-07T08:51:37Z

Any news on the requested changes? Did you encounter any trouble to implement them?

andrewbwogi · 2018-12-08T10:54:16Z

I would like to make some manual tests but I get an error when running the DSpot jar:

no main manifest attribute, in ./dspot-maven-1.2.2-SNAPSHOT.jar

I compile and package the project as instructed in the README except I skip tests, but that has not been a problem before. I will try to find whats wrong. Besides testing, your requested changes have been implemented.

danglotb · 2018-12-08T11:55:56Z

Hi @andrewbwogi

looks great! Gonna check them further.

For the manual testing, what are you doing?

no main manifest attribute, in ./dspot-maven-1.2.2-SNAPSHOT.jar

Do you try to run the maven plugin with a java -jar command line? I think it won't work, but maybe you should just try the dspot core jar, i.e. dspot/target/dspot-1.2.2-SNAPSHOT-jar-with-dependencies.jar rather than the dspot maven jar, i.e. dspot-maven/target/dspot-maven-1.2.2-SNAPSHOT.jar.

danglotb · 2018-12-08T14:15:56Z

I'm good with the changes, tell me when it is ready to be merged.

andrewbwogi · 2018-12-16T19:29:17Z

I found the dspot-1.2.2-SNAPSHOT-jar-with-dependencies.jar file but when trying to run it on the dhell project the program gets stuck. Here is how far the execution goes before freezing when enabling verbose mode: out.txt. DSpot freezes with latest commit (9e7fe94) also without the changes in this pull request. I'm using the following properties file:

project=./
src=src/main/java/
testSrc=src/test/java
javaVersion=8
outputDirectory=dspot-out/

With the basic command:

java -jar ./dspot-1.2.2-SNAPSHOT-jar-with-dependencies.jar -p ./dhell-dspot.properties \
--verbose

Running a DSpot jar from an older commit (caba4a7) with this properties file on dhell works. Is this a known issue?

danglotb · 2018-12-17T16:11:13Z

Hi @andrewbwogi

I do not know this issue. On my side, I cannot reproduce the stuck (DSpot ends well its execution).

According to your logs, PIT is stuck.

Could you try with: the Jacoco selector? i.e. --test-criterion JacocoCoverageSelector

Could you also try with the mutation engine gregor by using the pit mutation score selector (no flag --test-criterion and the flag --gregor?

Since I'm not able to reproduce the issue, it is hard to find the problem.

andrewbwogi · 2018-12-21T12:13:47Z

With --test-criterion JacocoCoverageSelector DSpot terminates on dhell but not with --gregor.

In test-projects, DSpot also freezes with the properties file:

#relative path to the project root from dspot project
project=./
#relative path to the source project from the project properties
src=src/main/java/
#relative path to the test source project from the project properties
testSrc=src/test/java
#java version used
javaVersion=8
#filter used to amplify specific test cases
filter=example.*
#path to the output folder
outputDirectory=target/trash/
# Constant amount of additional time to allow a test to run for
# before considering it to be stuck in an infinite loop
timeoutConstInMillis=10000
#Argument string to use when PIT launches child processes. This is most commonly used
# to increase the amount of memory available to the process,
# but may be used to pass any valid JVM argument.
# Use commas to separate multiple arguments, and put them within brackets
jvmArgs=['-Xmx2048m','-Xms1024m']

#Mutators to apply when using Descartes Mode
#descartesMutators=['void','null','true','false','empty','0','(byte)0','(byte)1','(short)1','(short)2','0L','1L','0.0','1.0','0.0f','1.0f','\\40','\\'A\\'','\\"\\"','\\"A\\"']
descartesMutators=['\\'A\\'']

With command:

java -jar dspot-1.2.2-SNAPSHOT-jar-with-dependencies.jar -p ./test-projects2.properties

When I run with --test-criterion JacocoCoverageSelector in test-projects I get a java.io.FileNotFoundException. Command:

java -jar dspot-1.2.2-SNAPSHOT-jar-with-dependencies.jar -p ./test-projects2.properties --test-criterion JacocoCoverageSelector --verbose

Execution trace: out2.txt
Error for this last execution:

Error: Could not find or load main class ['-Xmx2048m'
Exception in thread "main" java.lang.RuntimeException: java.io.FileNotFoundException: /home/andrew/Skrivbord/stamp/dspot/dspot/src/test/resources/test-projects/target/dspot/TestListener.ser does not exist.
at eu.stamp_project.testrunner.runner.Loader.load(Loader.java:25)
at eu.stamp_project.testrunner.listener.impl.TestListenerImpl.load(TestListenerImpl.java:113)
at eu.stamp_project.testrunner.EntryPoint.runTests(EntryPoint.java:190)
at eu.stamp_project.testrunner.EntryPoint.runTests(EntryPoint.java:180)
at eu.stamp_project.testrunner.EntryPoint.runTests(EntryPoint.java:148)
at eu.stamp_project.utils.execution.TestRunner.runGivenTestMethods(TestRunner.java:64)
at eu.stamp_project.utils.compilation.TestCompiler.compileAndRun(TestCompiler.java:99)
at eu.stamp_project.utils.compilation.TestCompiler.compileRunAndDiscardUncompilableAndFailingTestMethods(TestCompiler.java:55)
at eu.stamp_project.dspot.Amplification.amplification(Amplification.java:85)
at eu.stamp_project.dspot.DSpot._amplify(DSpot.java:275)
at eu.stamp_project.dspot.DSpot._amplifyTestClass(DSpot.java:266)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at eu.stamp_project.dspot.DSpot._amplifyTestClasses(DSpot.java:262)
at eu.stamp_project.dspot.DSpot.amplifyAllTests(DSpot.java:148)
at eu.stamp_project.Main.run(Main.java:59)
at eu.stamp_project.Main.main(Main.java:41)
Caused by: java.io.FileNotFoundException: /home/andrew/Skrivbord/stamp/dspot/dspot/src/test/resources/test-projects/target/dspot/TestListener.ser does not exist.
... 22 more

I will try to find at what commit DSpot starts to freeze.

danglotb · 2018-12-24T08:59:11Z

Hi @andrewbwogi

Where did you find the properties file?

There are at least two errors in it:

jvmArgs=['-Xmx2048m','-Xms1024m']

Since 38fe714, the jvmArgs must not wrapped by brackets, nor in single quote. They must only be separated by commas. You should have:

jvmArgs=-Xmx2048m,-Xms1024m

descartesMutators=['\\'A\\'']

Same here, you should have:

descartesMutators='A'

Here, the single quotes express the fact that the mutator is working on char.

Could you please:

fix your properties and retry?
tell me where this properties were? This is a bug in the documentation and we must fix it.

Thank you very much @andrewbwogi ! :-)

andrewbwogi · 2018-12-26T09:51:52Z

The properties file was a copy of a test-projects properties file from an old DSpot commit, I missed the updated syntax. However, even in the current DSpot commit, the use of descartesMutators is specified in the old way with brackets in that file.

With the updated properties file, DSpot terminates with --test-criterion JacocoCoverageSelector but not with --test-criterion PitMutantScoreSelector.

andrewbwogi · 2018-12-28T10:25:47Z

Suddenly, everything works. I've done manual tests now, the changes can be merged.

danglotb · 2018-12-28T13:34:56Z

Okay this is a great! We merged your pull request before the new year ;)

andrewbwogi added 18 commits November 18, 2018 10:21

create preliminary implementations

b483416

make code tighter

40dc32c

pass xml parser test

21b430c

add fields to toString()

696c985

fix special characters parse

b3db592

implement getPathOfMutationsFile and parseAndDelete

8c8b109

let caller determine file encoding

d3f0584

refactor PitResult and PitResultParser to CSV variants

423b822

clean up code

7f065b9

fix compilation error

6fdda8a

adapt to general code style

8a5104b

make pit produce both csv and xml results

0632974

change to xml parser

d12d379

fix test cases

709074a

switch to AbstractPitResult in tests

9cdfdad

use pit xml parser in tests

3e58fd5

merge with master

cf7379f

fix package name and imports

9c5ff3f

danglotb self-assigned this Nov 30, 2018

danglotb requested changes Nov 30, 2018

View reviewed changes

andrewbwogi added 2 commits November 30, 2018 15:12

fix code style

ade5191

parametrize AbstractParser

732c8d4

andrewbwogi added 3 commits December 7, 2018 14:28

add user choice for Pit output format

637cc83

fix selector call

801d921

handle pit selector configuration in JSAPOptions

7107966

andrewbwogi added 3 commits December 7, 2018 17:05

add help text

c04c9b9

fix constructor call

95f0e83

fix code style

4b56cc3

danglotb approved these changes Dec 8, 2018

View reviewed changes

danglotb assigned andrewbwogi and unassigned danglotb Dec 9, 2018

Merge branch 'master' into pit-output-format

1d51b75

add documentation for pit output format selection

3929aa0

danglotb merged commit 120548d into STAMP-project:master Dec 28, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: change Pit output format to XML, close #616 #661

feat: change Pit output format to XML, close #616 #661

andrewbwogi commented Nov 30, 2018

coveralls commented Nov 30, 2018 •

edited

Loading

danglotb commented Nov 30, 2018 •

edited

Loading

danglotb commented Nov 30, 2018

danglotb Nov 30, 2018

danglotb Nov 30, 2018

danglotb Nov 30, 2018

danglotb Nov 30, 2018

danglotb Nov 30, 2018

danglotb commented Nov 30, 2018

danglotb commented Dec 7, 2018

andrewbwogi commented Dec 8, 2018

danglotb commented Dec 8, 2018

danglotb commented Dec 8, 2018

andrewbwogi commented Dec 16, 2018

danglotb commented Dec 17, 2018

andrewbwogi commented Dec 21, 2018

danglotb commented Dec 24, 2018

andrewbwogi commented Dec 26, 2018

andrewbwogi commented Dec 28, 2018

danglotb commented Dec 28, 2018

feat: change Pit output format to XML, close #616 #661

feat: change Pit output format to XML, close #616 #661

Conversation

andrewbwogi commented Nov 30, 2018

coveralls commented Nov 30, 2018 • edited Loading

Pull Request Test Coverage Report for Build 1722

💛 - Coveralls

danglotb commented Nov 30, 2018 • edited Loading

danglotb commented Nov 30, 2018

danglotb Nov 30, 2018

Choose a reason for hiding this comment

danglotb Nov 30, 2018

Choose a reason for hiding this comment

danglotb Nov 30, 2018

Choose a reason for hiding this comment

danglotb Nov 30, 2018

Choose a reason for hiding this comment

danglotb Nov 30, 2018

Choose a reason for hiding this comment

danglotb commented Nov 30, 2018

danglotb commented Dec 7, 2018

andrewbwogi commented Dec 8, 2018

danglotb commented Dec 8, 2018

danglotb commented Dec 8, 2018

andrewbwogi commented Dec 16, 2018

danglotb commented Dec 17, 2018

andrewbwogi commented Dec 21, 2018

danglotb commented Dec 24, 2018

andrewbwogi commented Dec 26, 2018

andrewbwogi commented Dec 28, 2018

danglotb commented Dec 28, 2018

coveralls commented Nov 30, 2018 •

edited

Loading

danglotb commented Nov 30, 2018 •

edited

Loading