Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Error] Unable to resolve all the dependencies. #28

Closed
patbaumgartner opened this issue Jan 8, 2021 · 15 comments
Closed

[Error] Unable to resolve all the dependencies. #28

patbaumgartner opened this issue Jan 8, 2021 · 15 comments
Labels
bug Something isn't working

Comments

@patbaumgartner
Copy link
Contributor

Hi,

First of all, thanks for coming up with such a great idea of a Maven plugin. Love it!

Running the plugin as described in the Readme.md with versions 1.1.0 and 1.1.2 leads always to the error message
[Error] Unable to resolve all the dependencies.

Running Maven in debug mode with -X did not really help ... any idea what could be wrong? I tried to declare the plugin in the pom but also just ran the plugin directly from the command line with all the params in the CLI.

Btw. we develop the apps in an enterprise setup where we have proxies, self-signed certs and so on.

Best,
Patrick
@cesarsotovalero
Copy link
Collaborator

Hi @patbaumgartner,

I'm glad that you find this plugin useful and I'll enjoy giving you support.

DepClean needs to download all the dependencies of a Maven project in order to analyze the bytecode calls between your project and its dependencies. You're getting [Error] Unable to resolve all the dependencies which means that some dependencies cannot be downloaded successfully.

Btw. we develop the apps in an enterprise setup where we have proxies, self-signed certs and so on.

This may be the cause of the issue. Does your project use dependencies from other repositories than Maven Central?

@cesarsotovalero cesarsotovalero added the bug Something isn't working label Jan 14, 2021
@patbaumgartner
Copy link
Contributor Author

patbaumgartner commented Jan 15, 2021 •
edited
Loading

Hi @cesarsotovalero

i just tried to create a simple spring boot project, just with a web starter and then ran the command on the command line

mvn se.kth.castor:depclean-maven-plugin:1.1.0:depclean -Dcreate.pom.debloated=true -Dcreate.result.json=true

and the output was the following. Not sure what is wrong here. Btw. i also tried to add the plugin to the pom.

C:\Users\abc\Desktop\demo>mvn se.kth.castor:depclean-maven-plugin:1.1.0:depclean -Dcreate.pom.debloated=true -Dcreate.result.json=true
[INFO] Scanning for projects...
[INFO]
[INFO] --------------------------< com.example:demo >--------------------------
[INFO] Building demo 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- depclean-maven-plugin:1.1.0:depclean (default-cli) @ demo ---
-------------------------------------------------------
[INFO] Starting DepClean dependency analysis
[ERROR] Unable resolve all the dependencies.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.679 s
[INFO] Finished at: 2021-01-15T13:31:36+01:00
[INFO] ------------------------------------------------------------------------

@SagarGupta1
Copy link

Facing the same error, kindly let me know when the issue will be resolved.

@sebthom
Copy link

sebthom commented Jan 22, 2021

Same issue here. Would be helpful if the plugin would do the analysis at least for the dependencies it could download and it if could show which dependencies could not be downloaded.

I am trying to use the plugin in an enterprise context were we are using Artifactory as Maven central mirror.

@cesarsotovalero
Copy link
Collaborator

cesarsotovalero commented Jan 22, 2021 •
edited
Loading

Hi @patbaumgartner

i just tried to create a simple spring boot project, just with a web starter...

I just executed DepClean on a Spring project with the initial configuration. It contains the following dependencies:

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

DepClean worked fine. This is the output that I got:

-------------------------------------------------------
 D E P C L E A N   A N A L Y S I S   R E S U L T S
-------------------------------------------------------
USED DIRECT DEPENDENCIES [0]: 
USED INHERITED DEPENDENCIES [0]: 
USED TRANSITIVE DEPENDENCIES [4]: 
        org.springframework.boot:spring-boot-autoconfigure:2.4.2:compile (1 MB)
        org.springframework.boot:spring-boot:2.4.2:compile (1 MB)
        org.springframework.boot:spring-boot-test:2.4.2:test (213 KB)
        org.junit.jupiter:junit-jupiter-api:5.7.0:test (170 KB)
POTENTIALLY UNUSED DIRECT DEPENDENCIES [2]: 
        org.springframework.boot:spring-boot-starter-web:2.4.2:compile (4 KB)
        org.springframework.boot:spring-boot-starter-test:2.4.2:test (4 KB)
POTENTIALLY UNUSED INHERITED DEPENDENCIES [0]: 
POTENTIALLY UNUSED TRANSITIVE DEPENDENCIES [54]: 
        org.assertj:assertj-core:3.18.1:test (4 MB)
        net.bytebuddy:byte-buddy:1.10.19:test (3 MB)
        org.apache.tomcat.embed:tomcat-embed-core:9.0.41:compile (3 MB)
        org.springframework:spring-web:5.3.3:compile (1 MB)
        org.springframework:spring-core:5.3.3:compile (1 MB)
        com.fasterxml.jackson.core:jackson-databind:2.11.4:compile (1 MB)
        org.springframework:spring-context:5.3.3:compile (1 MB)
        org.springframework:spring-webmvc:5.3.3:compile (972 KB)
        org.springframework:spring-test:5.3.3:test (745 KB)
        org.springframework:spring-beans:5.3.3:compile (679 KB)
        org.mockito:mockito-core:3.6.28:test (659 KB)
        org.junit.jupiter:junit-jupiter-params:5.7.0:test (554 KB)
        ch.qos.logback:logback-core:1.2.3:compile (460 KB)
        org.springframework:spring-aop:5.3.3:compile (365 KB)
        com.fasterxml.jackson.core:jackson-core:2.11.4:compile (343 KB)
        org.yaml:snakeyaml:1.27:compile (302 KB)
        org.apache.logging.log4j:log4j-api:2.13.3:compile (285 KB)
        ch.qos.logback:logback-classic:1.2.3:compile (283 KB)
        org.springframework:spring-expression:5.3.3:compile (275 KB)
        org.apache.tomcat.embed:tomcat-embed-websocket:9.0.41:compile (265 KB)
        net.bytebuddy:byte-buddy-agent:1.10.19:test (253 KB)
        org.glassfish:jakarta.el:3.0.3:compile (232 KB)
        com.jayway.jsonpath:json-path:2.4.0:test (217 KB)
        org.junit.jupiter:junit-jupiter-engine:5.7.0:test (206 KB)
        org.springframework.boot:spring-boot-test-autoconfigure:2.4.2:test (177 KB)
        org.junit.platform:junit-platform-engine:1.7.0:test (176 KB)
        org.xmlunit:xmlunit-core:2.7.0:test (164 KB)
        org.hamcrest:hamcrest:2.2:test (120 KB)
        net.minidev:json-smart:2.3:test (117 KB)
        jakarta.xml.bind:jakarta.xml.bind-api:2.3.3:test (112 KB)
        com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.11.4:compile (108 KB)
        org.junit.platform:junit-platform-commons:1.7.0:test (97 KB)
        com.fasterxml.jackson.core:jackson-annotations:2.11.4:compile (70 KB)
        org.objenesis:objenesis:3.1:test (59 KB)
        org.ow2.asm:asm:5.0.4:test (52 KB)
        jakarta.activation:jakarta.activation-api:1.2.2:test (45 KB)
        org.slf4j:slf4j-api:1.7.30:compile (40 KB)
        com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.11.4:compile (33 KB)
        org.skyscreamer:jsonassert:1.5.0:test (29 KB)
        net.minidev:accessors-smart:1.2:test (29 KB)
        jakarta.annotation:jakarta.annotation-api:1.3.5:compile (24 KB)
        org.springframework:spring-jcl:5.3.3:compile (23 KB)
        com.vaadin.external.google:android-json:0.0.20131108.vaadin1:test (17 KB)
        org.apache.logging.log4j:log4j-to-slf4j:2.13.3:compile (17 KB)
        com.fasterxml.jackson.module:jackson-module-parameter-names:2.11.4:compile (9 KB)
        org.opentest4j:opentest4j:1.2.0:test (7 KB)
        org.junit.jupiter:junit-jupiter:5.7.0:test (6 KB)
        org.mockito:mockito-junit-jupiter:3.6.28:test (4 KB)
        org.springframework.boot:spring-boot-starter-tomcat:2.4.2:compile (4 KB)
        org.springframework.boot:spring-boot-starter:2.4.2:compile (4 KB)
        org.springframework.boot:spring-boot-starter-logging:2.4.2:compile (4 KB)
        org.springframework.boot:spring-boot-starter-json:2.4.2:compile (4 KB)
        org.slf4j:jul-to-slf4j:1.7.30:compile (4 KB)
        org.apiguardian:apiguardian-api:1.1.0:test (2 KB)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

What version of Java are you using?

@cesarsotovalero cesarsotovalero mentioned this issue Jan 23, 2021
Closed
@patbaumgartner
Copy link
Contributor Author

patbaumgartner commented Jan 25, 2021 •
edited
Loading

Hi @cesarsotovalero I just downloaded the spring boot app from my PC at home (no nexus, artefactory or so on in between).

You can use the following link to have the same project created - https://start.spring.io/#!type=maven-project&language=java&platformVersion=2.4.2.RELEASE&packaging=jar&jvmVersion=11&groupId=com.example&artifactId=demo&name=demo&description=Demo%20project%20for%20Spring%20Boot&packageName=com.example.demo&dependencies=web

  <properties>
    <java.version>11</java.version>
  </properties>
  <dependencies>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-test</artifactId>
      <scope>test</scope>
    </dependency>
  </dependencies>

Then I ran: mvn clean verify and mvnw se.kth.castor:depclean-maven-plugin:1.1.0:depclean -Dcreate.pom.debloated=true -Dcreate.result.json=true and it produced the same issue.

My versions of Java and Maven you see here.

C:\Users\myUser\Downloads\demo\demo>mvnw --version
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: C:\Users\patbaumgartner.MYWORKSTATION\.m2\wrapper\dists\apache-maven-3.6.3-bin\1iopthnavndlasol9gbrbg6bf2\apache-maven-3.6.3
Java version: 11.0.9.1, vendor: AdoptOpenJDK, runtime: C:\Program Files\AdoptOpenJDK\jdk-11.0.9.101-hotspot
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

@cesarsotovalero
Copy link
Collaborator

Hi @patbaumgartner

Thanks for the feedback. I think that I found and solved the issue. The problem was related to the execution of the mvn commands on Windows. I've added support for it: https://github.com/castor-software/depclean/blob/aa1f7610bb6e70b1d8f15b935e27eda7583787da/depclean-maven-plugin/src/main/java/se/kth/depclean/util/MavenInvoker.java#L59-L64

Please, clone and install the current version. Then, execute DepClean: mvn se.kth.castor:depclean-maven-plugin:1.1.2-SNAPSHOT:depclean. You can add the additional parameters -Dcreate.pom.debloated=true -Dcreate.result.json=true if you want more details.

Please let me know if it works for you now.

@patbaumgartner
Copy link
Contributor Author

Thanks @cesarsotovalero

It works now, but the result is not satisfying. Anyhow, wow! I am impressed! 👍

C:\Users\myUser\Downloads\demo>mvn se.kth.castor:depclean-maven-plugin:1.1.2-SNAPSHOT:depclean
[INFO] Scanning for projects...
[INFO]
[INFO] --------------------------< com.example:demo >--------------------------
[INFO] Building demo 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- depclean-maven-plugin:1.1.2-SNAPSHOT:depclean (default-cli) @ demo ---
-------------------------------------------------------
[INFO] Starting DepClean dependency analysis
-------------------------------------------------------
DIRECT DEPENDENCIES: [org.springframework.boot:spring-boot-starter-web:jar:2.4.2:compile, org.springframework.boot:spring-boot-starter-test:jar:2.4.2:test]
-------------------------------------------------------
TRANSITIVE DEPENDENCIES: [org.springframework.boot:spring-boot-starter:jar:2.4.2:compile, org.springframework.boot:spring-boot:jar:2.4.2:compile, org.springframework.boot:spring-boot-autoconfigure:jar:2.4.2:compile, org.springframework.boot:spring-boot-starter-logging:jar:2.4.2:compile, ch.qos.logback:logback-classic:jar:1.2.3:compile, ch.qos.logback:logback-core:jar:1.2.3:compile, org.apache.logging.log4j:log4j-to-slf4j:jar:2.13.3:compile, org.apache.logging.log4j:log4j-api:jar:2.13.3:compile, org.slf4j:jul-to-slf4j:jar:1.7.30:compile, jakarta.annotation:jakarta.annotation-api:jar:1.3.5:compile, org.yaml:snakeyaml:jar:1.27:compile, org.springframework.boot:spring-boot-starter-json:jar:2.4.2:compile, com.fasterxml.jackson.core:jackson-databind:jar:2.11.4:compile, com.fasterxml.jackson.core:jackson-annotations:jar:2.11.4:compile, com.fasterxml.jackson.core:jackson-core:jar:2.11.4:compile, com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:2.11.4:compile, com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.11.4:compile, com.fasterxml.jackson.module:jackson-module-parameter-names:jar:2.11.4:compile, org.springframework.boot:spring-boot-starter-tomcat:jar:2.4.2:compile, org.apache.tomcat.embed:tomcat-embed-core:jar:9.0.41:compile, org.glassfish:jakarta.el:jar:3.0.3:compile, org.apache.tomcat.embed:tomcat-embed-websocket:jar:9.0.41:compile, org.springframework:spring-web:jar:5.3.3:compile, org.springframework:spring-beans:jar:5.3.3:compile, org.springframework:spring-webmvc:jar:5.3.3:compile, org.springframework:spring-aop:jar:5.3.3:compile, org.springframework:spring-context:jar:5.3.3:compile, org.springframework:spring-expression:jar:5.3.3:compile, org.springframework.boot:spring-boot-test:jar:2.4.2:test, org.springframework.boot:spring-boot-test-autoconfigure:jar:2.4.2:test, com.jayway.jsonpath:json-path:jar:2.4.0:test, net.minidev:json-smart:jar:2.3:test, net.minidev:accessors-smart:jar:1.2:test, org.ow2.asm:asm:jar:5.0.4:test, org.slf4j:slf4j-api:jar:1.7.30:compile, jakarta.xml.bind:jakarta.xml.bind-api:jar:2.3.3:test, jakarta.activation:jakarta.activation-api:jar:1.2.2:test, org.assertj:assertj-core:jar:3.18.1:test, org.hamcrest:hamcrest:jar:2.2:test, org.junit.jupiter:junit-jupiter:jar:5.7.0:test, org.junit.jupiter:junit-jupiter-api:jar:5.7.0:test, org.apiguardian:apiguardian-api:jar:1.1.0:test, org.opentest4j:opentest4j:jar:1.2.0:test, org.junit.platform:junit-platform-commons:jar:1.7.0:test, org.junit.jupiter:junit-jupiter-params:jar:5.7.0:test, org.junit.jupiter:junit-jupiter-engine:jar:5.7.0:test, org.junit.platform:junit-platform-engine:jar:1.7.0:test, org.mockito:mockito-core:jar:3.6.28:test, net.bytebuddy:byte-buddy:jar:1.10.19:test, net.bytebuddy:byte-buddy-agent:jar:1.10.19:test, org.objenesis:objenesis:jar:3.1:test, org.mockito:mockito-junit-jupiter:jar:3.6.28:test, org.skyscreamer:jsonassert:jar:1.5.0:test, com.vaadin.external.google:android-json:jar:0.0.20131108.vaadin1:test, org.springframework:spring-core:jar:5.3.3:compile, org.springframework:spring-jcl:jar:5.3.3:compile, org.springframework:spring-test:jar:5.3.3:test, org.xmlunit:xmlunit-core:jar:2.7.0:test]
-------------------------------------------------------
USED DEPENDENCIES: []
-------------------------------------------------------
 D E P C L E A N   A N A L Y S I S   R E S U L T S
-------------------------------------------------------
USED DIRECT DEPENDENCIES [0]:
USED INHERITED DEPENDENCIES [0]:
USED TRANSITIVE DEPENDENCIES [0]:
POTENTIALLY UNUSED DIRECT DEPENDENCIES [2]:
        org.springframework.boot:spring-boot-starter-web:2.4.2:compile (4 KB)
        org.springframework.boot:spring-boot-starter-test:2.4.2:test (4 KB)
POTENTIALLY UNUSED INHERITED DEPENDENCIES [0]:
POTENTIALLY UNUSED TRANSITIVE DEPENDENCIES [58]:
        org.assertj:assertj-core:3.18.1:test (4 MB)
        net.bytebuddy:byte-buddy:1.10.19:test (3 MB)
        org.apache.tomcat.embed:tomcat-embed-core:9.0.41:compile (3 MB)
        org.springframework:spring-web:5.3.3:compile (1 MB)
        org.springframework.boot:spring-boot-autoconfigure:2.4.2:compile (1 MB)
        org.springframework:spring-core:5.3.3:compile (1 MB)
        com.fasterxml.jackson.core:jackson-databind:2.11.4:compile (1 MB)
        org.springframework.boot:spring-boot:2.4.2:compile (1 MB)
        org.springframework:spring-context:5.3.3:compile (1 MB)

Would be great if it could find all jars needed if I start the app. .e.g Tomcat, Spring Context/Beans, etc. should be shown.

@cesarsotovalero
Copy link
Collaborator

It works now, but the result is not satisfying.

Did you run mvn clean verify before executing DepClean?

@cesarsotovalero
Copy link
Collaborator

Facing the same error, kindly let me know when the issue will be resolved.

@SagarGupta1 The issue has been resolved.

@cesarsotovalero
Copy link
Collaborator

Same issue here.

@sebthom The issue has been resolved, see #28 (comment)

Would be helpful if the plugin would do the analysis at least for the dependencies it could download and it if could show which dependencies could not be downloaded.

I'm currently working on this #31. I'll let you know when it is done.

@patbaumgartner
Copy link
Contributor Author

mvn clean verify helps to reach this. But then still tomcat and maybe other deps are missing. Maybe loaded via reflection?

-------------------------------------------------------
 D E P C L E A N   A N A L Y S I S   R E S U L T S
-------------------------------------------------------
USED DIRECT DEPENDENCIES [0]:
USED INHERITED DEPENDENCIES [0]:
USED TRANSITIVE DEPENDENCIES [4]:
        org.springframework.boot:spring-boot-autoconfigure:2.4.2:compile (1 MB)
        org.springframework.boot:spring-boot:2.4.2:compile (1 MB)
        org.springframework.boot:spring-boot-test:2.4.2:test (213 KB)
        org.junit.jupiter:junit-jupiter-api:5.7.0:test (170 KB)
POTENTIALLY UNUSED DIRECT DEPENDENCIES [2]:
        org.springframework.boot:spring-boot-starter-web:2.4.2:compile (4 KB)
        org.springframework.boot:spring-boot-starter-test:2.4.2:test (4 KB)
POTENTIALLY UNUSED INHERITED DEPENDENCIES [0]:
POTENTIALLY UNUSED TRANSITIVE DEPENDENCIES [54]:
        org.assertj:assertj-core:3.18.1:test (4 MB)
        net.bytebuddy:byte-buddy:1.10.19:test (3 MB)
        org.apache.tomcat.embed:tomcat-embed-core:9.0.41:compile (3 MB)
        org.springframework:spring-web:5.3.3:compile (1 MB)

@cesarsotovalero
Copy link
Collaborator

mvn clean verify helps to reach this. But then still tomcat and maybe other deps are missing. Maybe loaded via reflection?

DepClean uses static analysis. Therefore, it may not detect some reflection calls between the project and its dependencies. Dealing with reflection, dynamic calls, dynamic proxies, etc., is a hard problem for Java code analysis. So, I recommend refining the dependencies of a Maven project as follows:

  1. Use DepClean to get the big picture about the dependencies used/unused,
  2. Exclude/remove the unused dependencies as per the DepClean report,
  3. Execute the test and build the project, if it pass go to 6., else continue,
  4. Add the missing dependency as per the build report
  5. Go to 3.
  6. Done!

@patbaumgartner
Copy link
Contributor Author

patbaumgartner commented Jan 25, 2021 via email

@cesarsotovalero
Copy link
Collaborator

Hi @patbaumgartner,

Is there a possibility to add hints like for Spring Boot with the web-starter that we could add the embedded Tomcat as a mandatory dependency and then we continue the search?

DepClean has an ignoreDependencies parameter to skip the analysis of custom dependencies (see https://github.com/castor-software/depclean#optional-parameters). Since we cannot cover all the possible reflection-based libraries, we prefer to let this decision to developers 😄

Would it be possible to search for strings which look like a package, method which is used during reflection and continue the search?

DepClean can detect classes or class members that are created or invoked dynamically using basic constructs such as class.forName("someClass") or class.getMethod("someMethod", null). To do so, DepClean parses the constant pool of the class files (see https://github.com/castor-software/depclean/blob/master/depclean-core/src/main/java/se/kth/depclean/core/analysis/asm/ConstantPoolParser.java#L94-L158)

If you know any other trick to deal with reflection: your contribution is more than welcome!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@patbaumgartner @sebthom @SagarGupta1 @cesarsotovalero