Automatic Dependency Assessment

[mircealungu]

Problem: Reckless Package Reuse

Do you remember the left-pad incident? A few years ago one developer singlehandedly took down the entire npm ecosystem just because so many projects depended on an insignificant single third-party function that anybody with half a year of programming experience should be able to implement?

Reuse is a fantastic thing, and we could not be where we are in the world of software without reuse. At the same time, there are also prices that are associated with reuse which sometimes might be higher than the benefits of reuse. An interesting recent article about the problems that are incurred by external dependencies was presented by Russ Cox in Our Software Dependency Problem.

Thus, the decision of introducing or keeping a dependency should be made on an as rational as possible basis, ideally taking into account the costs and benefits of that dependency. However, over the years projects accumulate dependencies and the reason for their introduction is sometimes obsolete and sometimes lost in the myst of time. It would be good if an automatic tool would allow the developers to quickly analyze the "strength" and "benefits" of various third-party dependencies based on source code analysis.

Approach: Automatic Dependency Assessment

The challenge of this project is to design a general framework that is capable of analyzing a given dependency, and computing its benefits to the subject system... Ideally you would even be able to provide a measure which informs the developer of whether it's worth keeping the dependency or maybe simply writing the code themselves is better.

Once one dependency can be evaluated, you would implement an analysis bot that would take as input a GitHub repository and a dependency specification (e.g. the following example presents the dependencies for a Python project) and would then provide information about the (relative) importance of every one of the packages that are listed in the dependency specification.

Note: Project can be extended to more than one student by taking into account more than a single language.

Further Reading

Why Google stores billions of lines of code in a single repository , R. Potvin and J. Levenberg

Several efforts at Google have sought to rein in unnecessary dependencies. Tooling exists to help identify and remove unused dependencies, or dependencies linked into the product binary for historical or accidental reasons, that are not needed. Tooling also exists to identify underutilized dependencies, or dependencies on large libraries that are mostly unneeded, as candidates for refactoring.

Searching for Build Debt: Experiences Managing Technical Debt at Google, J.D. Morgenthaler et al.

Clipper makes suggestions by ranking dependencies in terms of high cost and low removal effort.

A journey at the heart of 2.4 million Maven artifacts a lot of declared dependencies are never actually used; furthermore many are declared but only a small part of their interface is actually used.

Armin Ronacher on Dependencies and Trust - interesting read; touches the topics of auditing updates, classifying dependencies based on their "weight" (e.g. left-pad is a different category of dependency than jquery).

[mircealungu]

Nivo is trying to get rid of react-motion two years later; people can't upgrade to react 17 because that react-motion dependency down the tree has specified a max of react 16 in its dependencies.

• issue opened in August 2019 Tracking issue for removing react-motion dependency plouc/nivo#667