fix(ci): tests are failing after recent package bumps #2807
Assignees
Labels
bug
Something isn't working
dependencies
Pull requests that update a dependency file
P1
Priority 1: Highest
Milestone
Comments
|
|
|
@outSH Yeah, sorry, this is on me, going fast and breaking things (as much as I don't like to do that). Deadlines are a bit tight right now and that's been a forcing function. Below is my step by step update on each of the issues you mentioned: For the For the
|
I think the replacement for it is https://github.com/ipfs/js-kubo-rpc-client (for HTTP API), but it claims to be Work In Progress. They didn't commit any significant change this year so maybe they are done, but I'm not sure. Either way I didn't find any other alternative, so it seems we are between a rock and a hard place on ths one :S |
|
@petermetz BTW If you're OK with using https://github.com/ipfs/js-kubo-rpc-client I can migrate to it |
petermetz
added a commit
to petermetz/cacti
that referenced
this issue
Oct 20, 2023
This is a workaround for the problem Peter intorduced in the build with an earlier commit where he upgraded axios to 1.5.1 universally. The above had lead to a problem with the nano package, which is a dependency of the fabric node SDK packages. The longer term solution is to migrate to the newer Fabric Gateway client SDK and completely remove the older fabric dependencies from the project, but until we can do that (a big undertaking) we have to be content with this shorter term workaround and hope that axios 0.27.2 does not turn out to be vulnerable critically (because that would force our hand with upgrades again). Partially addresses hyperledger-cacti#2807 (not a full fix) Signed-off-by: Peter Somogyvari <[email protected]>
5 tasks
@outSH I'm 100% OK with that, whatever that actually works and isn't vulnerable is still better than something that is known to never again receive security updates. So the kubo RPC client is a step forward IMO even if it's right now unstable. On the topic of the axios fix: I just submitted a PR that should take care of that problem. Between that and your kubo migration we just down to the |
|
@petermetz Hi, I've prepared a draft with some comments for fixing IPFS issues, have a look - #2829 |
petermetz
added a commit
to petermetz/cacti
that referenced
this issue
Oct 24, 2023
…0-23 1. Couldn't get rid of vulnerable versions in a couple of dependencies because the underlying dependencies have gone ESM only which is a blocker for us at the moment unfortunately. 2. Swapped out the ubiquity TS client to a version of it that I self published onto npm after a full renovation of all of its dependencies. Depends on hyperledger-cacti#2807 (because that one also has a couple of dependency bumps that are needed to eliminate the vulnerabilities) Fixes hyperledger-cacti#2828 Signed-off-by: Peter Somogyvari <[email protected]>
5 tasks
petermetz
added a commit
that referenced
this issue
Oct 26, 2023
This is a workaround for the problem Peter intorduced in the build with an earlier commit where he upgraded axios to 1.5.1 universally. The above had lead to a problem with the nano package, which is a dependency of the fabric node SDK packages. The longer term solution is to migrate to the newer Fabric Gateway client SDK and completely remove the older fabric dependencies from the project, but until we can do that (a big undertaking) we have to be content with this shorter term workaround and hope that axios 0.27.2 does not turn out to be vulnerable critically (because that would force our hand with upgrades again). Partially addresses #2807 (not a full fix) Signed-off-by: Peter Somogyvari <[email protected]>
petermetz
added a commit
to petermetz/cacti
that referenced
this issue
Nov 1, 2023
…0-23 1. Couldn't get rid of vulnerable versions in a couple of dependencies because the underlying dependencies have gone ESM only which is a blocker for us at the moment unfortunately. 2. Swapped out the ubiquity TS client to a version of it that I self published onto npm after a full renovation of all of its dependencies. Depends on hyperledger-cacti#2807 (because that one also has a couple of dependency bumps that are needed to eliminate the vulnerabilities) Fixes hyperledger-cacti#2828 Signed-off-by: Peter Somogyvari <[email protected]>
5 tasks
petermetz
added a commit
to petermetz/cacti
that referenced
this issue
Nov 7, 2023
…0-23 1. Couldn't get rid of vulnerable versions in a couple of dependencies because the underlying dependencies have gone ESM only which is a blocker for us at the moment unfortunately. 2. Swapped out the ubiquity TS client to a version of it that I self published onto npm after a full renovation of all of its dependencies. Depends on hyperledger-cacti#2807 (because that one also has a couple of dependency bumps that are needed to eliminate the vulnerabilities) Fixes hyperledger-cacti#2828 Signed-off-by: Peter Somogyvari <[email protected]>
petermetz
added
dependencies
petermetz
added a commit
to petermetz/cacti
that referenced
this issue
Nov 10, 2023
…0-23 1. Couldn't get rid of vulnerable versions in a couple of dependencies because the underlying dependencies have gone ESM only which is a blocker for us at the moment unfortunately. 2. Swapped out the ubiquity TS client to a version of it that I self published onto npm after a full renovation of all of its dependencies. Depends on hyperledger-cacti#2807 (because that one also has a couple of dependency bumps that are needed to eliminate the vulnerabilities) Fixes hyperledger-cacti#2828 Signed-off-by: Peter Somogyvari <[email protected]>
petermetz
pushed a commit
to outSH/cactus
that referenced
this issue
Nov 10, 2023
- Replace deprecated ipfs-http-client with kubo-rpc-client.
- kubo-rpc-client must be imported dynamically since it's ESM-only and we
still use CJS.
Peter's additional changes:
---------------------------
build(typescript): project-wide fixes to allow us to use ESM-only deps
Apologies for the huge diff, this can't be broken down to smaller changes
that would still compile because of cross-package dependencies.
I realize that this change is not exactly the optimal solution, but
it is probably a step in the right direction.
If I somehow found the time to submit pull requests to the libraries
that I needed to fork and re-publish (see details below) and then get
the changes onto the upstream and get them released as the official
packages, then we could (in theory) arrive at a solution that is the
recommended way of fixing these problems (apart from going full ESM-only)
This work stands on the shoulders of the previous commits from @outSH
and takes a slightly different direction compared to what we've been
talking about earlier on account of the problem that the eval-import
workaround causes crashes in Jest.
Based on the above I went through the following adventures:
1. I migrated the build system of kubo-rpc-client myself so that it
correctly exports CJS and ESM and typings for both of those as well,
I put that code on my fork [1] and then published it onto npm as well [2]
After this, I was hoping that now we could just import the package in
our CJS code without issues, but what really happened is that instead of
crashing at the require call that pull in kubo itself, it started crashing
deeper in the require stack where kubo itself was requiring it's own
ESM only dependencies (of which there seem to be at least 10 or 15).
At this point I realized that me migrating and self-publishing all of
these additional packages might not be worth the effort and started looking
for something easier.
2. I gave dynamic imports + moduleResultion=Node16 as my next attempt to
get our build back to working order. With this, the kubo-rpc-client
can now be imported dynamically without issues in packages that declare
themselves as resolving modules as "Node16" in their tsconfig.json
Other issues here were encountered because not all of our ESM only packages
are used in a way that they can be imported dynamically (for example
if their types are part of our own types or are being re-exported).
The two libraries with this problem were `run-time-error` and
`socket.io-client` for both of which I ended up going through the same
treatment as for kubo-rpc-client above (but this time my effort wasn't)
in vain. They work and so I did some search and replace in the entire
codebase to use these re-published packages with the correct types:
[3] [4] [5] [6]
3. After this the project build was working, but Jest was still failing with
compiler errors which I determined to happen because it uses the
root tsconfig.json file for it's internal TS compilation and that root
tsconfig.json file was not setting module resolution to Node16.
4. After fixing that the final hurdle (hopefully) was to ensure that jest gets
execued with the custom node option as below:
NODE_OPTIONS=--experimental-vm-modules yarn jest
[1] https://github.com/petermetz/js-kubo-rpc-client-esm-cjs
[2] https://www.npmjs.com/package/kubo-rpc-client-esm-cjs
[3] https://github.com/petermetz/socket.io-client
[4] https://www.npmjs.com/package/socket.io-client-fixed-types
[5] https://github.com/petermetz/RuntimeError
[6] https://www.npmjs.com/package/run-time-error-cjs
Huge thanks for https://arethetypeswrong.github.io/ a tool I used
extensively to create the fixes for the libraries above.
One more thing that I tried just to collect more data points was to
set the moduleResultion project-wide to Node16 via setting it
in the root tsconfig.base.json but this broke the compiler itself,
as in, there is a bug in the Typescript compiler in v4.x as seen here:
microsoft/TypeScript#51221
So this is one more reason for us to upgrade to 5.x as soon as possible.
I also needed to add "run-time-error" to the root package.json as a
dependency because it was accidentally providing that to some sub-packages
and when we moved to "run-time-error-cjs" the tests that install
plugins from npm started failing (because those releases are still
using "run-time-error" and not "run-time-error-cjs")
------------------------------------
Fixes hyperledger-cacti#2807
Fixes hyperledger-cacti#2852
Depends on: hyperledger-cacti#2821
Co-authored-by: Peter Somogyvari <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
Signed-off-by: Michal Bajer <[email protected]>
petermetz
added a commit
to petermetz/cacti
that referenced
this issue
Nov 10, 2023
…0-23 1. Couldn't get rid of vulnerable versions in a couple of dependencies because the underlying dependencies have gone ESM only which is a blocker for us at the moment unfortunately. 2. Swapped out the ubiquity TS client to a version of it that I self published onto npm after a full renovation of all of its dependencies. Depends on hyperledger-cacti#2807 (because that one also has a couple of dependency bumps that are needed to eliminate the vulnerabilities) Fixes hyperledger-cacti#2828 Signed-off-by: Peter Somogyvari <[email protected]>
petermetz
pushed a commit
to petermetz/cacti
that referenced
this issue
Nov 10, 2023
- Replace deprecated ipfs-http-client with kubo-rpc-client.
- kubo-rpc-client must be imported dynamically since it's ESM-only and we
still use CJS.
Peter's additional changes:
---------------------------
build(typescript): project-wide fixes to allow us to use ESM-only deps
Apologies for the huge diff, this can't be broken down to smaller changes
that would still compile because of cross-package dependencies.
I realize that this change is not exactly the optimal solution, but
it is probably a step in the right direction.
If I somehow found the time to submit pull requests to the libraries
that I needed to fork and re-publish (see details below) and then get
the changes onto the upstream and get them released as the official
packages, then we could (in theory) arrive at a solution that is the
recommended way of fixing these problems (apart from going full ESM-only)
This work stands on the shoulders of the previous commits from @outSH
and takes a slightly different direction compared to what we've been
talking about earlier on account of the problem that the eval-import
workaround causes crashes in Jest.
Based on the above I went through the following adventures:
1. I migrated the build system of kubo-rpc-client myself so that it
correctly exports CJS and ESM and typings for both of those as well,
I put that code on my fork [1] and then published it onto npm as well [2]
After this, I was hoping that now we could just import the package in
our CJS code without issues, but what really happened is that instead of
crashing at the require call that pull in kubo itself, it started crashing
deeper in the require stack where kubo itself was requiring it's own
ESM only dependencies (of which there seem to be at least 10 or 15).
At this point I realized that me migrating and self-publishing all of
these additional packages might not be worth the effort and started looking
for something easier.
2. I gave dynamic imports + moduleResultion=Node16 as my next attempt to
get our build back to working order. With this, the kubo-rpc-client
can now be imported dynamically without issues in packages that declare
themselves as resolving modules as "Node16" in their tsconfig.json
Other issues here were encountered because not all of our ESM only packages
are used in a way that they can be imported dynamically (for example
if their types are part of our own types or are being re-exported).
The two libraries with this problem were `run-time-error` and
`socket.io-client` for both of which I ended up going through the same
treatment as for kubo-rpc-client above (but this time my effort wasn't)
in vain. They work and so I did some search and replace in the entire
codebase to use these re-published packages with the correct types:
[3] [4] [5] [6]
3. After this the project build was working, but Jest was still failing with
compiler errors which I determined to happen because it uses the
root tsconfig.json file for it's internal TS compilation and that root
tsconfig.json file was not setting module resolution to Node16.
4. After fixing that the final hurdle (hopefully) was to ensure that jest gets
execued with the custom node option as below:
NODE_OPTIONS=--experimental-vm-modules yarn jest
[1] https://github.com/petermetz/js-kubo-rpc-client-esm-cjs
[2] https://www.npmjs.com/package/kubo-rpc-client-esm-cjs
[3] https://github.com/petermetz/socket.io-client
[4] https://www.npmjs.com/package/socket.io-client-fixed-types
[5] https://github.com/petermetz/RuntimeError
[6] https://www.npmjs.com/package/run-time-error-cjs
Huge thanks for https://arethetypeswrong.github.io/ a tool I used
extensively to create the fixes for the libraries above.
One more thing that I tried just to collect more data points was to
set the moduleResultion project-wide to Node16 via setting it
in the root tsconfig.base.json but this broke the compiler itself,
as in, there is a bug in the Typescript compiler in v4.x as seen here:
microsoft/TypeScript#51221
So this is one more reason for us to upgrade to 5.x as soon as possible.
I also needed to add "run-time-error" to the root package.json as a
dependency because it was accidentally providing that to some sub-packages
and when we moved to "run-time-error-cjs" the tests that install
plugins from npm started failing (because those releases are still
using "run-time-error" and not "run-time-error-cjs")
------------------------------------
Fixes hyperledger-cacti#2807
Fixes hyperledger-cacti#2852
Depends on: hyperledger-cacti#2821
Co-authored-by: Peter Somogyvari <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
Signed-off-by: Michal Bajer <[email protected]>
petermetz
added a commit
to petermetz/cacti
that referenced
this issue
Nov 10, 2023
…0-23 1. Couldn't get rid of vulnerable versions in a couple of dependencies because the underlying dependencies have gone ESM only which is a blocker for us at the moment unfortunately. 2. Swapped out the ubiquity TS client to a version of it that I self published onto npm after a full renovation of all of its dependencies. Depends on hyperledger-cacti#2807 (because that one also has a couple of dependency bumps that are needed to eliminate the vulnerabilities) Fixes hyperledger-cacti#2828 Signed-off-by: Peter Somogyvari <[email protected]>
petermetz
pushed a commit
to petermetz/cacti
that referenced
this issue
Nov 14, 2023
- Replace deprecated ipfs-http-client with kubo-rpc-client.
- kubo-rpc-client must be imported dynamically since it's ESM-only and we
still use CJS.
Peter's additional changes:
---------------------------
build(typescript): project-wide fixes to allow us to use ESM-only deps
Apologies for the huge diff, this can't be broken down to smaller changes
that would still compile because of cross-package dependencies.
I realize that this change is not exactly the optimal solution, but
it is probably a step in the right direction.
If I somehow found the time to submit pull requests to the libraries
that I needed to fork and re-publish (see details below) and then get
the changes onto the upstream and get them released as the official
packages, then we could (in theory) arrive at a solution that is the
recommended way of fixing these problems (apart from going full ESM-only)
This work stands on the shoulders of the previous commits from @outSH
and takes a slightly different direction compared to what we've been
talking about earlier on account of the problem that the eval-import
workaround causes crashes in Jest.
Based on the above I went through the following adventures:
1. I migrated the build system of kubo-rpc-client myself so that it
correctly exports CJS and ESM and typings for both of those as well,
I put that code on my fork [1] and then published it onto npm as well [2]
After this, I was hoping that now we could just import the package in
our CJS code without issues, but what really happened is that instead of
crashing at the require call that pull in kubo itself, it started crashing
deeper in the require stack where kubo itself was requiring it's own
ESM only dependencies (of which there seem to be at least 10 or 15).
At this point I realized that me migrating and self-publishing all of
these additional packages might not be worth the effort and started looking
for something easier.
2. I gave dynamic imports + moduleResultion=Node16 as my next attempt to
get our build back to working order. With this, the kubo-rpc-client
can now be imported dynamically without issues in packages that declare
themselves as resolving modules as "Node16" in their tsconfig.json
Other issues here were encountered because not all of our ESM only packages
are used in a way that they can be imported dynamically (for example
if their types are part of our own types or are being re-exported).
The two libraries with this problem were `run-time-error` and
`socket.io-client` for both of which I ended up going through the same
treatment as for kubo-rpc-client above (but this time my effort wasn't)
in vain. They work and so I did some search and replace in the entire
codebase to use these re-published packages with the correct types:
[3] [4] [5] [6]
3. After this the project build was working, but Jest was still failing with
compiler errors which I determined to happen because it uses the
root tsconfig.json file for it's internal TS compilation and that root
tsconfig.json file was not setting module resolution to Node16.
4. After fixing that the final hurdle (hopefully) was to ensure that jest gets
execued with the custom node option as below:
NODE_OPTIONS=--experimental-vm-modules yarn jest
[1] https://github.com/petermetz/js-kubo-rpc-client-esm-cjs
[2] https://www.npmjs.com/package/kubo-rpc-client-esm-cjs
[3] https://github.com/petermetz/socket.io-client
[4] https://www.npmjs.com/package/socket.io-client-fixed-types
[5] https://github.com/petermetz/RuntimeError
[6] https://www.npmjs.com/package/run-time-error-cjs
Huge thanks for https://arethetypeswrong.github.io/ a tool I used
extensively to create the fixes for the libraries above.
One more thing that I tried just to collect more data points was to
set the moduleResultion project-wide to Node16 via setting it
in the root tsconfig.base.json but this broke the compiler itself,
as in, there is a bug in the Typescript compiler in v4.x as seen here:
microsoft/TypeScript#51221
So this is one more reason for us to upgrade to 5.x as soon as possible.
I also needed to add "run-time-error" to the root package.json as a
dependency because it was accidentally providing that to some sub-packages
and when we moved to "run-time-error-cjs" the tests that install
plugins from npm started failing (because those releases are still
using "run-time-error" and not "run-time-error-cjs")
------------------------------------
Fixes hyperledger-cacti#2807
Fixes hyperledger-cacti#2852
Depends on: hyperledger-cacti#2821
Co-authored-by: Peter Somogyvari <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
Signed-off-by: Michal Bajer <[email protected]>
petermetz
added a commit
to petermetz/cacti
that referenced
this issue
Nov 14, 2023
…0-23 1. Couldn't get rid of vulnerable versions in a couple of dependencies because the underlying dependencies have gone ESM only which is a blocker for us at the moment unfortunately. 2. Swapped out the ubiquity TS client to a version of it that I self published onto npm after a full renovation of all of its dependencies. Depends on hyperledger-cacti#2807 (because that one also has a couple of dependency bumps that are needed to eliminate the vulnerabilities) Fixes hyperledger-cacti#2828 Signed-off-by: Peter Somogyvari <[email protected]>
petermetz
pushed a commit
to outSH/cactus
that referenced
this issue
Nov 15, 2023
- Replace deprecated ipfs-http-client with kubo-rpc-client.
- kubo-rpc-client must be imported dynamically since it's ESM-only and we
still use CJS.
Peter's additional changes:
---------------------------
build(typescript): project-wide fixes to allow us to use ESM-only deps
Apologies for the huge diff, this can't be broken down to smaller changes
that would still compile because of cross-package dependencies.
I realize that this change is not exactly the optimal solution, but
it is probably a step in the right direction.
If I somehow found the time to submit pull requests to the libraries
that I needed to fork and re-publish (see details below) and then get
the changes onto the upstream and get them released as the official
packages, then we could (in theory) arrive at a solution that is the
recommended way of fixing these problems (apart from going full ESM-only)
This work stands on the shoulders of the previous commits from @outSH
and takes a slightly different direction compared to what we've been
talking about earlier on account of the problem that the eval-import
workaround causes crashes in Jest.
Based on the above I went through the following adventures:
1. I migrated the build system of kubo-rpc-client myself so that it
correctly exports CJS and ESM and typings for both of those as well,
I put that code on my fork [1] and then published it onto npm as well [2]
After this, I was hoping that now we could just import the package in
our CJS code without issues, but what really happened is that instead of
crashing at the require call that pull in kubo itself, it started crashing
deeper in the require stack where kubo itself was requiring it's own
ESM only dependencies (of which there seem to be at least 10 or 15).
At this point I realized that me migrating and self-publishing all of
these additional packages might not be worth the effort and started looking
for something easier.
2. I gave dynamic imports + moduleResultion=Node16 as my next attempt to
get our build back to working order. With this, the kubo-rpc-client
can now be imported dynamically without issues in packages that declare
themselves as resolving modules as "Node16" in their tsconfig.json
Other issues here were encountered because not all of our ESM only packages
are used in a way that they can be imported dynamically (for example
if their types are part of our own types or are being re-exported).
The two libraries with this problem were `run-time-error` and
`socket.io-client` for both of which I ended up going through the same
treatment as for kubo-rpc-client above (but this time my effort wasn't)
in vain. They work and so I did some search and replace in the entire
codebase to use these re-published packages with the correct types:
[3] [4] [5] [6]
3. After this the project build was working, but Jest was still failing with
compiler errors which I determined to happen because it uses the
root tsconfig.json file for it's internal TS compilation and that root
tsconfig.json file was not setting module resolution to Node16.
4. After fixing that the final hurdle (hopefully) was to ensure that jest gets
execued with the custom node option as below:
NODE_OPTIONS=--experimental-vm-modules yarn jest
[1] https://github.com/petermetz/js-kubo-rpc-client-esm-cjs
[2] https://www.npmjs.com/package/kubo-rpc-client-esm-cjs
[3] https://github.com/petermetz/socket.io-client
[4] https://www.npmjs.com/package/socket.io-client-fixed-types
[5] https://github.com/petermetz/RuntimeError
[6] https://www.npmjs.com/package/run-time-error-cjs
Huge thanks for https://arethetypeswrong.github.io/ a tool I used
extensively to create the fixes for the libraries above.
One more thing that I tried just to collect more data points was to
set the moduleResultion project-wide to Node16 via setting it
in the root tsconfig.base.json but this broke the compiler itself,
as in, there is a bug in the Typescript compiler in v4.x as seen here:
microsoft/TypeScript#51221
So this is one more reason for us to upgrade to 5.x as soon as possible.
I also needed to add "run-time-error" to the root package.json as a
dependency because it was accidentally providing that to some sub-packages
and when we moved to "run-time-error-cjs" the tests that install
plugins from npm started failing (because those releases are still
using "run-time-error" and not "run-time-error-cjs")
------------------------------------
Fixes hyperledger-cacti#2807
Fixes hyperledger-cacti#2852
Depends on: hyperledger-cacti#2821
Co-authored-by: Peter Somogyvari <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
Signed-off-by: Michal Bajer <[email protected]>
petermetz
pushed a commit
that referenced
this issue
Nov 15, 2023
- Replace deprecated ipfs-http-client with kubo-rpc-client.
- kubo-rpc-client must be imported dynamically since it's ESM-only and we
still use CJS.
Peter's additional changes:
---------------------------
build(typescript): project-wide fixes to allow us to use ESM-only deps
Apologies for the huge diff, this can't be broken down to smaller changes
that would still compile because of cross-package dependencies.
I realize that this change is not exactly the optimal solution, but
it is probably a step in the right direction.
If I somehow found the time to submit pull requests to the libraries
that I needed to fork and re-publish (see details below) and then get
the changes onto the upstream and get them released as the official
packages, then we could (in theory) arrive at a solution that is the
recommended way of fixing these problems (apart from going full ESM-only)
This work stands on the shoulders of the previous commits from @outSH
and takes a slightly different direction compared to what we've been
talking about earlier on account of the problem that the eval-import
workaround causes crashes in Jest.
Based on the above I went through the following adventures:
1. I migrated the build system of kubo-rpc-client myself so that it
correctly exports CJS and ESM and typings for both of those as well,
I put that code on my fork [1] and then published it onto npm as well [2]
After this, I was hoping that now we could just import the package in
our CJS code without issues, but what really happened is that instead of
crashing at the require call that pull in kubo itself, it started crashing
deeper in the require stack where kubo itself was requiring it's own
ESM only dependencies (of which there seem to be at least 10 or 15).
At this point I realized that me migrating and self-publishing all of
these additional packages might not be worth the effort and started looking
for something easier.
2. I gave dynamic imports + moduleResultion=Node16 as my next attempt to
get our build back to working order. With this, the kubo-rpc-client
can now be imported dynamically without issues in packages that declare
themselves as resolving modules as "Node16" in their tsconfig.json
Other issues here were encountered because not all of our ESM only packages
are used in a way that they can be imported dynamically (for example
if their types are part of our own types or are being re-exported).
The two libraries with this problem were `run-time-error` and
`socket.io-client` for both of which I ended up going through the same
treatment as for kubo-rpc-client above (but this time my effort wasn't)
in vain. They work and so I did some search and replace in the entire
codebase to use these re-published packages with the correct types:
[3] [4] [5] [6]
3. After this the project build was working, but Jest was still failing with
compiler errors which I determined to happen because it uses the
root tsconfig.json file for it's internal TS compilation and that root
tsconfig.json file was not setting module resolution to Node16.
4. After fixing that the final hurdle (hopefully) was to ensure that jest gets
execued with the custom node option as below:
NODE_OPTIONS=--experimental-vm-modules yarn jest
[1] https://github.com/petermetz/js-kubo-rpc-client-esm-cjs
[2] https://www.npmjs.com/package/kubo-rpc-client-esm-cjs
[3] https://github.com/petermetz/socket.io-client
[4] https://www.npmjs.com/package/socket.io-client-fixed-types
[5] https://github.com/petermetz/RuntimeError
[6] https://www.npmjs.com/package/run-time-error-cjs
Huge thanks for https://arethetypeswrong.github.io/ a tool I used
extensively to create the fixes for the libraries above.
One more thing that I tried just to collect more data points was to
set the moduleResultion project-wide to Node16 via setting it
in the root tsconfig.base.json but this broke the compiler itself,
as in, there is a bug in the Typescript compiler in v4.x as seen here:
microsoft/TypeScript#51221
So this is one more reason for us to upgrade to 5.x as soon as possible.
I also needed to add "run-time-error" to the root package.json as a
dependency because it was accidentally providing that to some sub-packages
and when we moved to "run-time-error-cjs" the tests that install
plugins from npm started failing (because those releases are still
using "run-time-error" and not "run-time-error-cjs")
------------------------------------
Fixes #2807
Fixes #2852
Depends on: #2821
Co-authored-by: Peter Somogyvari <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
Signed-off-by: Michal Bajer <[email protected]>
github-project-automation
bot
moved this from In review
to Done
in Cacti_Scrum_Project_v2_Release
petermetz
added a commit
to petermetz/cacti
that referenced
this issue
Nov 15, 2023
…0-23 1. Couldn't get rid of vulnerable versions in a couple of dependencies because the underlying dependencies have gone ESM only which is a blocker for us at the moment unfortunately. 2. Swapped out the ubiquity TS client to a version of it that I self published onto npm after a full renovation of all of its dependencies. Depends on hyperledger-cacti#2807 (because that one also has a couple of dependency bumps that are needed to eliminate the vulnerabilities) Fixes hyperledger-cacti#2828 Signed-off-by: Peter Somogyvari <[email protected]>
petermetz
added a commit
to petermetz/cacti
that referenced
this issue
Nov 15, 2023
…0-23 1. Couldn't get rid of vulnerable versions in a couple of dependencies because the underlying dependencies have gone ESM only which is a blocker for us at the moment unfortunately. 2. Swapped out the ubiquity TS client to a version of it that I self published onto npm after a full renovation of all of its dependencies. Depends on hyperledger-cacti#2807 (because that one also has a couple of dependency bumps that are needed to eliminate the vulnerabilities) Fixes hyperledger-cacti#2828 Signed-off-by: Peter Somogyvari <[email protected]>
petermetz
added a commit
that referenced
this issue
Nov 15, 2023
…0-23 1. Couldn't get rid of vulnerable versions in a couple of dependencies because the underlying dependencies have gone ESM only which is a blocker for us at the moment unfortunately. 2. Swapped out the ubiquity TS client to a version of it that I self published onto npm after a full renovation of all of its dependencies. Depends on #2807 (because that one also has a couple of dependency bumps that are needed to eliminate the vulnerabilities) Fixes #2828 Signed-off-by: Peter Somogyvari <[email protected]>
sandeepnRES
pushed a commit
to sandeepnRES/cacti
that referenced
this issue
Dec 21, 2023
This is a workaround for the problem Peter intorduced in the build with an earlier commit where he upgraded axios to 1.5.1 universally. The above had lead to a problem with the nano package, which is a dependency of the fabric node SDK packages. The longer term solution is to migrate to the newer Fabric Gateway client SDK and completely remove the older fabric dependencies from the project, but until we can do that (a big undertaking) we have to be content with this shorter term workaround and hope that axios 0.27.2 does not turn out to be vulnerable critically (because that would force our hand with upgrades again). Partially addresses hyperledger-cacti#2807 (not a full fix) Signed-off-by: Peter Somogyvari <[email protected]>
sandeepnRES
pushed a commit
to sandeepnRES/cacti
that referenced
this issue
Dec 21, 2023
- Replace deprecated ipfs-http-client with kubo-rpc-client.
- kubo-rpc-client must be imported dynamically since it's ESM-only and we
still use CJS.
Peter's additional changes:
---------------------------
build(typescript): project-wide fixes to allow us to use ESM-only deps
Apologies for the huge diff, this can't be broken down to smaller changes
that would still compile because of cross-package dependencies.
I realize that this change is not exactly the optimal solution, but
it is probably a step in the right direction.
If I somehow found the time to submit pull requests to the libraries
that I needed to fork and re-publish (see details below) and then get
the changes onto the upstream and get them released as the official
packages, then we could (in theory) arrive at a solution that is the
recommended way of fixing these problems (apart from going full ESM-only)
This work stands on the shoulders of the previous commits from @outSH
and takes a slightly different direction compared to what we've been
talking about earlier on account of the problem that the eval-import
workaround causes crashes in Jest.
Based on the above I went through the following adventures:
1. I migrated the build system of kubo-rpc-client myself so that it
correctly exports CJS and ESM and typings for both of those as well,
I put that code on my fork [1] and then published it onto npm as well [2]
After this, I was hoping that now we could just import the package in
our CJS code without issues, but what really happened is that instead of
crashing at the require call that pull in kubo itself, it started crashing
deeper in the require stack where kubo itself was requiring it's own
ESM only dependencies (of which there seem to be at least 10 or 15).
At this point I realized that me migrating and self-publishing all of
these additional packages might not be worth the effort and started looking
for something easier.
2. I gave dynamic imports + moduleResultion=Node16 as my next attempt to
get our build back to working order. With this, the kubo-rpc-client
can now be imported dynamically without issues in packages that declare
themselves as resolving modules as "Node16" in their tsconfig.json
Other issues here were encountered because not all of our ESM only packages
are used in a way that they can be imported dynamically (for example
if their types are part of our own types or are being re-exported).
The two libraries with this problem were `run-time-error` and
`socket.io-client` for both of which I ended up going through the same
treatment as for kubo-rpc-client above (but this time my effort wasn't)
in vain. They work and so I did some search and replace in the entire
codebase to use these re-published packages with the correct types:
[3] [4] [5] [6]
3. After this the project build was working, but Jest was still failing with
compiler errors which I determined to happen because it uses the
root tsconfig.json file for it's internal TS compilation and that root
tsconfig.json file was not setting module resolution to Node16.
4. After fixing that the final hurdle (hopefully) was to ensure that jest gets
execued with the custom node option as below:
NODE_OPTIONS=--experimental-vm-modules yarn jest
[1] https://github.com/petermetz/js-kubo-rpc-client-esm-cjs
[2] https://www.npmjs.com/package/kubo-rpc-client-esm-cjs
[3] https://github.com/petermetz/socket.io-client
[4] https://www.npmjs.com/package/socket.io-client-fixed-types
[5] https://github.com/petermetz/RuntimeError
[6] https://www.npmjs.com/package/run-time-error-cjs
Huge thanks for https://arethetypeswrong.github.io/ a tool I used
extensively to create the fixes for the libraries above.
One more thing that I tried just to collect more data points was to
set the moduleResultion project-wide to Node16 via setting it
in the root tsconfig.base.json but this broke the compiler itself,
as in, there is a bug in the Typescript compiler in v4.x as seen here:
microsoft/TypeScript#51221
So this is one more reason for us to upgrade to 5.x as soon as possible.
I also needed to add "run-time-error" to the root package.json as a
dependency because it was accidentally providing that to some sub-packages
and when we moved to "run-time-error-cjs" the tests that install
plugins from npm started failing (because those releases are still
using "run-time-error" and not "run-time-error-cjs")
------------------------------------
Fixes hyperledger-cacti#2807
Fixes hyperledger-cacti#2852
Depends on: hyperledger-cacti#2821
Co-authored-by: Peter Somogyvari <[email protected]>
Signed-off-by: Peter Somogyvari <[email protected]>
Signed-off-by: Michal Bajer <[email protected]>
sandeepnRES
pushed a commit
to sandeepnRES/cacti
that referenced
this issue
Dec 21, 2023
…0-23 1. Couldn't get rid of vulnerable versions in a couple of dependencies because the underlying dependencies have gone ESM only which is a blocker for us at the moment unfortunately. 2. Swapped out the ubiquity TS client to a version of it that I self published onto npm after a full renovation of all of its dependencies. Depends on hyperledger-cacti#2807 (because that one also has a couple of dependency bumps that are needed to eliminate the vulnerabilities) Fixes hyperledger-cacti#2828 Signed-off-by: Peter Somogyvari <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
@petermetz @izuru0 @jagpreetsinghsasan
Describe the bug
After recent dependency bumps many tests are failing in CI - see #2805. I confirmed some of these errors on main branch (run locally).
TLDR I'd propose to rollback the following commits and investigate how to bump these packages without breaking the CI:
As for now I've identified the following issues:
iroha-helpers
This is caused by invalid dependency in iroha-helpers package - after we upgraded to grpc-js it can't find grpc. I've opened an issue in their repository and proposed to add grpc dependency.
Axios
I think this one is caused by axios-cookiejar-support that lists axios as peer depdendency
"axios": ">=0.16.2",(so matches 1.* we use) but it should list only 0.* releases. I'll open an issue in their repo once I confirm this.Axios 2
ipfs-http-client
The text was updated successfully, but these errors were encountered: