feat: storeDirectory accepts iter of file objects #1924
Conversation
gobengo
force-pushed
the
storeDirectoryFileObject
branch
from
3834c60 to
4aac6a2
Compare
Deploying with
|
| Latest commit: |
a668f71
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://beaff0aa.nft-storage-1at.pages.dev |
Codecov Report
@@ Coverage Diff @@
## main #1924 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 5 5
Lines 1269 1259 -10
=========================================
- Hits 1269 1259 -10
Continue to review full report at Codecov.
|
gobengo
changed the title
packages/client/src/lib/interface.ts
Outdated
| stream: () => AsyncIterable<any> | ||
| } | ||
|
|
||
| export type FilesSource = Iterable<File>|Iterable<FileObject>|AsyncIterable<File>|AsyncIterable<FileObject> |
There was a problem hiding this comment.
Definitely worth adding the type alias 😄
packages/client/src/lib.js
Outdated
| const file = | ||
| fileObject instanceof File | ||
| ? fileObject | ||
| : new File(await all(fileObject.stream()), fileObject.name) |
There was a problem hiding this comment.
Do we need to await the stream here? It would be cool if we could just pass in [fileObject.stream()]
There was a problem hiding this comment.
I think that toImportCandidate inside of encodeDirectory will accept anything with a stream() method that returns an async iterable. So you may be able to skip the coercion to File here and just pass everything through.
Unless you tried that already and it didn't work, lol
There was a problem hiding this comment.
it wasn't so simple. I didn't want to augment more functions than needed, but insofar as somethings were typed as wanting File, this required buffering. afaict constructing a dom File requires not just a stream but a fully-read Uint8Array.
I just pushed a change though that allows the FileObject to encodeDirectory and makes this code less ugly.
… before passing to encodeDirectory, which now takes a FilesSource
| @@ -758,10 +747,11 @@ const decodePin = (pin) => ({ ...pin, created: new Date(pin.created) }) | |||
| * the stream is created only when needed. | |||
| * | |||
| * @param {string} path | |||
| * @param {Blob} blob | |||
| * @param {Pick<Blob, 'stream'>|{ stream: () => AsyncIterable<Uint8Array> }} blob | |||
There was a problem hiding this comment.
I really like this type definition 😄 - I should start using Pick and Omit more...
There was a problem hiding this comment.
yeah :) it would be cool if there was something like an eslint rule that helped you specify your inputs as 'narrowly'/minimally as possible. e.g. in this case we didn't really need a whole Blob, just a subset
| */ | ||
| function toImportCandidate(path, blob) { | ||
| /** @type {ReadableStream} */ | ||
| /** @type {AsyncIterable<Uint8Array>} */ |
There was a problem hiding this comment.
yeah its so cool how node made stream.Readables implement AsyncIterable
Motivation:
files-from-path