DataCloneError under fake-IndexedDB #647
Comments
|
Are you using the latest versions of dexie and dexie-observable and dexie-syncable (in case you are using it)? I don't remember having an issue in the browser and I think I had fixed dexie-syncable so the save method is not written into the db. |
|
these are the versions I'm using: |
|
I believe this is an issue with Edit: reading a bit more through the code, I have no real clue what is going on.. Can you check in the browser if the sync nodes have a save method when saved in the db? |
|
using Chrome the _syncNodes db has an object that does not have a |
|
Although Chrome allows it, I'm not convinced that the spec does. |
|
We are saving We could also copy the object but I would rather fix the real issue instead of just implementing something. |
|
The code in question is not the Dexie.Syncable }).then(()=>{
// Add our node to DB and start subscribing to events
return db._syncNodes.add(/*>>>>>*/mySyncNode.node /*<<<<<*/).then(function() {
Observable.on('latestRevisionIncremented', onLatestRevisionIncremented); // Wakeup when a new revision is available.
Observable.on('beforeunload', onBeforeUnload);
Observable.on('suicideNurseCall', onSuicide);
Observable.on('intercomm', onIntercomm);
// Start polling for changes and do cleanups:
pollHandle = setTimeout(poll, LOCAL_POLL);
// Start heartbeat
heartbeatHandle = setTimeout(heartbeat, HEARTBEAT_INTERVAL);
}); |
|
Hm okay, just got caught up in dexie-syncable because that is the code actually defining the From the isPlainObject docs: From that I would assume the check fails because we are using a class. From my understanding of the spec, what we are doing in dexie-observable is allowed, which would mean the issue is with fake-indexeddb. Cloning the SyncNode object and saving that should be possible but assuming that we are doing everything correct currently, I don't feel comfortable deciding this change. I would rather have @dfahlander have a look at this and let us know if cloning would work just as good. |
|
There is nothing in the structured clone algorithm that forbids objects with __proto pointing to other than Object.prototype to be cloned (https://www.w3.org/TR/html52/single-page.html#structuredserializeforstorage). Normal objects should be clonable but their prototype chain will not be preserved (https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm). However, putting a function property directly on the object should not be supported. But in this case, the save() method lies in the prototype. |
|
The issue will persist for anyone storing their classes into Dexie. According to IndexedDB specification, refering to HTML5 spec structuredserializeforstorage, refering to structuredserialize_internal, this should be possible. If we do a fix in Dexie.Observable, people storing classes in their app code will still suffer. So the fix should instead be done within realistic-structured-clone somewhere at this location by allowing objects with custom prototypes. An alternative would also be for fake-indexedDB to replace realistic-structured-clone with typeson, which is developed by me and @brettz9 and used in IndexedDBShim for the same purpose. If so, use the structured-cloning-throwing preset from typeson-registry |
|
Any updates on this and any workarounds? Just ran into this issue on None of my objects have a My error looks like: |
|
As of my understanding, fixes have been made in both fake-indexeddb and typeson-registry. I think we should close this original issue now, is that right, @dumbmatter / @brettz9? If so, I think @stevenxxiu could have other reason than this original issue. |
|
Yeah, fake-indexeddb now uses typeson, so this is either a legitimately invalid object (containing a function or something) or a bug in typeson. I'd guess the former is more likely, but who knows :) |
|
Yes, we'd need more info on the triggering object. There could also be some built-in type that is potentially serializable but for which we have no encapsulator in the registry. |
|
Ahh the issue is due to import Dexie from 'dexie'
import indexedDB from 'fake-indexeddb'
import FDBKeyRange from 'fake-indexeddb/lib/FDBKeyRange'
describe('DeflateWriter', () => {
test('dummy', async () => {
const db = new Dexie('recordings', {indexedDB: indexedDB, IDBKeyRange: FDBKeyRange})
db.version(1).stores({files: 'name'})
// this works
await db.files.put({name: 'file.bin', data: [new Uint8Array([1, 2, 3])]})
// this doesn't
await db.files.put({name: 'file.bin', data: new Blob([new Uint8Array([1, 2, 3])])})
})
}) |
|
It appears to me that the issue may be that I would see about patching |
|
I would also be sure to use the latest |
|
It already uses the latest typeson-registry, version 1.0.0-alpha.28. The problem seems to be that typeson-registry assumes the presence of various web APIs when processing Blobs. Like in this code: You get: If you put all the jsdom stuff in global scope with something like I didn't go further down the rabbit hole of polyfilling web APIs, but maybe it's possible to make it work. It'd be nice if it worked more easily, though. I don't know enough about Blobs to say if the problem lies in typeson-registry or jsdom, or if there's just no good solution. It's definitely not Dexie's fault though :) |
|
@dumbmatter , in looking at https://github.com/dumbmatter/realistic-structured-clone/blob/master/package.json#L32-L33 (in |
|
The ^ means it gets versions 5.13.0 and 1.0.0-alpha.28 if you install it today. |
|
Oops, I knew that. :) (Sorry, am too used to using If @stevenxxiu is actually using the latest versions, including the dependent typeson/typeson-registry, it sounds like the issue is being reported by Safari/iOS and somehow typeson is attempting to clone (i.e., something did not get serialized as it should have). But the latest typeson has no problem with the structure given, so I'm thinking he's probably using an outdated version. |
|
I think I'm using latest versions, just did a The error has an issue at jsdom/jsdom#1721. I tried but couldn't fix it. |
|
If there is an issue in typeson-registry, it should be reproducible in typeson-registry. (You can confirm the versions you are using, @stevenxxiu , by looking in |
|
As far as |
|
Tried that just now. For some reason |
|
Yes, using From Node, you should be able to define a global The |
|
Thanks so much, that got it working! I would expect getting this to work in |
|
Almost a year later I've come back to this. I still couldn't get it to work in Jest after trying for quite a while now. The simplifed version of the test I'm trying ot run in import Dexie from 'dexie'
import IndexedDB from 'fake-indexeddb'
import FDBKeyRange from 'fake-indexeddb/lib/FDBKeyRange'
test('test', async () => {
const db = new Dexie('recordings', {indexedDB: IndexedDB, IDBKeyRange: FDBKeyRange})
db.version(1).stores({files: 'name'})
const table = db.files
const blob = new Blob([new Uint8Array([1, 2, 3])])
await table.put({name: 'file.bin', data: blob})
})I got the following working in const jsdom = require('jsdom')
const Typeson = require('typeson')
const structuredCloningThrowing = require('typeson-registry/dist/presets/structured-cloning-throwing')
const { JSDOM } = jsdom
const { window } = new JSDOM('', {
url: 'https://example.com/',
})
Object.assign(global, window)
global.XMLHttpRequest = window.XMLHttpRequest
const { createObjectURL, xmlHttpRequestOverrideMimeType} = require('typeson-registry/polyfills/createObjectURL-cjs.js')
URL.createObjectURL = createObjectURL
XMLHttpRequest.prototype.overrideMimeType = xmlHttpRequestOverrideMimeType()
const data = new window.Blob([new Uint8Array([1, 2, 3])])
console.log(data)
const TSON = new Typeson().register(structuredCloningThrowing)
const encapsulated = TSON.encapsulate(data)
console.log(encapsulated)In @brettz9 Any pointers? Much appreciated. |
|
@stevenxxiu : |
|
My code is now the following: import Dexie from 'dexie'
import IndexedDB from 'fake-indexeddb'
import FDBKeyRange from 'fake-indexeddb/lib/FDBKeyRange'
import { JSDOM } from 'jsdom'
import { createObjectURL, xmlHttpRequestOverrideMimeType} from 'typeson-registry/polyfills/createObjectURL-cjs.js'
test('test', async () => {
const { window } = new JSDOM('', {
url: 'https://example.com/',
})
global.XMLHttpRequest = window.XMLHttpRequest
URL.createObjectURL = createObjectURL
XMLHttpRequest.prototype.overrideMimeType = xmlHttpRequestOverrideMimeType()
const db = new Dexie('recordings', {indexedDB: IndexedDB, IDBKeyRange: FDBKeyRange})
db.version(1).stores({files: 'name'})
const table = db.files
const blob = new window.Blob([new Uint8Array([1, 2, 3])])
await table.put({name: 'file.bin', data: blob})
})
|
|
That function is used inside typeson-registry' When running in Node, typeson-registry needs the polyfills you have added for |
|
Ok I tried IndexedDBShim with the code: import Dexie from 'dexie'
import setGlobalVars from 'indexeddbshim'
import { JSDOM } from 'jsdom'
import { createObjectURL, xmlHttpRequestOverrideMimeType} from 'typeson-registry/polyfills/createObjectURL-cjs.js'
test('test', async () => {
const { window } = new JSDOM('', {
url: 'https://example.com/',
})
setGlobalVars(window)
global.XMLHttpRequest = window.XMLHttpRequest
URL.createObjectURL = createObjectURL
XMLHttpRequest.prototype.overrideMimeType = xmlHttpRequestOverrideMimeType()
const db = new Dexie('recordings', {indexedDB: window.shimIndexedDB, IDBKeyRange: window.IDBKeyRange})
db.version(1).stores({files: 'name'})
const table = db.files
const blob = new window.Blob([new Uint8Array([1, 2, 3])])
await table.put({name: 'file.bin', data: blob})
})I'm getting the error: Not really sure how to proceed. |
|
Try pegging jsdom for now to For further discussion on this issue though, why don't you file an issue at https://github.com/indexeddbshim/indexeddbshim so we can discuss there, as this isn't really related to this issue anymore. (Also, note that the package is |
I'm running under node (for unit testing) with
fake-indexeddbanddexie-observableI get a'DataCloneError'("The data being stored could not be cloned by the internal structured cloning algorithm.") because instartObserving(Dexie.Observable.js:255 --db._syncNodes.add(mySyncNode.node)) the object being saved has a 'save' method and failsisPlainObjectinrealistic-structured-clone:The text was updated successfully, but these errors were encountered: