Duplicate chunks throw an error #51
Description
If you setContent with repetitive data on a collection, an error is thrown when trying to get that content again.
TypeError: Cannot read properties of undefined (reading 'length')
at reduce (/etebase-js/src/Crypto.ts:36:49)
at Array.reduce (<anonymous>)
at concatArrayBuffersArrays (/etebase-js/src/Crypto.ts:36:26)
...
It looks like the index list and/or chunks created here are the issue (the indices are pointers at the original chunks array, but the chunks array is mutated here, so the pointers are incorrect?)
The following tests illustrate the issue (large files are fine, small files are fine, but repetitive data (that would result in chunks that would be deduplicated) throws the error
describe.only("chunking files", () => {
it("Duplicate Chunks", async () => {
const collectionManager = etebase.getCollectionManager();
const col = await collectionManager.create(colType, {}, "");
const buf = randomBytesDeterministic(10 * 1024, new Uint8Array(32)); // 10kb of psuedorandom data
const content = JSON.stringify([buf, buf, buf, buf]);
await col.setContent(content);
await collectionManager.transaction(col);
const decryptedContent = await col.getContent(); // <---
/*
TypeError: Cannot read properties of undefined (reading 'length')
at reduce (/etebase-js/src/Crypto.ts:36:49)
at Array.reduce (<anonymous>)
at concatArrayBuffersArrays (/etebase-js/src/Crypto.ts:36:26)
...
*/
const out = to_string(decryptedContent);
expect(out).toEqual(content);
});
it("Regular Chunks", async () => {
const collectionManager = etebase.getCollectionManager();
const col = await collectionManager.create(colType, {}, "");
const buf = randomBytesDeterministic(100 * 1024, new Uint8Array(32));
const content = JSON.stringify(buf);
await col.setContent(content);
await collectionManager.transaction(col);
const decryptedContent = await col.getContent();
const out = to_string(decryptedContent);
expect(out).toEqual(content);
});
it("Small file, no chunks", async () => {
const collectionManager = etebase.getCollectionManager();
const col = await collectionManager.create(colType, {}, "");
const buf = randomBytesDeterministic(10, new Uint8Array(32));
const content = JSON.stringify(buf);
await col.setContent(content);
await collectionManager.transaction(col);
const decryptedContent = await col.getContent();
const out = to_string(decryptedContent);
expect(out).toEqual(content);
});
});I'll do a PR with a fix shortly