-
Notifications
You must be signed in to change notification settings - Fork 1k
Open
Labels
t-toolingIssues with this label are in the ownership of the tooling team.Issues with this label are in the ownership of the tooling team.
Milestone
Description
- follow up to https://apify.slack.com/archives/C02JQSN79V4/p1759231330021089
- Python
FileSystemKeyValueStoreClient
currently does not automatically append extensions to the file names (keys) - In JS, it does -
crawlee/packages/memory-storage/src/resource-clients/key-value-store.ts
Lines 283 to 302 in 8c8e1f4
const isValueStreamOrBuffer = valueIsStream || isBuffer(value); // To allow saving Objects to JSON without providing content type if (!contentType) { if (isValueStreamOrBuffer) contentType = 'application/octet-stream'; else if (typeof value === 'string') contentType = 'text/plain; charset=utf-8'; else contentType = 'application/json; charset=utf-8'; } const extension = mime.extension(contentType) || DEFAULT_LOCAL_FILE_EXTENSION; const isContentTypeJson = extension === 'json'; if (isContentTypeJson && !isValueStreamOrBuffer && typeof value !== 'string') { try { value = JSON.stringify(value, null, 2); } catch (err: any) { const msg = `The record value cannot be stringified to JSON. Please provide other content type.\nCause: ${err.message}`; throw new Error(msg); } } - This behavior needs to be unified and documented in both flavors
- There was a concern that the Apify Key-value store does not accept extensions in the keys, but this has been proven false
- in fact, many well-known Actors specify the extensions explicitly
- Adding extensions automatically on the other hand can be considered user friendly for those who want to open the "values" using common editors/viewers
vdusek
Metadata
Metadata
Assignees
Labels
t-toolingIssues with this label are in the ownership of the tooling team.Issues with this label are in the ownership of the tooling team.