Skip to content

Decide and document how file name extensions are handled in MemoryStorage KeyValueStore #3189

@janbuchar

Description

@janbuchar
  • follow up to https://apify.slack.com/archives/C02JQSN79V4/p1759231330021089
  • Python FileSystemKeyValueStoreClient currently does not automatically append extensions to the file names (keys)
  • In JS, it does -
    const isValueStreamOrBuffer = valueIsStream || isBuffer(value);
    // To allow saving Objects to JSON without providing content type
    if (!contentType) {
    if (isValueStreamOrBuffer) contentType = 'application/octet-stream';
    else if (typeof value === 'string') contentType = 'text/plain; charset=utf-8';
    else contentType = 'application/json; charset=utf-8';
    }
    const extension = mime.extension(contentType) || DEFAULT_LOCAL_FILE_EXTENSION;
    const isContentTypeJson = extension === 'json';
    if (isContentTypeJson && !isValueStreamOrBuffer && typeof value !== 'string') {
    try {
    value = JSON.stringify(value, null, 2);
    } catch (err: any) {
    const msg = `The record value cannot be stringified to JSON. Please provide other content type.\nCause: ${err.message}`;
    throw new Error(msg);
    }
    }
  • This behavior needs to be unified and documented in both flavors
  • There was a concern that the Apify Key-value store does not accept extensions in the keys, but this has been proven false
    • in fact, many well-known Actors specify the extensions explicitly
  • Adding extensions automatically on the other hand can be considered user friendly for those who want to open the "values" using common editors/viewers

Metadata

Metadata

Assignees

No one assigned

    Labels

    t-toolingIssues with this label are in the ownership of the tooling team.

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions