关于增加一个环境变量，配置是否删除知识库内容图片

[kenxt]

最近测试时发现知识库的图片过一天就被清理了，使用起来很不方便，建议增加一个配置选项，本地测试可行，在在docker-compose.ymll中增加变量配置：
ALLOW_IMAGE_DELETION = false

修改两处代码：

packages/service/core/dataset/image/controller.ts

import { Types } from 'mongoose';
import { getGridBucket, addLog } from '../utils';
import * as process from 'process';

// 读取环境变量
const ALLOW_IMAGE_DELETE = process.env.ALLOW_IMAGE_DELETION === 'true';

export const deleteDatasetImage = async (imageId: string) => {
  if (!ALLOW_IMAGE_DELETE) {
    addLog.info('Image deletion is disabled by environment variable.');
    return;
  }
  const gridBucket = getGridBucket();

  try {
    await gridBucket.delete(new Types.ObjectId(imageId));
  } catch (error: any) {
    const msg = error?.message;
    if (msg.includes('File not found')) {
      addLog.warn('Delete dataset image error', error);
      return;
    } else {
      return Promise.reject(error);
    }
  }
};

packages/service/core/dataset/collection/controller.ts

import { retryFn, mongoSessionRun, delImgByRelatedId, delFileByFileIdList, deleteDatasetDataVector, clearCollectionImages } from '../utils';
import { DatasetCollectionSchemaType } from '../schema';
import { ClientSession } from 'mongoose';
import * as process from 'process';

// 读取环境变量
const ALLOW_IMAGE_DELETE = process.env.ALLOW_IMAGE_DELETION === 'true';

/**
 * delete collection and it related data
 */
export async function delCollection({
  collections,
  session,
  delImg = true,
  delFile = true
}: {
  collections: DatasetCollectionSchemaType[];
  session: ClientSession;
  delImg: boolean;
  delFile: boolean;
}) {
  if (collections.length === 0) return;

  const teamId = collections[0].teamId;

  if (!teamId) return Promise.reject('teamId is not exist');

  const datasetIds = Array.from(new Set(collections.map((item) => String(item.datasetId))));
  const collectionIds = collections.map((item) => String(item._id));

  await retryFn(async () => {
    await Promise.all([
      // Delete training data
      MongoDatasetTraining.deleteMany({
        teamId,
        datasetId: { $in: datasetIds },
        collectionId: { $in: collectionIds }
      }),
      // Delete dataset_data_texts
      MongoDatasetDataText.deleteMany({
        teamId,
        datasetId: { $in: datasetIds },
        collectionId: { $in: collectionIds }
      }),
      // Delete dataset_datas
      MongoDatasetData.deleteMany({
        teamId,
        datasetId: { $in: datasetIds },
        collectionId: { $in: collectionIds }
      }),
      // Delete dataset_images
      clearCollectionImages(collectionIds),
      // Delete images if needed and allowed
      ...(delImg && ALLOW_IMAGE_DELETE
        ? [
            delImgByRelatedId({
              teamId,
              relateIds: collections
                .map((item) => item?.metadata?.relatedImgId || '')
                .filter(Boolean)
            })
          ]
        : []),

      // Delete files if needed
      ...(delFile
        ? [
            delFileByFileIdList({
              bucketName: BucketNameEnum.dataset,
              fileIdList: collections.map((item) => item?.fileId || '').filter(Boolean)
            })
          ]
        : []),
      // Delete vector data
      deleteDatasetDataVector({ teamId, datasetIds, collectionIds })
    ]);

    // delete collections
    await MongoDatasetCollection.deleteMany(
      {
        teamId,
        _id: { $in: collectionIds }
      },
      { session }
    );
  });
}

只是测试使用，没有进行过多验证

[c121914yu]

上传成功就不会删除

[kenxt]

4.9.11存在这个现象，具体原因未知，下面是6月17日重新训练的一个内容，完成后有图片，18日早上看图片没有了，不知道什么原因，上传成功的状态应该如何确认？

[c121914yu]

4.9.11存在这个现象，具体原因未知，下面是6月17日重新训练的一个内容，完成后有图片，18日早上看图片没有了，不知道什么原因，上传成功的状态应该如何确认？

看最新 release，有修复文档里图片问题。
但是你主题描述的问题，和你现在这个问题不是同一个内容。

[kenxt]

看到了，再测一下。
不知道删除机制，改代码是想保留图片