Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
147 changes: 147 additions & 0 deletions packages/s3-store/IMPLEMENTATION_SUMMARY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
# S3 Object Prefix Implementation - Summary

## Changes Made

I've successfully implemented the S3 object prefix feature that allows you to organize uploaded files in folder structures within your S3 bucket.

### Modified Files

**`/packages/s3-store/src/index.ts`**

1. **Added `objectPrefix` option to the `Options` type** (line ~49):
```typescript
objectPrefix?: string
```
- Optional parameter
- Used to create pseudo-directory structures in S3
- Example: `"uploads/"` or `"my-app/files/2024/"`

2. **Added `objectPrefix` class property** (line ~109):
```typescript
protected objectPrefix: string
```

3. **Initialize `objectPrefix` in constructor** (line ~116):
```typescript
this.objectPrefix = objectPrefix ?? ''
```
- Defaults to empty string if not provided (maintains backward compatibility)

4. **Updated `infoKey()` method** (line ~224):
```typescript
protected infoKey(id: string) {
return `${this.objectPrefix}${id}.info`
}
```

5. **Updated `partKey()` method** (line ~228):
```typescript
protected partKey(id: string, isIncomplete = false) {
if (isIncomplete) {
id += '.part'
}
return `${this.objectPrefix}${id}`
}
```

6. **Updated all S3 operations to use the prefix**:
- `uploadPart()` - Uses `this.partKey(metadata.file.id)`
- `create()` - Uses `this.partKey(upload.id)` for Key
- `read()` - Uses `this.partKey(id)`
- `finishMultipartUpload()` - Uses `this.partKey(metadata.file.id)`
- `retrieveParts()` - Uses `this.partKey(id)`
- `remove()` - Uses `this.partKey(id)` and `this.infoKey(id)`

### Created Files

**`/packages/s3-store/OBJECT_PREFIX_EXAMPLE.md`**
- Complete usage documentation
- Before/after examples showing folder structures
- Multiple use cases (single folder, nested folders)

## How It Works

### Without objectPrefix (default behavior):
```
my-bucket/
├── file-abc123
├── file-abc123.info
├── file-xyz789
└── file-xyz789.info
```

### With objectPrefix: 'uploads/':
```
my-bucket/
└── uploads/
├── file-abc123
├── file-abc123.info
├── file-xyz789
└── file-xyz789.info
```

### With objectPrefix: 'my-app/uploads/2024/':
```
my-bucket/
└── my-app/
└── uploads/
└── 2024/
├── file-abc123
├── file-abc123.info
├── file-xyz789
└── file-xyz789.info
```

## Usage Example

```typescript
import {S3Store} from '@tus/s3-store';

const store = new S3Store({
objectPrefix: 'uploads/', // Add this line!

s3ClientConfig: {
bucket: 'my-bucket',
region: 'us-east-1',
credentials: {
accessKeyId: 'YOUR_ACCESS_KEY',
secretAccessKey: 'YOUR_SECRET_KEY',
},
},
});

// Now all uploads will be stored in the 'uploads/' folder
```

## Backward Compatibility

✅ **Fully backward compatible** - If `objectPrefix` is not provided, it defaults to an empty string, maintaining the current behavior of storing files at the bucket root.

## Testing Recommendations

1. Test with no prefix (default behavior)
2. Test with single folder: `'uploads/'`
3. Test with nested folders: `'my-app/uploads/2024/'`
4. Test with prefix without trailing slash (should still work)
5. Verify all operations work:
- Upload files
- Resume uploads
- Read files
- Delete files
- List expired files

## Benefits

- **Organization**: Keep uploads organized in logical folder structures
- **Multi-tenant**: Separate uploads by user/tenant: `'users/user-123/uploads/'`
- **Time-based**: Organize by date: `'uploads/2024/10/'`
- **Environment separation**: Different folders for dev/staging/prod
- **Easier lifecycle policies**: Apply S3 lifecycle rules to specific prefixes
- **Better billing analysis**: Track storage costs by prefix

## Notes

- The prefix applies to all S3 objects: main files, `.info` metadata files, and `.part` incomplete files
- Make sure to include trailing slash for folder-like structures
- S3 doesn't have real folders - prefixes create a "pseudo-directory" structure
- All existing code continues to work without any changes
66 changes: 66 additions & 0 deletions packages/s3-store/OBJECT_PREFIX_EXAMPLE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# S3 Object Prefix Example

The S3Store now supports an `objectPrefix` option that allows you to organize your uploads in a folder structure within your S3 bucket.

## Usage

```typescript
import {S3Store} from '@tus/s3-store';

const store = new S3Store({
// Specify a folder path for your uploads
objectPrefix: 'uploads/', // Files will be stored as: uploads/<file-id>

// Or create nested folders
// objectPrefix: 'my-app/uploads/2024/', // Files: my-app/uploads/2024/<file-id>

s3ClientConfig: {
bucket: 'my-bucket',
region: 'us-east-1',
credentials: {
accessKeyId: 'your-access-key',
secretAccessKey: 'your-secret-key',
},
},
});
```

## Before and After

### Without `objectPrefix` (default):
```
my-bucket/
├── file-id-1
├── file-id-1.info
├── file-id-2
└── file-id-2.info
```

### With `objectPrefix: 'uploads/'`:
```
my-bucket/
└── uploads/
├── file-id-1
├── file-id-1.info
├── file-id-2
└── file-id-2.info
```

### With `objectPrefix: 'my-app/uploads/2024/'`:
```
my-bucket/
└── my-app/
└── uploads/
└── 2024/
├── file-id-1
├── file-id-1.info
├── file-id-2
└── file-id-2.info
```

## Notes

- The prefix is optional - if not provided, files will be stored at the bucket root (current behavior)
- Make sure to include a trailing slash if you want a folder-like structure
- The prefix applies to both the upload file and its `.info` metadata file
- The prefix is also applied to incomplete parts (`.part` files)
130 changes: 130 additions & 0 deletions packages/s3-store/QUICK_REFERENCE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
# Quick Reference: S3 Object Prefix

## TL;DR

Add `objectPrefix` to your S3Store configuration to organize files in folders:

```typescript
const store = new S3Store({
objectPrefix: 'uploads/', // ← Add this!
s3ClientConfig: { /* ... */ }
});
```

## Common Use Cases

### 1. Simple folder organization
```typescript
objectPrefix: 'uploads/'
// Result: bucket/uploads/file-id
```

### 2. Multi-tenant application
```typescript
objectPrefix: `tenants/${tenantId}/uploads/`
// Result: bucket/tenants/abc-123/uploads/file-id
```

### 3. Date-based organization
```typescript
const date = new Date();
objectPrefix: `uploads/${date.getFullYear()}/${date.getMonth() + 1}/`
// Result: bucket/uploads/2024/10/file-id
```

### 4. Environment separation
```typescript
const env = process.env.NODE_ENV;
objectPrefix: `${env}/uploads/`
// Result: bucket/production/uploads/file-id or bucket/dev/uploads/file-id
```

### 5. File type segregation
```typescript
objectPrefix: 'media/images/'
// Result: bucket/media/images/file-id
```

### 6. User-specific uploads
```typescript
objectPrefix: `users/${userId}/files/`
// Result: bucket/users/user-456/files/file-id
```

## What Gets Prefixed?

✅ Main upload file: `bucket/prefix/file-id`
✅ Info metadata file: `bucket/prefix/file-id.info`
✅ Incomplete parts: `bucket/prefix/file-id.part`

## Tips

- Always use trailing slash for folder-like structure: `'uploads/'` ✅ not `'uploads'`
- Prefix is optional - without it, files go to bucket root (backward compatible)
- S3 doesn't have real folders - prefixes create visual organization
- Use prefixes to apply different lifecycle policies to different file types
- Combine with S3 bucket policies for fine-grained access control

## S3 Console View

With `objectPrefix: 'uploads/'`, your S3 console will show:

```
📁 my-bucket
└── 📁 uploads
├── 📄 file-abc123
├── 📄 file-abc123.info
├── 📄 file-xyz789
└── 📄 file-xyz789.info
```

Without prefix:

```
📁 my-bucket
├── 📄 file-abc123
├── 📄 file-abc123.info
├── 📄 file-xyz789
└── 📄 file-xyz789.info
```

## Complete Example

```typescript
import {Server} from '@tus/server';
import {S3Store} from '@tus/s3-store';

const tusServer = new Server({
path: '/files',
datastore: new S3Store({
objectPrefix: 'uploads/',
s3ClientConfig: {
bucket: 'my-bucket',
region: 'us-east-1',
credentials: {
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
},
},
}),
});
```

## Migration from No Prefix

If you're adding a prefix to an existing setup:

1. **Option A**: Keep old files where they are, new files use prefix (recommended)
2. **Option B**: Use AWS CLI to move existing files:
```bash
aws s3 mv s3://my-bucket/ s3://my-bucket/uploads/ --recursive
```

## Benefits

🎯 **Organization**: Logical folder structure
🔐 **Security**: Apply IAM policies per prefix
💰 **Cost**: Track storage costs by prefix
⏰ **Lifecycle**: Different retention policies per folder
🧹 **Cleanup**: Easy to delete all files in a prefix
📊 **Analytics**: Better insights with S3 analytics
Loading