RFC-0001: Save Synchronization System

[gantoine]

RFC Number: 0001
Title: Device Registration and Multi-Protocol Save Synchronization
Author: @gantoine
Status: Draft
Created: 2025-08-04
Updated: 2025-08-04

Abstract

This RFC proposes a save synchronization system where clients register with the server using pre-built yet configurable profiles and receive a unique device ID. The system maintains sync state tracking to enable seamless cross-device gameplay continuation, and supports three sync modes: API, file transfer, and push-pull.

Motivation

Synchronization

Cross-device save sync is a an ambitious goal, but one that is fundamental to the success of this project. The expanding ecosystem of client applications demands the ability to pick-up-and-play on any device with seamless gameplay continuation, and existing solutions are inadequate.

Device Registration

As each system has different technical capabilities, a one-size-fits-all approach won't work. A device registration method allows for platform-specific sync mode selection, save format awareness and conversion (file structure, naming, emulator), and flexible sync scheduling based on device capabilities.

Architecture Overview

The save synchronization system consists of three main components: the server-side sync orchestrator, client-side sync agents, and a device registry. The architecture is designed to handle the diverse technical capabilities of different client platforms.

Device Registry Service

Clients register with platform-specific capabilities and receive a unique device ID. Pre-built profiles are available for supported platforms (Playnite, muOS, Android, etc.), but options can be customized.

Sync Orchestrator

The orchestrator tracks the state (last sync time, sync status, etc.) of each device and schedules sync jobs based on device capabilities.

The orchestrator handles conflict resolution naively by always favoring the most recent save. Since device timestamps are not reliable, an alternative conflict resolution strategy is required. In file transfer and push-pull modes, sync state metadata can be pushed to the device and updated on each sync for better conflict resolution, while in API mode, clients handle storing their own metadata.

The orchestrator will prefer devices that use API mode, as it's the most reliable way to sync saves, but will support file transfer and push-pull modes for devices that don't have native client apps.

Sync Engine

API Mode

In API mode, clients negotiate the sync state with the orchestrator and then perform parallel save operations. Clients track which saves have been created/updated and send the list to the orchestrator, which returns operation details for execution. In essence, the client sends the state-of-saves to the orchestrator, and the orchestrator returns the operations to perform.

Device Registration:

{
  "device_name": "Anbernic RG40XXV",
  "platform": "muos",
  "sync_mode": "api",
}

{
  "device_id": "83372e3b-3b30-48a9-a585-40cd1a4da810"
}

Sync Negotiation:

{
  "device_id": "83372e3b-3b30-48a9-a585-40cd1a4da810",
  "device_time": "2025-08-04T12:00:00Z",
  "client_state": {
    "last_sync_time": "2025-08-04T10:30:00Z",
    "saves": {
      "created": [
        {
          "operation_id": "op-1",
          "rom_id": 8273,
          "filename": "Advance Wars (1).srm",
          "sha1": "ebd8b595cdc30873a0d17476e396571463fdb731",
          "file_size": 24785,
          "updated_at": "2025-08-04T11:00:00Z"
        }
      ],
      "modified": [
        {
          "operation_id": "op-2",
          "save_id": 283,
          "filename": "Advance Wars.srm",
          "sha1": "5123655a27403c1a0697b3583b530c70d16862d0",
          "file_size": 1024000,
          "updated_at": "2025-08-04T11:00:00Z"
        },
        {
          "operation_id": "op-3",
          "save_id": 1238,
          "filename": "Crash Nitro Kart (2).srm",
          "sha1": "7830537639ecd86d6419f365d67aede697978893",
          "file_size": 8276943,
          "updated_at": "2025-08-04T11:00:00Z"
        }
      ],
      "deleted": [
        {
          "operation_id": "op-4",
          "save_id": 7345,
          "filename": "Jet Grind Radio (3).srm"
        }
      ]
    },
    "states": {}
  }
}

{
  "sync_session_id": "f32d6ef4-b49a-4634-a2d5-664a5c32841f",
  "server_time": "2025-08-04T12:00:00Z",
  "operations": {
    "upload": [
      {
        "operation_id": "op-1",
        "save_id": 8273,
        "upload_url": "/api/saves/8273?sync_session=f32d6ef4&op_id=op-1"
      },
      {
        "operation_id": "op-2", 
        "save_id": 283,
        "upload_url": "/api/saves/283?sync_session=f32d6ef4&op_id=op-2"
      }
    ],
    "download": [
      {
        "operation_id": "op-3",
        "save_id": 1238,
        "download_url": "/api/saves/1238?sync_session=f32d6ef4&op_id=op-4"
      }
    ],
    "delete": [
      {
        "operation_id": "op-4",
        "save_id": 7345
      },
      {
        "operation_id": "op-5",
        "save_id": 9274,
      }
    ]
  }
}

Clients execute operations in parallel using the provided URLs, where each save endpoint accepts sync context parameters:

# Upload/update saves
POST /api/saves/8273?sync_session=f32d6ef4&op_id=op-1
Content-Type: application/octet-stream
[Save file data]

POST /api/saves/283?sync_session=f32d6ef4&op_id=op-2  
Content-Type: application/octet-stream
[Save file data]

# Download saves
GET /api/saves/1238?sync_session=f32d6ef4&op_id=op-4

# Delete local saves

The server tracks operation completion within the sync session and handles partial failures gracefully. The success/failure of each operation is handled by the HTTP response codes, and the client updates their own sync state metadata. Note that while save uploads/updates are handled by the client, save deletion is handled by the server.

File Transfer Mode

File Transfer mode is designed for devices that use external file synchronization systems like Syncthing, Resilio Sync, or cloud storage folders. In this mode, RomM monitors designated sync folders and automatically processes save files as they appear.

Devices register with a file_transfer sync mode and specify their sync folder path. The external sync system maintains the device's save folder structure as-is. The server monitors these folders and handles all processing internally.

Device Registration:

  "device_name": "Batocera",
  "platform": "batocera_os",
  "sync_mode": "file_transfer",
  "sync_folder": "/romm_sync/device_83372e3b-3b30-48a9-a585-40cd1a4da810",
  "save_formats": {
    "gba": {
      "extension": ".srm",
      "emulator": "mGBA"
    },
    "psx": {
      "extension": ".srm",
      "emulator": "DuckStation"
    }
  }
}

Folder Structure:

romm_sync/
├── device_83372e3b-3b30-48a9-a585-40cd1a4da810/
│   ├── gba/
│   │   ├── Advance Wars (USA) (Rev 1).srm
│   │   └── Pokemon Fire Red.srm
│   ├── psx/
│   │   └── Crash Bandicoot.srm
│   └── n64/
│       └── Super Mario 64.srm
└── device_f47ac10b-58cc-4372-a567-0e02b2c3d479/
    ├── gba/
    │   └── Pokemon Fire Red.srm
    └── psx/
        └── Crash Bandicoot.srm

When a file is modified, the orchestrator compares the file's timestamp and hash against the last known state. If the modification is new, the orchestrator will update the RomM database with the new save data.

When a save is created/updated/deleted by another device (or the system), the orchestrator will modify the sync folder to reflect the change. The orchestrator will not attempt to resolve conflicts as external sync systems handle their own conflict resolution.

This mode is only available for devices that are recognized by the system, i.e. where the folder structure and external sync system is known.

Push-Pull Mode

Push-Pull mode enables server-initiated synchronization using SFTP/SSH connections. The server actively connects to devices, scans their save directories, and performs bidirectional sync operations. This mode is ideal for devices that can't run RomM clients but support SSH access.

Device Registration:

{
  "device_name": "RetroPie",
  "platform": "linux",
  "sync_mode": "push_pull",
  "ssh_config": {
    "host": "192.168.1.100",
    "port": 22,
    "username": "pi",
    "auth_method": "key",
    "private_key_path": "/romm/ssh/retropie_key",
    "save_paths": {
      "gba": "/home/pi/RetroPie/roms/gba/saves",
      "psx": "/home/pi/RetroPie/roms/psx/memcards",
      "n64": "/home/pi/RetroPie/roms/n64/saves"
    }
  },
  "save_formats": {
    "gba": {
      "extension": ".srm",
      "emulator": "mGBA"
    },
    "psx": {
      "extension": ".srm",
      "emulator": "PCSX-ReARMed"
    }
  }
}

The server establishes SSH connection using stored credentials and scans configured save directories for all save files. The orchestrator determines which files need to be pulled, pushed, or deleted and performs SFTP operations to sync files bidirectionally. Finally the orchestrator updates the internal tracking and RomM database.

File Discovery:

# Server scans device directories
$ ssh pi@192.168.1.100 "find /home/pi/RetroPie/roms -name '*.srm' -exec stat -c '%n|%Y|%s' {} \;"

/home/pi/RetroPie/roms/gba/saves/Advance Wars (USA) (Rev 1).srm|1733232000|32768
/home/pi/RetroPie/roms/gba/saves/Pokemon Fire Red.srm|1733235600|32768
/home/pi/RetroPie/roms/psx/memcards/Crash Bandicoot.srm|1733239200|131072

Sync Operations:

# Pull new/modified saves from device
$ sftp pi@192.168.1.100 << EOF
get "/home/pi/RetroPie/roms/gba/saves/Pokemon Fire Red.srm" "/romm/temp/device_83372e3b/pokemon_fire_red.srm"
get "/home/pi/RetroPie/roms/psx/memcards/Crash Bandicoot.srm" "/romm/temp/device_83372e3b/crash_bandicoot.srm"
EOF

# Push saves to device
$ sftp pi@192.168.1.100 << EOF
put "/romm/saves/gba/Advance Wars (USA) (Rev 1).srm" "/home/pi/RetroPie/roms/gba/saves/Advance Wars (USA) (Rev 1).srm"
EOF

# Delete saves from device
$ ssh pi@192.168.1.100 "rm '/home/pi/RetroPie/roms/gba/saves/Old Save.srm'"

System Diagram

Device Registration and Multi-Protocol Save Synchronization -2025-08-04.json

Implementation Details

Database Schema

CREATE TABLE device (
    device_id UUID PRIMARY KEY,
    device_name VARCHAR(255) NOT NULL,
    platform VARCHAR(50) NOT NULL,
    sync_mode VARCHAR(20) NOT NULL,
    sync_config JSONB,
    created_at TIMESTAMP DEFAULT NOW(),
    last_sync TIMESTAMP,
    status VARCHAR(20) DEFAULT 'active'
);

CREATE TABLE sync_state (
    device_id UUID REFERENCES device(device_id),
    save_id INTEGER REFERENCES save(id),
    version INTEGER DEFAULT 1,
    status VARCHAR(20),
    file_hash VARCHAR(64),
    file_size BIGINT,
    last_modified TIMESTAMP,
    PRIMARY KEY (device_id, save_id)
);

CREATE TABLE sync_session (
    session_id UUID PRIMARY KEY,
    device_id UUID REFERENCES device(device_id),
    started_at TIMESTAMP DEFAULT NOW(),
    completed_at TIMESTAMP,
    status VARCHAR(20),
    operations_count INTEGER DEFAULT 0
);

API Endpoints

• POST /api/devices - Device registration
• GET /api/devices/{device_id} - Get device info
• PUT /api/devices/{device_id} - Update device config
• DELETE /api/devices/{device_id} - Deregister device
• POST /api/sync/negotiate - Sync negotiation (API mode)
• GET /api/sync/status - Get sync status
• POST /api/saves/{save_id} - Upload save file
• GET /api/saves/{save_id} - Download save file
• DELETE /api/saves/{save_id} - Delete save file

Conflict Resolution Strategy

Detection Methods

1. Timestamp Comparison: Compare file modification times across devices (not reliable)
2. Hash + Size Comparison: Detect content changes even with same timestamps
3. Version Tracking: Maintain save file version history

Resolution Policies

• Most Recent Wins: Default policy using last synced device
• User Choice: Manual selection in the web app
• Keep Both: Create timestamped copies for manual resolution

Conflict Notification

When conflicts are detected in FTM or PPM, the orchestrator will display a list of conflicts in the web app. The user can then choose to resolve the conflicts manually or automatically. In API mode, the orchestrator will return a list of conflicts to resolve and the client will handle the resolution.

Sync Frequency

• API Mode: Real-time or on-demand (client controlled)
• File Transfer Mode: On filesystem change detection with delay (configurable)
• Push-Pull Mode: Every 10 minutes (configurable)

Metrics

• Sync Success Rate: Percentage of successful sync operations
• Sync Latency: Time from change detection to completion
• File Transfer Rates: Bandwidth utilization and throughput
• Error Rates: Breakdown of sync failures by type and device