Sharded Task Runner

A custom Java Executor that allows you to submit tasks which are executed sequentially per key, but in parallel across different keys. This is useful for scenarios where you want to ensure that tasks for the same "shard" (or key) are not run concurrently, but you still want to maximize concurrency across different shards.

Features

• Key-based Task Sharding: Tasks are grouped and executed sequentially per key.
• Parallelism Across Keys: Tasks for different keys can run in parallel.
• Bounded Queue per Key: Each key has a maximum number of pending tasks (default: 100).
• Customizable Thread Pool: Uses a configurable ExecutorService for task execution.
• Supports Runnable, Callable, and FutureTask: Flexible task submission.
• Automatic Cleanup: Cleans up resources when queues are empty.

Usage

1. Add the Source Files

Copy ShardedTaskRunner.java and ShardedTask.java into your project.

2. Create an Instance

import com.stryhx.shardedtaskrunner.ShardedTaskRunner;

import java.util.concurrent.*;

public class Example {
    public static void main(String[] args) {
        int threadPoolSize = 4;
        long keepAliveTime = 60L;
        TimeUnit unit = TimeUnit.SECONDS;
        BlockingQueue<Runnable> queue = new ArrayBlockingQueue<>(100);
        ThreadFactory threadFactory = Executors.defaultThreadFactory();
        RejectedExecutionHandler handler = new ThreadPoolExecutor.CallerRunsPolicy();

        ShardedTaskRunner runner = new ShardedTaskRunner(
            threadPoolSize, keepAliveTime, unit, queue, threadFactory, handler
        );

        // Submit tasks
        runner.submit("user-1", () -> System.out.println("Task for user-1"));
        runner.submit("user-2", () -> System.out.println("Task for user-2"));
        runner.submit("user-1", () -> System.out.println("Another task for user-1"));

        // Shutdown when done
        runner.shutdown();
    }
}

3. Submitting Tasks

You can submit tasks using a key (shard):

• submit(String key, Runnable task)
• submit(String key, Callable<V> task)
• submit(String key, FutureTask<V> task)

You can also use execute(Runnable) for ad-hoc tasks (a random key will be generated).

4. Shutdown

Call shutdown() to gracefully stop the executor.

How It Works

• Each key gets its own queue and semaphore.
• Only one task per key is executed at a time.
• When a task finishes, the next task for that key is dispatched.
• If too many tasks are queued for a key, new submissions are rejected.