Effective Update

[realfakenerd]

Hi folks! currently, I am rewriting the database codebase and changing the design a bit, shifting the focus from a very fast database to a very reliable one. I hope that's a good move. With this change, I am moving from zero dependencies to one (I know devalue was already there, but this is a more significant one).

I learned about Effect.ts from this video by Matt Pocock.

Effect works wonders. It has a bit of a learning curve, but it has great documentation and is made with TypeScript, making it a really great dependency.

It calls itself "the missing standard library for TypeScript," and it really feels like that, filling some gaps that JS and TS currently have. It will help me make the database more composable, like Lego bricks.

It already has most of the features I need. Jason will have better error handling, making it more reliable. It will be easier to catch bugs and faster to introduce new features (I have some plans for it). I'm also learning a lot while writing the new codebase.

As an example, the new API will look like this:

// Before
interface User {
  name: string;
  email: string;
}

interface Database {
  users: User[];
}

const db = new Jason<Database>('db_dir');
const user_collection = await db.collection('user', {
    schema: (user) => user.name && user.email.includes("@")
});

const user_john = user_collection.create({ name: 'John Noodle', email: 'john.noodle@pasta.com' });

// After
const db = await createJasonDB({
  path: "db_dir",
  collections: {
    user: Schema.Struct({
      name: Schema.String,
      email: Schema.String,
    })
  }
})

const user_collection = db.collections.user;
const user_john = await user_collection.create({ name: 'John Inferred Noodle', email: 'john.noodle@pasta.com'});

// Or you can do this
const {collections: {user}} = await createJasonDB({/** same options */});

The schema is inferred, so there's no need to pass an interface beforehand. Also, taking inspiration from Dexie.js, you can do this:

const db = await createJasonDB({
  path: "db_dir",
  collections: {
    user: 'name, email'
  }
}); // this will be { name: string, email: string }

It will properly transform the string into a schema. But if you noticed, everything is a string. That's the default behavior I chose, but what if you want to have an age or a boolean value?

const db = await createJasonDB({
  path: "db_dir",
  collections: {
    user: 'name, email, age:number, isStudying:boolean'
  }
}); // this will result in a different type { name: string, email: string, age: number, isStudying: boolean }

Why do this? Why not just use the Schema? Well, this way you don't need to install Effect as a dependency for your project. You certainly can if you want to, but it's not required. If you do install it, you can use the full power of Effect with the DB Layer:

const AppLayer = createJasonDBLayer({
  path: "db_dir",
  collections: {
    users: Schema.Struct({
      id: Schema.String,
      name: Schema.String,
      age: Schema.Number,
    }),
  },
});

const program = Effect.gen(function* () {
  const db = yield* JasonDB;

  const user1 = yield* db.collections.users.create({ name: "Jane", age: 25 });
  const foundUser = yield* db.collections.users.findById(user1.id);

  yield* Effect.match(foundUser, {
      onSuccess: (user) => Effect.log(`User found: ${user.name}`),
      onFailure: () => Effect.log("User not found."),
  });
});

So, that's it for this update. If you want to check out what I'm doing, it's on the refactor/add-effect-ts branch here: refactor branch.

Do you have any ideas, opinions, or critiques? Just let me know

---

Update 1:

Currently, i am working on making metadata useful, and indexes.

Update 2:

Metadata is almost ready, same for indexes with BTree. Working on improving the Config Service and improving the Types for better inference.