Internal: Using indexeddb and something like jsstore?

[sytone]

So was looking at options to simplify the query management. Did you consider using indexeddb as a cache and then something like jsstore as the app on top. It using sql like query so we can just translate the existing query over to it.

Was considering a hard fork to do a prototype to accelerate task management in obsidian for me, but it is a larger rewrite and not sure if you want to take this approach long term in which case I will look at a potential fork of the plug-in.

[claremacrae]

Thanks for the suggestion.

I don't know enough about the consequences of using indexedDB within Obsidian to understand the proposal...

So was looking at options to simplify the query management.

But I am very confident about this: there are plenty of simple, small, safe incremental refactorings that could be made to greatly simplify the current query code without needing a complete rewrite of the underlying code and adoption of new technology.

I have held off making them because other PRs are in flight that make changes to the same areas of code.

When we have a few days of stability with all recent PRs merged, I would love to go ahead and simplify the current code, to remove repetition and unify existing concepts.

[claremacrae]

I should have said that I do totally agree with your desire to simplify the code for query managing, by which I presume you mean for parsing instruction lines, executing the instructions, and testing the behaviour!

[claremacrae]

I would love to learn more about how this would work, what you have in mind... I don't even know what I don't know! :-)

So was looking at options to simplify the query management. Did you consider using indexeddb as a cache and then something like jsstore as the app on top. It using sql like query so we can just translate the existing query over to it.

Would data be stored in Obsidian's IndexedDB? If so, what kinds of data? Would it be local to each Obsidian install, or does it get shared via Obsidian Sync?

Searching for indexeddb in all plugins, I see a bunch of uses.

I can't see any jsstore references though. Which is fine, it just means I can't see any examples of use.

[sytone]

It would be in the obsidian cache local to the install, you can see its usage if you go to the dev tools in Obsidian and look at the indexedDB instances. As for data, you can think database here. We would have a table for the tasks with all the task properties separated out into separate columns.

jsStore is a library that sits over the IndexedDB to make interaction with it much simpler and also provides a SQL like API along with a plugin that can actually take a SQL query if you wanted to get really flexible in the future.

Query would sit in front of it as a facade and take the current code block directives and instead of making custom filters would just make the jsStore objects to make the query and run it. The jsStore supports other query logic like grouping, and, or, etc that users have asked for. See https://jsstore.net/ for more information.

The Cache class we have now would be able to do the same thing but push it to the IndexedDB instead of an array of Task object in memory.

For example, some complex queries:

SQL - Select * from Table_Name where Column1=value1 or Column2=value2 or Column3=value3;

connection.select({
    from: "Table_Name",
    where: {
        Column1: value1,
        or: {
            Column2: value2,
            Column3: value3
        }
    }
});

SQL - Select * from Table_Name where Column1=value1 and (Column2=value2 or Column3=value3);

connection.select({
    from: "Table_Name",
    where: [{
            Column1: value1
        },
        {
            Column2: value2,
            or: {
                Column3: value3
            }
        }
    ]
});


SQL - Select * from Table_Name where Column1=value1 or (Column2=value2 and Column3=value3);

connection.select({
    from: "Table_Name",
    where: [{
            Column1: value1
        },
        {
            or: {
                Column2: value2,
                Column3: value3
            }
        }
    ]
});

[sytone]

@schemar and @claremacrae I threw caution to the wind and hack the tasks plugin, I have a hard fork called task-x. However, this is now the query you can use. Needs a bit of cleanup but full SQL queries are now supported.

I mostly implemented it so old and new works but made a ton of changes as I was playing do definitely not a single change type commit. https://github.com/sytone/obsidian-tasks-x has the code.

SELECT * FROM ? WHERE status->indicator != "x" AND description LIKE '%Dinner%'

[schemar]

Really cool, @sytone! Would it be possible to allow both, the old queries and SQL? I think SQL would be too advanced for most users.

[sytone]

Yes. I have both running in my code base. I added a new code block type 'taskx' to handle the sql based queries.

[claremacrae]

Thank you! There's lots in here that's really exciting, and seeing concrete code definitely helps explain what you mean.

[claremacrae]

Was considering a hard fork to do a prototype to accelerate task management in obsidian for me, but it is a larger rewrite and not sure if you want to take this approach long term in which case I will look at a potential fork of the plug-in.

By hard fork, I presume you mean what you've done here, to give the plugin a new name and URL... Which is great for prototyping, and allowing both versions to be installed at once.

In case it helps, I really hope that we can work together on this and avoid you needing a permanent hard fork!

[sytone]

Yes. Happy for it to all go back in at some point. I just wanted to be able to run ahead and try it all out.

The ability to run the queries is pretty powerful and simplified the query and sort.

Looking at some other options that the library provides.

[claremacrae]

Yes. Happy for it to all go back in at some point. I just wanted to be able to run ahead and try it all out.

The ability to run the queries is pretty powerful and simplified the query and sort.

Looking at some other options that the library provides.

Sounds good.

Hope you don't mind, in the meantime, I'll plan to carry on with the refactoring started in #690, to reduce that amount of code that needs to be changed, and make fixes and features easier to provide in the meantime.

I like your IQuery idea too. That's another thing that can be implemented in the main project, as a separate step from adding a new implementation of it.

Just examples of ways that I think I can streamline stuff for you, in smaller chunks.