Syntax verification query

[aubreyz]

From #910 (comment)

Tasks are important things. People don't want to miss them by mistake or as a result of a syntax error. Obviously it is important to make syntax errors less likely by making the rules less easy to flout. However, would it be worth having a task query strategy that detects likely malformed tasks. This could include things in the wrong order, similar icons which look the same but are not, and most importantly invalid dates.

Because checkboxes may be used for many other things, I'd restrict this to #tasks (and people would have to specifically say that something is intended to be a valid task for it to be checked).

[claremacrae]

Tasks are important things. People don't want to miss them by mistake or as a result of a syntax error. Obviously it is important to make syntax errors less likely by making the rules less easy to flout. However, would it be worth having a task query strategy that detects likely malformed tasks.

Yes, I do think this is a good idea. You describe it in the subject line as a query, and I think that is an interesting idea. I think it should be done on request, rather than all the time, and I think that is the implication of your wording.

This could include things in:
the wrong order

We are moving towards making the parsing more forgiving of order, so this problem should hopefully go away

similar icons which look the same but are not

I'd be interested to hear from someone who knows about unicode, as to how feasible that is. I mean, there are a lot of unicode characters, and manually curating a list of 'looks the same' might be onerous.

The Task.ts code already has some alternative emoji supported, which also seems like a good way to go:

    public static readonly scheduledDateRegex = /[⏳⌛] *(\d{4}-\d{2}-\d{2})$/u;
    public static readonly dueDateRegex = /[📅📆🗓] *(\d{4}-\d{2}-\d{2})$/u;

Another possibility is to recognise a Unicode character followed by a date, and say if the character was unrecognised.

and most importantly invalid dates.

This is an interesting one. How far away does something have to be from a valid date?
One definite cause of error is YYYY-DD-MM - getting the day and month the wrong way round.

Because checkboxes may be used for many other things, I'd restrict this to #tasks (and people would have to specifically say that something is intended to be a valid task for it to be checked).

What do you mean by "I'd restrict this to #tasks"?

The plugin doesn't require a global filter. So whatever we do needs to work for people without global filters.

Also, I've found that by far my most common mistake that causes tasks to be missed is to omit the [ ]. So my own personal Python script to check for malformed tasks looks for emoji recognised by Task plugin, that is in a vanilla list item. The current Tasks parsing code completely ignores those lines, so would not even treat them as an error.

[aubreyz]

Responses in the same order without directly quoting:

a) Yes I think as a query would be a good way. That would be discretionary, should have no performance impact. Could have one single query which would flag up all likely malformed tasks.
b) I agree just allowing different near identical unicode symbols might be good. I guess this type of error is rare but hard to detect.
c) For dates there is already available Regex that will do this

For example
(?<=\D|^)(?\d{4})(?[^\w\s])(?1[0-2]|0[1-9])\k(?0[1-9]|[12][0-9]|(?<=11\k|[^1][4-9]\k)30|(?<=1[02]\k|[^1][13578]\k)3[01])(?=\D|$)

will detect dates where

Year must be preceded by nothing or a non-digit character

Year must have 4 digits

Month must be between 01 and 12
Month must have 2 digits
Day must be between 01 and the maximum number of days for the month (e.g. february can't have more than 29 days)
Day must have 2 digits
Day must be followed by nothing or a non-digit character
Separator must be any single character that is not a space or an alphanumeric character
Separator must be the same between each date part

That could be modified accordingly but the general idea is there. If month and day are swopped it would obviously only detect it if the result is an impossible date, but obviously without doing mind-reading more than that would be impossible.

I suppose apart from detecting things that are not dates at all, you could also flag up highly implausible dates, say more than 5 years in the past or more than 30 years in the future. I think the idea would be to alert people to possible things that MIGHT need attention.

d) All I meant by restricting it to the global filter (e.g. #tasks) was that we would not be detecting the type of error where something was intended to be a task, has at least one date emoji or priority emoji, but does not have #task because that was forgotten (if it is required). But maybe that is something to know about. Along similar lines I think your detection of things intended to be tasks, but where user does not type exactly - [ ] would be tricky probably....

[claremacrae]

Thanks for the suggestions. Have you offered a regex that only matches valid dates?