Deprecated libmei

[ahankinson]

The main libmei library hasn't received any updates in several years. To dissuade people from using it I have marked it as deprecated and archived it.

The parseschema2.py script there, however, can still be used to generate the libmei component of the Sibelius plugin. I can see a few possible courses of action here, from least to most work:

• Do nothing. The script should still work with new schemas for a while, so there's nothing that needs to be done.
• Move the parseschema2.py and manuscript generator code to this repo, and make it self-contained. This means maintaining a fork of the code here.
• Update the Verovio fork of libmei to also include the manuscript generator, and any updates to the sibmei code generator get added there. I don't know exactly how much has changed in the Verovio fork -- it has received continuous updates -- so this may need quite a bit of work to bring them in line.

Anyway, I thought I would note this change here. Feel free to chime in with any suggestions, but note that I probably won't have much time in the next few years, so if you have time and feel passionately about any of the above options, then feel free to take it on.

[th-we]

Another option I see: We can integrate the "handwritten" parts of libmei into Sibmei and drop the auto-generated functions. I created a proof-of-concept branch, which is working (all tests passing).

The template based approach for generating lines, text, lyrics and symbols bypasses the auto-generated libmei functions anyway. I think that the benefits of libmei.StaffDef() over the generic CreateElement('staffDef') do not weigh enough to justify the effort of maintaining libmei code generation for Manuscript. Unlike C++, we do not get the benefit of static compiler errors when we use outdated MEI elements, we only get hard crashes at runtime. So we have to analyze the code anyway.

We have an analysis script that checks if all used attribute names are present in the current MEI schema. We can easily extend that to element names. With templates (for which we currently do not have automated schema checks), we could even check if the used attributes are valid for the respective element.

If this is acceptable for you, @ahankinson, @rettinghaus, @annplaksin, I will use that approach for upgrading Sibmei to MEI 5.1. Of course it does not make sense to have the XML generation code in the dialogs folder - this was just for the purpose of a quick proof of concept without having to convert to JavaScript like syntax as dialog/*.msd files are copied verbatim by the compiler.

[th-we]

O.K., here's a surprisingly obvious idea: We let CreateElement() check if the passed element name is legal. This has the huge benefit that Sibmei will not crash hard and can report the error in a meaningful way. We only need a lookup table that has all the legal elements. The compiler can add a list of all element names as a global variable.

[th-we]

I updated the branch. CreateElement() now checks if the element name is legal:

https://github.com/notengrafik/sibmei/blob/d767c110981e568ce4b924c02ad3c87ac04ff4f1/src/Xml.msd#L6-L10

[ahankinson]

It’s getting quite complicated to review the changes that you are proposing.

where is the list of allowed XML elements stored?

[th-we]

It’s getting quite complicated to review the changes that you are proposing.

Sorry, a lot of code is affected by the changes, but a lot of it is merely replacing libmei.Foo() with CreateElement('foo').

To give a better overview: These are the complete changes:

253a76d...8251ac3

This is the commit that introduces the check for allowed elements:

d767c11

where is the list of allowed XML elements stored?

The compiler inserts a list of the allowed elements as global variable here:

sibmei/tools/build.mjs

Lines 52 to 58 in d767c11

	async function buildPlg(sourceFiles, target) {
	fs.writeFileSync(target, `${BOM}{
	${compile(sourceFiles)}
	${GLOBALS}
	LegalElements {"${[...await getLegalElements()].join('" "')}"}
	}`, { encoding: "utf16le" });
	}

During initialization, this is converted to a lookup table:

sibmei/src/Initialize.mss

Lines 87 to 91 in d767c11

	Self._property:IsLegalElement = CreateDictionary('<!--', true);
	for each elementName in LegalElements
	{
	IsLegalElement[elementName] = true;
	}

[th-we]

If the remaining pull requests are merged, I'd like to prepare a release and continue with moving to MEI 5. Rather than re-compiling libmei, I'll integrate the libmei code, following the approach taken in the linked branch. As requested I'll assert if elements are part of the respective MEI version.

Can I go ahead with this, @ahankinson and @rettinghaus?