Preparing virtual corpora for FCS

[margaretha]

The virtual corpora at https://korap.ids-mannheim.de/doc/corpus should be made available for FCS. Some required properties to provide the VC as described at https://github.com/KorAP/Kustvakt/wiki/Setting-FCS-Resources, are missing from the table: id, pid, en_title.

The layers of all resources are the same as Wikipedia.

The 1st table is directly generated from a database. The 2nd table is available as a Stylesheet and has been converted into the required format. See 2dee932.

To do:

1. Add the missing properties: id (must contain at least 3-characters), pid, en_title
2. Create a file for the 1st table to process the data.
3. Create a script to convert the 1st table to the required JSON format.

[luengen]

I have added the columns ID, Beschreibung, English Titel and English Description in the 2nd table. Otherwise there is nothing to do for the 2nd table according to the issue

[luengen]

I have the following remarks as to the current VC definitions in table2:

• What, from a user's perspective, is the difference between the VC called "Zeitschriften" and the one called "Publikumszeitschriften" ? I would suggest to have only one of those.

• Does a corpus "Wikipedia-Diskussionsseiten" where all talk pages from dumps from all the years that we have are lumped together actually make sense? It will contain many duplicate and nearly similar pages that are versions of each other. I would instead suggest to have Wikipeda VCs by year (i.e. ARTICLES+TALK+USERTALK) and VCs for Talk pages by years (i.e. TALK + USERTALK)

• Should we here use the term Trivialliteratur when we are currently building a corpus to be called Schemaliteratur that will probably also go in this VC?

[luengen]

Ok I saw Publikumszeitschriften is a subset and makes sense as separate VC

[margaretha]

These are questions for Marc @kupietz . Could you please clarify, Marc?

I agree with Wikipedia VC by year.

[margaretha]

@luengen Thank you for preparing the data, Harald! I have registered them at the test instance.

[kupietz]

I agree that …

• Publikumszeitschriften (English IMHO: "General Interest Magazines") are just a subset of Zeitschriften
• we need to group the Wikipedia discussions by the corpus release year
• we just need "Schemaliteratur" (English: "Paraliterature")

What other questions are open?

[luengen]

The named VC "Belletristik/Schemaliteratur" contains both according to the regex, so I have now renamed it in "Belletristik inklusive Schemaliteratur". And I have added a new, pure Schemaliteratur VC. So far only docSigle=/MK2\/TRI/ is in it. Q: Do we count the Gmeiner Krimis as Schemaliteratur? I think implicitly we have not done so in the past

[luengen]

Also I have added year-based WP talk corpora. I noticed that a Wiki talk VC actually only makes sense for 2013 and 2015, as for the other years, we have no user talk corpus in KorAP so nothing to combine with wdd to form a VC

[luengen]

https://cloud.ids-mannheim.de/s/rJQdSSspWRcHGTJ

[luengen]

The named VC "Belletristik/Schemaliteratur" contains both according to the regex, so I have now renamed it in "Belletristik inklusive Schemaliteratur". And I have added a new, pure Schemaliteratur VC. So far only docSigle=/MK2\/TRI/ is in it. Q: Do we count the Gmeiner Krimis as Schemaliteratur? I think implicitly we have not done so in the past

I have now included the GM corpus and all texttype="Roman: Kriminalroman" in Schemaliteratur. Please let me know if this should be changed