LapDevelopment_Giellatekno

Background

In wrapping the UiT Giellatekno (GT) pipeline for Sami segmentation, tokenization, morphological analysis, and dependency parsing, we need to (a) decide how many individual steps of processing to distinguish (from the LAP point of view) and (b) how to map the input and output data from the GT pipeline to LAF-compatible annotations.

Following is a sample invocation (against the LAP Tree), which was originally suggested by Trond Trosterud of UiT:

  cat $LAPTREE/gt/etc/test.txt \
  | $LAPTREE/perl/lap/perl $LAPTREE/gt/script/preprocess --abbr=$LAPTREE/gt/sme/abbr.txt \
  | $LAPTREE/hfst/lap/hfst-lookup $LAPTREE/gt/sme/analyser-gt-norm.hfstol \
  | $LAPTREE/perl/lap/perl $LAPTREE/gt/script/lookup2cg | sed 's/0.000000//g' \
  | $LAPTREE/vislcg3/lap/vislcg3 -g $LAPTREE/gt/sme/disambiguation.cg3 \
  | $LAPTREE/vislcg3/lap/vislcg3 -g $LAPTREE/gt/sme/functions.cg3 \
  | $LAPTREE/vislcg3/lap/vislcg3 -g $LAPTREE/gt/sme/dependency.cg3

It probably makese sense to conceptually distinguish the following layers:

• (a) (maybe normalization and) tokenization (preprocess)

  (b) morphological analysis (hfst-lookup)

  (c) segmentation and morpho-syntactic tagging (disambiguation.cg3)

  (e) grammatical function analysis (functions.cg3)

  (f) dependency parsing (dependency.cg3)

Sample Intermediate Outputs