ZhongHistory

Note

columns

1. ordinary: the ordinary grammar (using neither unknown word handling nor bridge rules)
2. unk: unknown word handling (the YY mode (ZhongYYMode))
3. br: robust parsing with bridge rules
4. unk+br: unknown word handling and robust parsing
5. generation: using the result of [1] as the input MRSes (# of generated items / # of parse items)
6. end-to-end: # of generated items / # of input sentences

abbreviations

1. auto: lexical items automatically constructed from the Sinica treebank
2. punct: robust text handling, such as punctuations, symbols, interjection

options

1. nbest: 5
2. timeout: 5 sec.
3. max-chart-megabytes: 256mb
4. max-unpack-megabytes: 256mb

Development Data

mrs

Changes	Date	ordinary	unk	br	unk+br	generation	end-to-end
w/o auto	2015/03/12	38.3	38.3	97.2	93.5	generation	end-to-end
auto	2015/03/12	93.5	88.8	100	94.4	generation	end-to-end
interjections	2015/03/13	93.5	88.8	100	94.4	generation	end-to-end
AABB	2015/03/13	93.5	88.8	100	94.4	generation	end-to-end
change	YYYY/MM/DD	ordinary	unk	br	unk+br	generation	end-to-end

ntumc

Changes	Date	ordinary	unk	br	unk+br	generation	end-to-end
w/o auto	2015/03/12	0.0	1.4	0.0	1.9	generation	end-to-end
auto	2015/03/12	0.8	1.5	1.0	2.8	generation	end-to-end
interjections	2015/03/13	0.8	1.5	1.0	2.7	generation	end-to-end
AABB	2015/03/13	0.8	1.5	1.0	3.0	generation	end-to-end
change	YYYY/MM/DD	ordinary	unk	br	unk+br	generation	end-to-end

pctb-dev (the first 5,000 sentences)

Changes	Date	ordinary	unk	br	unk+br	generation	end-to-end
w/o auto	2015/03/12	0.0	2.3	0.0	3.0	generation	end-to-end
auto	2015/03/12	0.0	2.5	0.1	4.1	generation	end-to-end
interjections	2015/03/13	0.0	2.5	0.1	4.1	generation	end-to-end
AABB	2015/03/13	0.0	2.5	0.1	4.1	generation	end-to-end
change	YYYY/MM/DD	ordinary	unk	br	unk+br	generation	end-to-end

sinica-dev (the first 5,000 sentences (nltk_data))

Changes	Date	ordinary	unk	br	unk+br	generation	end-to-end
w/o auto	2015/03/12	0.1	8.1	0.7	10.0	generation	end-to-end
auto	2015/03/12	1.2	8.6	2.4	11.1	generation	end-to-end
interjections	2015/03/13	1.2	8.6	2.5	11.1	generation	end-to-end
AABB	2015/03/13	1.2	8.7	2.5	11.1	generation	end-to-end
change	YYYY/MM/DD	ordinary	unk	br	unk+br	generation	end-to-end

Test Data

jec

Changes	Date	ordinary	unk	br	unk+br	generation	end-to-end
w/o auto	2015/03/12	0.0	18.3	0.1	22.7	generation	end-to-end
auto	2015/03/12	2.5	20.2	4.3	30.6	generation	end-to-end
interjections	2015/03/13	2.5	20.2	4.3	30.4	generation	end-to-end
AABB	2015/03/13	2.7	20.5	4.5	31.1	generation	end-to-end
change	YYYY/MM/DD	ordinary	unk	br	unk+br	generation	end-to-end

fu-berin

Changes	Date	ordinary	unk	br	unk+br	generation	end-to-end
w/o auto	2015/03/12	1.2	48.1	3.7	64.2	generation	end-to-end
auto	2015/03/12	4.9	40.7	9.9	65.4	generation	end-to-end
interjections	2015/03/13	4.9	40.7	9.9	65.4	generation	end-to-end
AABB	2015/03/13	4.9	39.5	9.9	65.4	generation	end-to-end
change	YYYY/MM/DD	ordinary	unk	br	unk+br	generation	end-to-end

mcg-smallest

Changes	Date	ordinary	unk	br	unk+br	generation	end-to-end
w/o auto	2015/03/12	4.5	46.8	7.2	56.8	generation	end-to-end
auto	2015/03/12	26.1	42.3	42.3	62.2	generation	end-to-end
interjections	2015/03/13	26.1	42.3	42.3	62.2	generation	end-to-end
AABB	2015/03/13	27.9	44.1	44.1	64.0	generation	end-to-end
change	YYYY/MM/DD	ordinary	unk	br	unk+br	generation	end-to-end

mcg-wxl

Changes	Date	ordinary	unk	br	unk+br	generation	end-to-end
w/o auto	2015/03/12	3.8	46.6	5.5	56.2	generation	end-to-end
auto	2015/03/12	23.4	37.3	43.8	65.5	generation	end-to-end
interjections	2015/03/13	23.4	37.3	43.8	65.5	generation	end-to-end
AABB	2015/03/13	23.9	37.8	44.5	66.3	generation	end-to-end
change	YYYY/MM/DD	ordinary	unk	br	unk+br	generation	end-to-end

pctb-test (the second 5,000 sentences)

Changes	Date	ordinary	unk	br	unk+br	generation	end-to-end
w/o auto	2015/03/12	0.0	9.2	0.0	8.5	generation	end-to-end
auto	2015/03/12	0.0	8.5	0.0	9.9	generation	end-to-end
interjections	2015/03/13	0.0	8.5	0.0	9.9	generation	end-to-end
AABB	2015/03/13	0.0	8.5	0.0	9.9	generation	end-to-end
change	YYYY/MM/DD	ordinary	unk	br	unk+br	generation	end-to-end

sinica-test (the second 5,000 sentences (nltk_data))

Changes	Date	ordinary	unk	br	unk+br	generation	end-to-end
w/o auto	2015/03/12	0.0	1.7	0.0	1.1	generation	end-to-end
auto	2015/03/12	0.0	1.7	0.1	1.5	generation	end-to-end
interjections	2015/03/13	0.0	1.7	0.1	1.5	generation	end-to-end
AABB	2015/03/13	0.0	1.8	0.1	1.5	generation	end-to-end
change	YYYY/MM/DD	ordinary	unk	br	unk+br	generation	end-to-end