Weight Pushing and Binarization for Fixed-Grammar Parsing

Matt Post and Daniel Gildea

11th International Conference on Parsing Technology (IWPT 2009)
Paris, France, 7th-9th October, 2009

---

Summary

We introduce the technique of weight diffusion, which is applied to fixed context-free grammars after binarization. Diffusion takes the rule probability from the original grammar and pushes it down across its binarized pieces, allowing the parser to make better pruning decisions earlier in the parsing process. We examine its effect on parsing efficiency with various binarization schemes and different fixed grammars, including an original tree substitution grammar produced by Gibbs sampling with a Dirichlet process prior. We find that weight diffusion produces dramatic improvements in efficiency, especially with small amounts of time and with large grammars.

---

START Conference Manager (V2.56.8 - Rev. 780)