10 - Narrative Homogeneity and Heterogeneity in Document Categories

Summary

This chapter presents techniques for examining the distributional properties of narrative schemas in a subset of the New York Times (NYT) Corpus. In one technique, the narrative argument salience through entities annotated (NASTEA) task, we use the event participants indicated by narrative schemas to replicate salient entity annotations from the NYT Corpus. In another technique, we measure narrative schema stability by generating schemas with various permutations of input documents. Both of these techniques show differences between homogeneous and heterogeneous document categories. Homogeneous categories tend to perform better on the NASTEA task using fewer schemas and exhibit more stability, whereas heterogeneous categories require more schemas applied on average to peak in performance at the NASTEA task and exhibit less stability. This suggests that narrative schemas succeed at detecting and modeling the repetitive nature of template-written text, whereas more sophisticated models are required to understand and interpret the complex novelty found in heterogeneous categories.