Hostname: page-component-586b7cd67f-l7hp2 Total loading time: 0 Render date: 2024-11-27T14:29:30.134Z Has data issue: false hasContentIssue false
  • English
  • Français

Using automatically labelled examples to classify rhetorical relations: an assessment

Published online by Cambridge University Press:  01 July 2008

CAROLINE SPORLEDER  and
ALEX LASCARIDES
CAROLINE SPORLEDER
Affiliation:
ILK/Language and Information Science, Tilburg University, P.O. Box 90153, 5000 LE Tilburg, The Netherlands e-mail: [email protected]
ALEX LASCARIDES
Affiliation:
School of Informatics, The University of Edinburgh, 2 Buccleuch Place, Edinburgh EH8 9LW, UK e-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Being able to identify which rhetorical relations (e.g., contrast or explanation) hold between spans of text is important for many natural language processing applications. Using machine learning to obtain a classifier which can distinguish between different relations typically depends on the availability of manually labelled training data, which is very time-consuming to create. However, rhetorical relations are sometimes lexically marked, i.e., signalled by discourse markers (e.g., because, but, consequently etc.), and it has been suggested (Marcu and Echihabi, 2002) that the presence of these cues in some examples can be exploited to label them automatically with the corresponding relation. The discourse markers are then removed and the automatically labelled data are used to train a classifier to determine relations even when no discourse marker is present (based on other linguistic cues such as word co-occurrences). In this paper, we investigate empirically how feasible this approach is. In particular, we test whether automatically labelled, lexically marked examples are really suitable training material for classifiers that are then applied to unmarked examples. Our results suggest that training on this type of data may not be such a good strategy, as models trained in this way do not seem to generalise very well to unmarked data. Furthermore, we found some evidence that this behaviour is largely independent of the classifiers used and seems to lie in the data itself (e.g., marked and unmarked examples may be too dissimilar linguistically and removing unambiguous markers in the automatic labelling process may lead to a meaning shift in the examples).

Type
Papers
Information
Natural Language Engineering , Volume 14 , Issue 3 , July 2008 , pp. 369 - 416
Copyright
Copyright © Cambridge University Press 2006

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Achinstein, P. (1980). The Nature of Explanation. Oxford University Press.Google Scholar
Asher, N. and Lascarides, A. (2003). Logics of Conversation. Cambridge University Press.Google Scholar
Baldridge, J. and Lascarides, A. (2005). Probabilistic head-driven parsing for discourse structure. In Proceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL).CrossRefGoogle Scholar
Bromberger, S. (1962). An approach to explanation. In Butler, R. J. (Ed.), Analytical Philosophy, pp. 75–105. Oxford University Press.Google Scholar
Carlson, L., Marcu, D., and Okurowski, M. E. (2002). RST Discourse Treebank. Linguistic Data Consortium.Google Scholar
Carlson, L., Marcu, D., and Okurowski, M. E. (2003). Building a discourse-tagged corpus in the framework of rhetorical structure theory. In van Kuppevelt, J. and Smith, R. (Eds.), Current Directions in Discourse and Dialogue, pp. 85–112. Kluwer Academic Publishers.CrossRefGoogle Scholar
Charniak, E. (2000). A maximum-entropy-inspired parser. In Proceedings of the 1st Conference of the North American Chapter of the Assocation for Computational Linguistics, Seattle, WA, pp. 132–139.Google Scholar
Chawla, N. V., Bowyer, K. W., Hall, L. O., and Kegelmeyer, W. P. (2002). SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research 16, 321–357.CrossRefGoogle Scholar
Collins, M. (2003). Head-driven statistical models for natural language parsing. Computational Linguistics 29 (4), 589–638.CrossRefGoogle Scholar
Corston-Oliver, S. H. (1998). Identifying the linguistic correlates of rhetorical relations. In Proceedings of the ACL Workshop on Discourse Relations and Discourse Markers, pp. 8–14.Google Scholar
Fellbaum, C. (Ed.) (1998). WordNet: An Electronic Database. Cambridge, MA: MIT Press.CrossRefGoogle Scholar
Forbes, K., Miltsakaki, E., Prasad, R., Sarkar, A., Joshi, A., and Webber, B. (2001). D-LTAG System – discourse parsing with a lexicalized tree adjoining grammar. In Proceedings of the ESSLLI-01 Workshop on Information Structure, Discourse Structure and Discourse Semantics.Google Scholar
Grice, H. P. (1975). Logic and conversation. In Cole, P. and Morgan, J. L. (Eds.), Synax and Semantics Volume 3: Speech Acts, pp. 41–58. Academic Press.Google Scholar
Hobbs, J. R., Stickel, M., Appelt, D., and Martin, P. (1993). Interpretation as abduction. Artificial Intelligence 63 (1–2), 69–142.CrossRefGoogle Scholar
Hutchinson, B. (2004). Acquiring the meaning of discourse markers. In Proceedings of ACL-04, pp. 685–692.CrossRefGoogle Scholar
Kamp, H. and Reyle, U. (1993). From Discourse To Logic. Introduction to Modeltheoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory. Dordrecht, The Netherlands: Kluwer Academic Publishers.Google Scholar
Lapata, M. and Lascarides, A. (2004). Inferring sentence-internal temporal relations. In Proceedings of NAACL-04, pp. 153–160.Google Scholar
Le Thanh, H., Abeysinghe, G., and Huyck, C. (2004). Generation discourse structures for written text. In Proceedings of COLING-04, pp. 329–335.Google Scholar
Litman, D. J. (1996). Cue phrase classification using machine learning. Journal of Artificial Intelligence Research 5, 53–94.CrossRefGoogle Scholar
Mann, W. C. and Thompson, S. A. (1987). Rhetorical structure theory: A theory of text organization. Technical Report ISI/RS-87-190, ISI, Los Angeles, CA.Google Scholar
Marcu, D. (1997). The Rhetorical Parsing, Summarization, and Generation of Natural Language Texts. Ph. D. thesis, Department of Computer Science, University of Toronto.Google Scholar
Marcu, D. (1998). Improving summarization through rhetorical parsing tuning. In The 6th Workshop on Very Large Corpora, pp. 206–215.Google Scholar
Marcu, D. (1999). A decision-based approach to rhetorical parsing. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics (ACL-99), pp. 365–372.CrossRefGoogle Scholar
Marcu, D. and Echihabi, A. (2002). An unsupervised approach to recognizing discourse relations. In Proceedings of ACL-02, pp. 368–375.Google Scholar
Minnen, G., Carroll, J., and Pearce, D. (2001). Applied morphological processing of English. Natural Language Engineering 7 (3), 207–223.CrossRefGoogle Scholar
Moore, J. D. and Pollack, M. E. (1992). A problem for RST: The need for multi-level discourse analysis. Computational Linguistics 18 (4), 537–544.Google Scholar
Nomoto, T. and Matsumoto, Y. (1999). Learning discourse relations with active data selection. In Proceedings of EMNLP-99.Google Scholar
Oates, S. L. (2000). Multiple discourse marker occurrence: Creating hierarchies for natural language generation. In Proceedings of the North American Chapter of the Association for Computational Linguistics, pp. 41–45.Google Scholar
Pardo, T. A. S., das Graşas Volpe Nunes, M., and Rino, L. H. M. (2004). DiZer: An automatic discourse analyzer for brazilian portuguese. In Proceedings of the 17th Brasilian Symposium on Artificial Intelligence (SBIA).CrossRefGoogle Scholar
Polanyi, L. (1985). A theory of discourse structure and discourse coherence. In Eilfort, P. D. K. W. H. and Peterson, K. L. (Eds.), Papers from the General Session at the 21st Regional Meeting of the Chicago Linguistics Society.Google Scholar
Polanyi, L., Culy, C., van den Berg, M., Thione, G. L., and Ahn, D. (2004a). A rule based approach to discourse parsing. In Proceedings of the 5th SIGDIAL Workshop in Discourse and Dialogue, pp. 108–117.Google Scholar
Polanyi, L., Culy, C., van den Berg, M., Thione, G. L., and Ahn, D. (2004b). Sentential structure and discourse parsing. In Proceedings of the ACL-04 Workshop on Discourse Annotation.CrossRefGoogle Scholar
Porter, M. F. (1980). An algorithm for suffix stripping. Program 14, 130–137.CrossRefGoogle Scholar
Reynar, J. C. and Ratnaparkhi, A. (1997). A maximum entropy approach to identifying sentence boundaries. In Proceedings of ANLP-97, pp. 16–19.CrossRefGoogle Scholar
Schapire, R. E. and Singer, Y. (2000). BoosTexter: A boosting-based system for text categorization. Machine Learning 39 (2/3), 135–168.CrossRefGoogle Scholar
Siegel, S. and Castellan, N. J. (1988). Nonparametric Statistics for the Behavioral Sciences. New York: McGraw-Hill.Google Scholar
Soria, C. and Ferrari, G. (1998). Lexical marking of discourse relations – some experimental findings. In Proceedings of the ACL-98 Workshop on Discourse Relations and Discourse Markers.Google Scholar
Soricut, R. and Marcu, D. (2003). Sentence level discourse parsing using syntactic and lexical information. In Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics.CrossRefGoogle Scholar
Sporleder, C. and Lapata, M. (2005). Discourse chunking and its application to sentence compression. In Proceedings of the 2005 Human Language Technology Conference and the Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP-05).CrossRefGoogle Scholar
Sporleder, C. and Lascarides, A. (2005). Exploiting linguistic cues to classify rhetorical relations. In Proceedings of Recent Advances in Natural Language Processing (RANLP-05).Google Scholar
Webber, B. L., Knott, A., Stone, M., and Joshi, A. (2003). Anaphora and discourse structure. Computational Linguistics 29 (4), 545–588.CrossRefGoogle Scholar