Give Me More Feedback II: Annotating Thesis Strength and Related Attributes in Student Essays

Zixuan Ke, Hrishikesh Inamdar, Hui Lin, and Vincent Ng.
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3994-4004, 2019.

Click here for the PDF version. The talk slides are available here.

Abstract

While the vast majority of existing work on automated essay scoring has focused on holistic scoring, %a holistic score provides little feedback to a student on what went wrong and how she can improve her essay if she receives a low score. To address this problem, researchers have recently begun work on scoring specific dimensions of essay quality. Nevertheless, progress in dimension-specific essay scoring research is hindered in part by the lack of annotated corpora. To facilitate advances in this area of research, we design a rubric for scoring an important, yet unexplored dimension of persuasive essay quality, thesis strength, and annotate a corpus of essays with thesis strength scores. We additionally identify the attributes that could impact thesis strength and annotate the essays with the values of these attributes, which, when predicted by computational models, could provide %further feedback to students on why her essay receives a particular thesis strength score.

Dataset

The human annotation used in this paper is available from this page.

BibTeX entry

@InProceedings{Ke+etal:19a,
  author = {Zixuan Ke and Hrishikesh Inamdar and Hui Lin and Vincent Ng},
  title = {Give Me More Feedback II: Annotating Thesis Strength and Related Attributes in Student Essays},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  pages = {3994--4004}, 
  year = 2019}