The 7th Workshop on Innovative Use of NLP for Building Educational Applications

NAACL-HLT 2012 Workshops

Post-Workshop Information - Workshop Description - HOO-2012 Shared Task - Workshop Program - Submission Info - Important Dates - Program Committee - Related Links - Upcoming Conferences

Conference: NAACL-HLT 2012

Organization: Joel Tetreault (Educational Testing Service), Jill Burstein (Educational Testing Service), Claudia Leacock (CTB McGraw-Hill)

Contact person: Joel Tetreault:

Date : Thursday, June 07, 2012

Venue : Montreal, Canada

Post-Workshop Information

We would like to thank the authors, our program committee, and workshop attendees for helping put together the 7th NAACL-BEA Workshop. In addition, individual papers and slides, as well as the opening remark slides, can be downloaded from the Workshop Program section below. The complete proceedings can also be found in the ACL Anthology.

For those attending the workshop, we will once again have our world famous post-workshop dinner! Please join us at Restaurant Kashmir at 7:00 (Thursday, June 07). Please email Joel before noon Thursday so he can get a headcount to make reservations. Directions to the workshop can be found here. It is a nice, 30 minute walk. We will meet in the first floor lobby at 6:15 to walk over.

Workshop Description

Research in NLP applications for education continues to progress using innovative NLP techniques - statistical, rule-based, or most commonly, a combination of the two. As a community we are continuing to improve existing capabilities and to identify and generate innovative ways to use NLP in applications for writing, reading, speaking, critical thinking, curriculum development, and assessment. Steady growth in the development of NLP-based applications for education has prompted an increased number of workshops, typically focusing on one specific subfield.

In this workshop, we solicit papers from all subfields: automated scoring, intelligent tutoring, learner cognition, use of corpora, grammatical error detection, and tools for teachers and test developers. Since the first workshop in 1997, "Innovative Use of NLP in Building Educational Applications" has continued to bring together all NLP subfields to foster interaction and collaboration among researchers in both academic institutions and industry. The workshop offers a venue for researchers to present and discuss their work in these areas. Each year, we see steady growth in workshop submissions and attendance, and the research has become more innovative and advanced. In 2012, we expect that the workshop (consistent with previous workshops at ACL 1997, NAACL/HLT 2003, ACL 2005, ACL 2008, NAACL/HLT 2009, NAACL/HLT 2010, and ACL 2011), will continue to expose the NLP research community to technologies that identify novel opportunities for the use of NLP techniques and tools in educational applications. At ACL 2011, the workshop introduced a poster session that was lively and well-attended. We plan to continue to have poster sessions as a regular feature.

The practical need for language-analysis capabilities has been driven by increased requirements for state and national assessments, and a growing population of foreign and second language learners. There are currently a number of commercial systems that handle automated scoring of free-text and speech in the context of assessment as well as systems that address linguistic complexity in text - commonly referred to as readability measures. More recently, the need for applications for language analysis is emphasized by a new influence in the educational landscape in the United States, specifically, the Common Core State Standards initiative: ( that is coordinated by the National Governors Association Center for Best Practices and the Council of Chief State School Officers. The initiative has now been adopted by 46 states for use in Kindergarten through 12th grade (K-12) classrooms. This initiative is likely to have a strong influence on teaching standards in K-12 education. The Common Core standards describe what K-12 students should be learning with regard to Reading, Writing, Speaking, Listening, Language, and Media and Technology. In addition, the Common Core recently released a Publishers Criteria document that describes the array of linguistic elements that learners need to grasp as they progress to the higher grades ( The Common Core thereby introduces language analysis scenarios that have clear alignments with NLP research and applications.

The workshop will solicit both full papers and short papers for either oral or poster presentation. This year, the Helping Our Own (HOO-2) Shared Task on grammatical error detection will be co-located with the BEA7 workshop.

Given the broad scope of the workshop, we organize the workshop around three central themes in the educational infrastructure:

  1. Development of curriculum and assessment (e.g., applications that help teachers develop reading materials)
  2. Delivery of curriculum and assessments (e.g., applications where the student receives instruction and interacts with the system);
  3. Deporting of assessment outcomes (e.g., automated scoring of free responses)

Topics will include, but will not be limited to, the following:

HOO-2012 Shared Task

We are pleased to announce that the second edition of the "Helping Our Own" Shared Task on grammatical error detection will be co-located with BEA7 this year. The Shared Task will be organized independently from the BEA7. System description papers submitted and accepted to the HOO Shared Task will be presented as posters at the BEA Poster Session. In addition, the HOO organizers will summarize the results of the Shared Task in an oral presentation during the BEA. For more information on the task, as well as important dates and submission information, please go to:

Workshop Program

8:40 - 9:00 Loading in of Presentations
Speakers, please bring your presentations so we can preload them before the workshop.
9:00 - 9:15 Opening Remarks
[ slides ]
9:15 - 9:40 Question Ranking and Selection in Tutorial Dialogues
Lee Becker, Martha Palmer, Sarel van Vuuren and Wayne Ward
[ paper ] [ slides ]
9:40 - 10:05 Identifying science concepts and student misconceptions in an interactive essay writing tutor
Steven Bethard, Ifeyinwa Okoye, Md. Arafat Sultan, Haojie Hang, James H. Martin and Tamara Sumner
[ paper ] [ slides ]
10:05 - 10:30 Automatic Grading of Scientific Inquiry
Avirup Sil, Angela Shelton, Diane Jass Ketelhut and Alexander Yates
[ paper ] [ slides ]
10:30 - 11:00 Break
11:00 - 11:25 Modeling coherence in ESOL learner texts
Helen Yannakoudakis and Ted Briscoe
[ paper ] [ slides ]
11:25 - 11:50 Exploring Grammatical Error Correction with Not-So-Crummy Machine Translation
Nitin Madnani, Joel Tetreault and Martin Chodorow
[ paper ] [ slides ]
11:50 - 12:15 HOO 2012: A Report on the Preposition and Determiner Error Correction Shared Task
Robert Dale, Ilya Anisimoff and George Narroway
[ paper ] [ slides ]
12:15 - 1:45 Lunch
1:45 - 3:30 BEA7 and HOO 2012 Poster Session
BEA7 Posters
Measuring the Use of Factual Information in Test-Taker Essays
Beata Beigman Klebanov and Derrick Higgins
[ paper ]
Utilizing Cumulative Logit Model and Human Computation on Automated Speech Assessment
Lei Chen
[ paper ]
PREFER: Using a Graph-Based Approach to Generate Paraphrases for Language Learning
Mei-Hua Chen, Shi-Ting Huang, Chung-Chi Huang, Hsien-Chin Liou and Jason S. Chang
[ paper ]
Using an Ontology for Improved Automated Content Scoring of Spontaneous Non-Native Speech
Miao Chen and Klaus Zechner
[ paper ]
Predicting Learner Levels for Online Exercises of Hebrew
Markus Dickinson, Sandra Kübler and Anthony Meyer
[ paper ]
On using context for automatic correction of non-word misspellings in student essays
Michael Flor and Yoko Futagi
[ paper ]
Judging Grammaticality with Count-Induced Tree Substitution Grammars
Francis Ferraro, Matt Post and Benjamin Van Durme
[ paper ]
Scoring Spoken Responses Based on Content Accuracy
Fei Huang, Lei Chen and Jana Sukkarieh
[ paper ]
Developing ARET: An NLP-based Educational Tool Set for Arabic Reading Enhancement
Mohammed Maamouri, Wajdi Zaghouani, Violetta Cavalli-Sforza, Dave Graff and Mike Ciul
[ paper ]
Generating Diagnostic Multiple Choice Comprehension Cloze Questions
Jack Mostow and Hyeju Jang
[ paper ]
Generating Grammar Exercises
Laura Perez-Beltrachini, Claire Gardent and German Kruszewski
[ paper ]
A Comparison of Greedy and Optimal Assessment of Natural Language Student Input Using Word-to-Word Similarity Metrics
Vasile Rus and Mihai Lintean
[ paper ]
On Improving the Accuracy of Readability Classification using Insights from Second Language Acquisition
Sowmya Vajjala and Detmar Meurers
[ paper ]
An Interactive Analytic Tool for Peer-Review Exploration
Wenting Xiong, Diane Litman, Jingtao Wang and Christian Schunn
[ paper ]
Vocabulary Profile as a Measure of Vocabulary Sophistication
Su-Youn Yoon, Suma Bhat and Klaus Zechner
[ paper ]
Short Answer Assessment: Establishing Links Between Research Strands
Ramon Ziai, Niels Ott and Detmar Meurers
[ paper ]
HOO2012 Posters
Detection and Correction of Preposition or Determiner Error in English: HOO 2012
Pinaki Bhaskar, Aniruddha Ghosh, Santanu Pal and Sivaji Bandyopadhyay
[ paper ]
Informing Determiner and Preposition Error Correction with Hierarchical Word Clustering
Adriane Boyd, Marion Zepf and Detmar Meurers
[ paper ]
NUS at the HOO 2012 Shared Task
Daniel Dahlmeier, Hwee Tou Ng and Eric Jun Feng Ng
[ paper ]
VTEX determiner and preposition correction system for the HOO2012 shared task
Vidas Daudaravicius
[ paper ]
Precision Isn't Everything: A Hybrid Approach to Grammatical Error Detection
Michael Heilman, Aoife Cahill and Joel Tetreault
[ paper ]
HOO 2012 Error Recognition and Correction Shared Task: Cambridge University Submission Report
Ekaterina Kochmar, Øistein Andersen and Ted Briscoe
[ paper ]
Korea University System in HOO 2012 Shared Task
Jieun Lee, Jung-Tae Lee and Hae-Chang Rim
[ paper ]
A Naive Bayes classifier for automatic correction of preposition and determiner errors in ESL text
Gerard Lynch, Erwan Moreau and Carl Vogel
[ paper ]
KU Leuven at HOO-2012: A hybrid approach to detection and correction of determiner and preposition errors in non-native English text
Li Quan, Oleksandr Kolomiyets and Marie-Francine Moens
[ paper ]
The UI System in the HOO-2012 Shared Task on Error Correction
Alla Rozovskaya, Mark Sammons and Dan Roth
[ paper ]
NAIST at the HOO 2012 Shared Task
Keisuke Sakaguchi, Yuta Hayashibe, Shuhei Kondo, Lis Kanashiro, Tomoya Mizumoto, Mamoru Komachi and Yuji Matsumoto
[ paper ]
Memory-based text correction for preposition and determiner errors
Antal van den Bosch and Peter Berck
[ paper ]
Helping Our Own: NTHU NLPLAB System Description
Jian-Cheng Wu, Joseph Chang, Yi-Chun Chen, Shih-Ting Huang, Mei-Hua Chen and Jason S. Chang
[ paper ]
HOO Challenge 2012: UKP Lab System Description
Torsten Zesch and Jens Haase
[ paper ]
3:30 - 4:00 Break
4:00 - 4:25 Crowdsourced Comprehension: Predicting Prerequisite Structure in Wikipedia
Partha Talukdar and William Cohen
[ paper ] [ slides ]
4:25 - 4:50 Sense-Specific Lexical Information for Reading Assistance
Soojeong Eom, Markus Dickinson and Rebecca Sachs
[ paper ] [ slides ]
4:50 - 5:15 Evaluating the Meaning of Answers to Reading Comprehension Questions: A Semantics-Based Approach
Michael Hahn and Detmar Meurers
[ paper ] [ slides ]
5:15 - 5:30 Closing Remarks
7:00 - Post-workshop dinner a Resaurant Kashmir at 138, Rue Saint-Paul Est, Montreal.
[ Directions from the NAACL conference site to Restaurant Kashmir ]
All are invited, but please let Joel know you are coming so we can get a headcount..

Submission Information

We will be using the NAACL-HLT 2012 Submission Guidelines for the BEA-7 Workshop this year. Authors are invited to submit a full paper of up to 8 pages in electronic, PDF format, with up to 2 additional pages for references. We also invite short papers of up to 4 pages, including 2 additional pages for references. Please note that the short paper guidelines (4 + 2) differ from the NAACL main conference guidelines (4 pages total). Papers which describe systems are also invited to give a demo of their system. If you would like to present a demo in addition to presenting the paper, please make sure to select either "full paper + demo" or "short paper + demo" under "Submission Category" in the START submission page.

Previously published papers cannot be accepted. The submissions will be reviewed by the program committee. As reviewing will be blind, please ensure that papers are anonymous. Self-references that reveal the author's identity, e.g., "We previously showed (Smith, 1991) ...", should be avoided. Instead, use citations such as "Smith previously showed (Smith, 1991) ...".

Please use the 2012 NAACL-HLT style sheet for composing your paper:

We will be using the START conference system to manage submissions:

Please note that we are using the same submission page for both the BEA7 (deadline April 04 at 23:59 EST) and the HOO-2012 Shared Task (paper deadline May 04). When submitting to the HOO2012, please select "HOO2012" under "Submission Category."

Important Dates

Program Committee

Related Links

Other Upcoming Educational/NLP Conferences