Proposal for the 12th Workshop on Innovative Use of NLP for Building Educational Applications

Workshop Description - NLI Shared Task - Submission Info - Organizing Committee - Program Committee - Related Links - Upcoming Education / NLP Events

Organization: Joel Tetreault (Grammarly), Jill Burstein (Educational Testing Service), Claudia Leacock (Consultant), Helen Yannakoudakis (University of Cambridge)

Contact Email:

Workshop Description

[ BEA12 Workshop Proposal ]

The BEA Workshop, one of the largest one-day workshops in the ACL community, is a leading venue for NLP innovation for educational applications. The workshop’s continuous growth illustrates an alignment between societal need and technological advances. NLP capabilities now support an array of learning domains, including writing, speaking, reading, science, and mathematics, as well as the related intra- (e.g., self-confidence) and inter-personal (e.g., collaboration) domains that support achievement in those domains. Automated writing evaluation (AWE) and speech scoring applications are commercially deployed in high-stakes assessment and instructional contexts, including primary and secondary educational settings. Commercially-deployed plagiarism detection is also prevalent. The current educational and assessment landscape in K-12, higher education, and adult learning (in academic and workplace settings) fosters a strong interest in technologies that yield analytics to support proficiency measures for complex constructs. For writing, there is a focus on innovation that supports writing tasks requiring source use, argumentative discourse, and factual content accuracy. For speech, there is an interest in advancing automated scoring to include the evaluation of discourse and content features in responses to spoken assessments. General advances in speech technology have promoted a renewed interest in spoken dialog and multimodal systems for instruction and assessment in, for example, workplace interviews and simulated teaching environments. The explosive growth of mobile applications for game-based and simulation applications for instruction and assessment is another place where NLP has begun to play a large role, especially in language learning.

NLP for educational applications has gained visibility outside of the NLP community. First, the Hewlett Foundation reached out to public and private sectors and sponsored two competitions – one for automated essay scoring, and another for scoring of short response items – to engage the larger scientific community in this enterprise. Subsequently, EdX, a non-profit enterprise founded by Harvard and MIT, announced the release of software designed to automatically grade text using AI technologies. Learning@Scale, a relatively new venue for NLP research in education, promotes interdisciplinary research on learning and teaching. Massive Open Online Courses (MOOCs) now incorporate AWE systems to manage the thousands of assignments that may be received during a single MOOC course. MOOCs for Refugees have more recently popped up in response to current social situations. Courses include language learning, and AWE and other NLP capabilities could support the coursework.

Research into education-related problems has also continued to grow within the NLP community. The last five years saw eight shared tasks (including five on grammatical error correction), two workshops on NLP and Computer-Assisted Language Learning (TEA series), and two sessions of the ACL devoted exclusively to the role of NLP in assisting language learners. Three of the shared tasks were sponsored by BEA. At ACL 2016, there were over 12 papers devoted to NLP in education, the highest number ever at a conference of this caliber. All of this has served to increase the visibility of, and interest in, the field of education applications in NLP.

The 12th workshop will have oral presentation sessions and a large poster session in order to maximize the amount of original work presented (the workshop normally has 50+ attendees). We will invite both full papers and short papers on topics including: automated scoring of textual and spoken responses, intelligent tutoring, peer review, grammatical error detection/correction, learner cognition, spoken dialog, multimodal applications, tools for teachers and test developers, and use of corpora. Research that incorporates NLP for use with mobile and game-based platforms will be of special interest. Specific topics include:

NLI Shared Task

The workshop will also host the 2nd Shared Task on Native Language Identification. The first edition, also held with the BEA workshop in 2013, was a total success with 29 teams competing, making it one of the largest shared tasks that year. Native Language Identification (NLI) is the goal of automatically classifying the native language (L1) of a writer based solely on his or her essay writing in another language. For the second edition, we are partnering with speech researchers and are modifying the task to make the prediction based not only an essay sample but also a speech sample where the same writer is asked to orally produce a response to a short answer essay question. There will be three tracks for the task: NLI on the essay only, NLI on the speech response only, and NLI using both responses. We feel this will make for a more challenging shared task while building on the methods and results from 2013.

Submission Information

We will be using the ACL Submission Guidelines for the BEA12 Workshop this year. Authors are invited to submit a full paper of up to 9 pages of content with up to 2 additional pages for references. We also invite short papers of up to 5 pages of content, including 2 additional pages for references. Please note that unlike previous years, final, camera ready versions of accepted papers will not be given an additional page to address reviewer comments.

Papers which describe systems are also invited to give a demo of their system. If you would like to present a demo in addition to presenting the paper, please make sure to select either "full paper + demo" or "short paper + demo" under "Submission Category" in the START submission page.

Previously published papers cannot be accepted. The submissions will be reviewed by the program committee. As reviewing will be blind, please ensure that papers are anonymous. Self-references that reveal the author's identity, e.g., "We previously showed (Smith, 1991) ...", should be avoided. Instead, use citations such as "Smith previously showed (Smith, 1991) ...".

We have also included conflict of interest in the submission form. You should mark all potential reviewers who have been authors on the paper, are from the same research group or institution, or who have seen versions of this paper or discussed it with you.

We will be using the START conference system to manage submissions.

Organizing Committee

Program Committee

Related Links

Other 2016 and 2017 Educational/NLP Events