The 12th Workshop on Innovative Use of NLP for Building Educational Applications
EMNLP 2017 Workshops
Conference: EMNLP 2017
Organization: Joel Tetreault (Grammarly), Jill Burstein (Educational Testing Service), Ekaterina Kochmar (University of Cambridge), Claudia Leacock (Consultant), Helen Yannakoudakis (University of Cambridge)
Contact Email: email@example.com
Date : September 08, 2017
Venue : Copenhagen, Denmark
We are pleased to announce that Turnitin Lightside Labs, Grammarly and Duolingo are all gold level sponsors of the BEA12 workshop. Silver level sponsors are Educational Testing Service, Pacific Metrics, National Board of Medical Examinders and iLexIR. Cognii is our bronze level sponsor. If you or your company or institution are interested in sponsoring the BEA12, please send us an email at firstname.lastname@example.org. Sponsorship goes toward subsidizing dinner for students attending the workshop and free t-shirts with registration.
The BEA Workshop, one of the largest one-day workshops in the ACL community, is a leading venue for NLP innovation for educational applications. The workshop’s continuous growth illustrates an alignment between societal need and technological advances. NLP capabilities now support an array of learning domains, including writing, speaking, reading, science, and mathematics, as well as the related intra- (e.g., self-confidence) and inter-personal (e.g., collaboration) domains that support achievement in those domains. Automated writing evaluation (AWE) and speech scoring applications are commercially deployed in high-stakes assessment and instructional contexts, including primary and secondary educational settings. Commercially-deployed plagiarism detection is also prevalent. The current educational and assessment landscape in K-12, higher education, and adult learning (in academic and workplace settings) fosters a strong interest in technologies that yield analytics to support proficiency measures for complex constructs. For writing, there is a focus on innovation that supports writing tasks requiring source use, argumentative discourse, and factual content accuracy. For speech, there is an interest in advancing automated scoring to include the evaluation of discourse and content features in responses to spoken assessments. General advances in speech technology have promoted a renewed interest in spoken dialog and multimodal systems for instruction and assessment in, for example, workplace interviews and simulated teaching environments. The explosive growth of mobile applications for game-based and simulation applications for instruction and assessment is another place where NLP has begun to play a large role, especially in language learning.
NLP for educational applications has gained visibility outside of the NLP community. First, the Hewlett Foundation reached out to public and private sectors and sponsored two competitions – one for automated essay scoring, and another for scoring of short response items – to engage the larger scientific community in this enterprise. Subsequently, EdX, a non-profit enterprise founded by Harvard and MIT, announced the release of software designed to automatically grade text using AI technologies. Learning@Scale, a relatively new venue for NLP research in education, promotes interdisciplinary research on learning and teaching. Massive Open Online Courses (MOOCs) now incorporate AWE systems to manage the thousands of assignments that may be received during a single MOOC course. MOOCs for Refugees have more recently popped up in response to current social situations. Courses include language learning, and AWE and other NLP capabilities could support the coursework.
Research into education-related problems has also continued to grow within the NLP community. The last five years saw eight shared tasks (including five on grammatical error correction), two workshops on NLP and Computer-Assisted Language Learning (TEA series), and two sessions of the ACL devoted exclusively to the role of NLP in assisting language learners. Three of the shared tasks were sponsored by BEA. At ACL 2016, there were over 12 papers devoted to NLP in education, the highest number ever at a conference of this caliber. All of this has served to increase the visibility of, and interest in, the field of education applications in NLP.
The 12th workshop will have oral presentation sessions and a large poster session in order to maximize the amount of original work presented (the workshop normally has 50+ attendees). We will invite both full papers and short papers on topics including: automated scoring of textual and spoken responses, intelligent tutoring, peer review, grammatical error detection/correction, learner cognition, spoken dialog, multimodal applications, tools for teachers and test developers, and use of corpora. Research that incorporates NLP for use with mobile and game-based platforms will be of special interest. Specific topics include:
The workshop will also host a Shared Task on Native Language Identification (NLI). NLI is the process of automatically identify the native language (L1) of a non-native speaker based solely on language that he or she produces in another language. Two previous shared tasks on NLI have been organized in which the task was to identify the native language of non-native speakers of English-based on essays and spoken responses they provided during a standardized assessment of academic English proficiency. The first shared task was based on the essays only and was also held with the BEA workshop in 2013. It was a total success with 29 teams competing, making it one of the largest shared tasks that year. Three years later, Computational Paralinguistics Challenge at Interspeech 2016 hosted a sub-challenge on identifying the native language based solely on the spoken responses.
This year's shared task combines the inputs from the two previous tasks. There will be three tracks: NLI on the essay only, NLI on the speech response only, and NLI using both responses from a test taker. We feel this will make for a more challenging shared task while building on the methods and results from the previous two shared tasks. The training and development data for the shared task will be available in February 2017.
The organizing committee for the shared task is: Aoife Cahill (Educational Testing Service), Keelan Evanini (Educational Testing Service), Shervin Malmasi (Harvard Medical School), Joel Tetreault (Grammarly).
More details can be found on the shared task website: https://sites.google.com/site/nlisharedtask/.
We will be using the EMNLP Submission Guidelines for the BEA12 Workshop this year. (Please note we initially had the ACL Guidelines as the point of reference, but use the EMNLP ones. The two are extremely similar and you shouldn't see any differences when it comes to formatting and paper length). Authors are invited to submit a full paper of up to 8 pages of content with unlimited pages for references. We also invite short papers of up to 4 pages of content, including unlimited pages for references. Final camera ready versions of accepted papers will be given an additional page of content to address reviewer comments.
Papers which describe systems are also invited to give a demo of their system. If you would like to present a demo in addition to presenting the paper, please make sure to select either "full paper + demo" or "short paper + demo" under "Submission Category" in the START submission page.
Previously published papers cannot be accepted. The submissions will be reviewed by the program committee. As reviewing will be blind, please ensure that papers are anonymous. Self-references that reveal the author's identity, e.g., "We previously showed (Smith, 1991) ...", should be avoided. Instead, use citations such as "Smith previously showed (Smith, 1991) ...".
We have also included conflict of interest in the submission form. You should mark all potential reviewers who have been authors on the paper, are from the same research group or institution, or who have seen versions of this paper or discussed it with you.
We will be using the START conference system to manage submissions: https://www.softconf.com/emnlp2017/bea2017/
As in the above, please make sure you are using the EMNLP Submission Guidelines. Thanks!
Oral Presentations: Long papers accepted for oral presentations are allotted 20 minutes for the talk and 5 minutes for questions. Short papers that are accepted for oral presentations are allotted 15 minutes for the talk and 5 minutes for questions.
Poster Presentations: All papers accepted for a poster presentation will be presented in the session after lunch between 2:00 and 3:30. The posterboards will be self-standing, on top of tables (giving room for laptops, business cards, handouts, etc). The posters should be sized for A0 Landscape. Double-sided tape, pushpins, etc. for affixing the posters to the boards will be provided.
|[ BEA12 Proceedings ]
|8:45 - 9:00||Loading in of Oral Presentations|
|9:00 - 9:15||Opening Remarks|
|9:15 - 9:40||Question Difficulty – How to Estimate Without Norming, How to Use for Automated
|9:40 - 10:05||Combining CNNs and Pattern Matching for Question Interpretation in a Virtual
Patient Dialogue System
Lifeng Jin, Michael White, Evan Jaffe, Laura Zimmerman and Douglas Danforth
|10:05 - 10:30||Continuous fluency tracking and the challenges of varying text complexity
Beata Beigman Klebanov, Anastassia Loukina, John Sabatini and Tenaha O’Reilly
|10:30 - 11:00||Break|
|11:00 - 11:25||Auxiliary Objectives for Neural Error Detection Models
Marek Rei and Helen Yannakoudakis
|11:25 - 11:50||Linked Data for Language-Learning Applications
Robyn Loughnane, Kate McCurdy, Peter Kolb and Stefan Selent
|11:50 - 12:10||Predicting Specificity in Classroom Discussion
Luca Lugini and Diane Litman
|12:10 - 12:35||A Report on the 2017 Native Language Identification Shared Task
Shervin Malmasi, Keelan Evanini, Aoife Cahill, Joel Tetreault, Robert Pugh, Christopher Hamill, Diane Napolitano and Yao Qian
|12:35 - 14:00||Lunch|
|14:00 - 15:30||BEA12 Poster and Demo Session|
|14:00 - 14:45||BEA12 Poster and Demo Session A|
|Evaluation of Automatically Generated Pronoun Reference Questions
Arief Yudha Satria and Takenobu Tokunaga
|Predicting Audience’s Laughter During Presentations Using Convolutional Neural
Lei Chen and Chong Min Lee
|Collecting fluency corrections for spoken learner English
Andrew Caines, Emma Flint and Paula Buttery
|Exploring Relationships Between Writing & Broader Outcomes With Automated
Jill Burstein, Dan McCaffrey, Beata Beigman Klebanov and Guangming Ling
|An Investigation into the Pedagogical Features of Documents
Emily Sheng, Prem Natarajan, Jonathan Gordon and Gully Burns
|Combining Multiple Corpora for Readability Assessment for People with Cognitive
Victoria Yaneva, Constantin Orasan, Richard Evans and Omid Rohanian
|Automatic Extraction of High-Quality Example Sentences for Word Learning Using
a Determinantal Point Process
Arseny Tolmachev and Sadao Kurohashi
|Distractor Generation for Chinese Fill-in-the-blank Items
Shu Jiang and John Lee
|An Error-Oriented Approach to Word Embedding Pre-Training
Youmna Farag, Marek Rei and Ted Briscoe
|Investigating neural architectures for short answer scoring
Brian Riordan, Andrea Horbach, Aoife Cahill, Torsten Zesch and Chong Min Lee
|Human and Automated CEFR-based Grading of Short Answers
Anaïs Tack, Thomas François, Sophie Roekhaut and Cédrick Fairon
|GEC into the future: Where are we going and how do we get there?
Keisuke Sakaguchi, Courtney Napoles and Joel Tetreault
|Detecting Off-topic Responses to Visual Prompts
|Combining Textual and Speech Features in the NLI Task Using State-of-the-Art Machine
Pavel Ircing, Jan Svec, Zbynek Zajic, Barbora Hladka and Martin Holub
|Native Language Identification Using a Mixture of Character and Word N-grams
Elham Mohammadi, Hadi Veisi and Hessam Amini
|Ensemble Methods for Native Language Identification
Sophia Chan, Maryam Honari Jahromi, Benjamin Benetti, Aazim Lakhani and Alona Fyshe
|Can string kernels pass the test of time in Native Language Identification?
Radu Tudor Ionescu and Marius Popescu
|Neural Networks and Spelling Features for Native Language Identification
Johannes Bjerva, Gintare Grigonyte, Robert Östling and Barbara Plank
|A study of N-gram and Embedding Representations for Native Language Identification
Sowmya Vajjala and Sagnik Banerjee
|A Shallow Neural Network for Native Language Identification with Character Ngrams
Yunita Sari, Muhammad Rifqi Fatchurrahman and Meisyarah Dwiastuti
|Fewer features perform well at Native Language Identification task
Taraka Rama and Ça˘grı Çöltekin
|14:45 - 15:30||BEA11 Poster and Demo Session B|
|Structured Generation of Technical Reading Lists
Jonathan Gordon, Stephen Aguilar, Emily Sheng and Gully Burns
|Effects of Lexical Properties on Viewing Time per Word in Autistic and Neurotypical
Sanja Štajner, Victoria Yaneva, Ruslan Mitkov and Simone Paolo Ponzetto
|Transparent text quality assessment with convolutional neural networks
Robert Östling and Gintare Grigonyte
|Artificial Error Generation with Machine Translation and Syntactic Patterns
Marek Rei, Mariano Felice, Zheng Yuan and Ted Briscoe
|Modelling semantic acquisition in second language learning
Ekaterina Kochmar and Ekaterina Shutova
|Multiple Choice Question Generation Utilizing An Ontology
Katherine Stasaski and Marti A. Hearst
|Simplifying metaphorical language for young readers: A corpus study on news text
Magdalena Wolska and Yulia Clausen
|Language Based Mapping of Science Assessment Items to Skills
Farah Nadeem and Mari Ostendorf
|Connecting the Dots: Towards Human-Level Grammatical Error Correction
Shamil Chollampatt and Hwee Tou Ng
|Question Generation for Language Learning: From ensuring texts are read to supporting
Maria Chinkina and Detmar Meurers
|Systematically Adapting Machine Translation for Grammatical Error Correction
Courtney Napoles and Chris Callison-Burch
|Fine-grained essay scoring of a complex writing task for native speakers
Andrea Horbach, Dirk Scholten-Akoun, Yuning Ding and Torsten Zesch
|Exploring Optimal Voting in Native Language Identification
Cyril Goutte and Serge Léger
|CIC-FBK Approach to Native Language Identification
Ilia Markov, Lingzhen Chen, Carlo Strapparava and Grigori Sidorov
|The Power of Character N-grams in Native Language Identification
Artur Kulmizev, Bo Blankers, Johannes Bjerva, Malvina Nissim, Gertjan van Noord, Barbara Plank and Martijn Wieling
|Classifier Stacking for Native Language Identification
Wen Li and Liang Zou
|Native Language Identification on Text and Speech
Marcos Zampieri, Alina Maria Ciobanu and Liviu P. Dinu
|Native Language Identification using Phonetic Algorithms
Charese Smiley and Sandra Kübler
|A deep-learning based native-language classification by using a latent semantic
analysis for the NLI Shared Task 2017
Yoo Rhee Oh, Hyung-Bae Jeon, Hwa Jeon Song, Yun-Kyung Lee, Jeon-Gue Park and Yun-Keun Lee
|Fusion of Simple Models for Native Language Identification
Fabio Kepler, Ramón Astudillo and Alberto Abad
|Stacked Sentence-Document Classifier Approach for Improving Native Language
Andrea Cimino and Felice Dell’Orletta
|15:30 - 16:00||Break|
|16:00 - 16:25||Using Gaze to Predict Text Readability
Ana Valeria Gonzalez-Garduño and Anders Søgaard
|16:25 - 16:50||Annotating Orthographic Target Hypotheses in a German L1 Learner Corpus
Ronja Laarmann-Quante, Katrin Ortmann, Anna Ehlert, Maurice Vogel and Stefanie Dipper
|16:50 - 17:15||A Large Scale Quantitative Exploration of Modeling Strategies for Content Scoring
Nitin Madnani, Anastassia Loukina and Aoife Cahill
|17:15 - 17:30||Closing Remarks