The Monroe Corpus

Home | People | Projects | Publications | Resources

General | Participants | Tasks | Setup | Dialogue stats | Processing | Transcripts and annotations

Comments on the dialogues

Table 1: Comments on dialogues
Dialogue Participants Task Completed Extra task Completed
      initial task? assigned? extra task?
1 S1R, S1B 1 yes yes yes
2 S1B, S1R 2 yes yes yes
3 S2B, S2R 3 yes yes no
4 S2R, S2B 4 yes yes yes
5 S3B, S3R 5 yes no  
6 S3R, S3B 6 yes no  
7 S4R, S4B 8 yes no  
8 S4B, S4R 7 yes no  
9 S5B, S5R 5 yes no  
10 S5R, S5B 1 yes no  
11 S6R, S6B 6 yes no  
12 S6B, S6R 2 yes no  
13 S7B, S7R 7 yes no  
14 S7R, S7B 3 yes yes yes
15 S8R, S8B 8 yes yes yes
16 S8B, S8R 4 yes yes yes
17 S9R, S9B 8 yes yes no
18 S9B, S9R 1 yes yes yes
19 S10B, S10R 4 yes yes yes
20 S10R, S10B 5 yes no  

Dialogue 1: S1R has trouble hearing S1B, who speaks with the microphone a little far away from his face. Tape starts a little into the dialogue.
Dialogues 7, 8: S4B says ``three'' like ``tree''. In dialogue 7, S4R invents vehicles.
Dialogues 13, 14: Both speakers have colds. Phone rings midway through dialogue 14.
Dialogues 15, 16: S8R says ``ambulance'' in an unusual way.
Dialogues 17, 18: S9R either chews his fingers or puts things in his mouth.
Dialogues 19, 20: S10B has a cold.

All dialogues ended with the participants thinking that they had solved every part of the task. While this was generally true, in one case, participants failed to solve the extra problem given at the end, and in another case, participants invented extra police units and ambulances to help them solve the task (table 1). Participants did not limit themselves to discussing the task. They made meta-comments about the domain or task, occasionally commented if they couldn't hear the other participant, and sometimes made jokes to each other. There was more of this type of interaction when the participants knew each other. Monroe county was known to most of the participants. This was helpful because fewer participants spent long periods of time trying to find locations, and so they were able to concentrate on solving the task. However, it did also mean that the participants made use of locations (e.g. roads) and names not on the map or in the task description. Because the dialogue participants were not allowed to make eye contact during the data collection, we can observe a wide variety of language use (including location descriptions), but fewer deictics and longer dialogues. In order to test our hypothesis that a lack of grounding does significantly affect dialogue participants, we had two of our participants return and conduct two dialogues each with one experimenter, solving tasks they had not previously solved. In the first dialogue, the experimenter behaved as a human would, processing speech in real-time and providing feedback. In the second, the experimenter did not back-channel, perform grounding acts or interrupt. The people took longer to solve the task in the second dialogue, paused longer between turns, and said they thought the experimenter was not paying attention or not understanding. One exhibited a great deal of frustration, and eventually stopped trying to solve the task and fell silent, forcing the experimenter to take over. These four dialogues do not form part of the Monroe corpus. They provide merely anecdotal evidence supporting claims that humans rely on other humans' providing appropriate conversational behaviors. They neither prove nor disprove that humans would find interactions with dialogue systems more effective if the system provided human-like conversational behaviors, but there are some intriguing clues. For instance, in the dialogues where the experimenter failed to provide feedback or take appropriate initiative, the inter-speaker pauses increased to more closely resemble those in typical human-computer dialogues, and the other participant's contributions became less free-flowing. It would be interesting to collect dialogues with a system using a similar experimental setup.

Monroe dialogue statistics

Table 2: Monroe dialogues: basic statistics
Dialogue Length Number of Number of Average Average
  (seconds) turns utterances turn length utt. length
1 458 112 129 3.31 4.26
2 705 206 352 1.68 3.37
3 557 77 145 2.38 6.35
4 1071 170 337 2.30 5.59
5 2341 540 918 2.06 4.21
6 1621 384 647 2.16 4.26
7 1711 459 735 1.66 3.25
8 1434 310 473 2.39 4.33
9 2432 490 783 2.59 4.79
10 584 111 163 3.12 5.19
11 973 198 295 2.74 4.64
12 415 99 164 2.30 4.27
13 2144 404 605 2.77 4.88
14 769 188 285 2.08 3.81
15 579 100 152 3.27 5.51
16 1015 201 329 2.66 4.98
17 890 218 342 1.84 3.42
18 1229 288 445 1.98 3.82
19 1603 372 564 2.28 3.96
20 1371 271 325 3.41 5.10
Average 1195.1 252 399 2.45 4.50

Table 2 shows, for each dialogue, the length of the dialogue, the number of turns and the number of utterances in that dialogue, and the average turn and utterance lengths. The total number of words in the corpus is approximately 52000; the number of unique words is approximately 1550 including partial words.

The length of the dialogues varies widely, depending partly on task complexity, but most of the dialogues exhibit a similar structure. The initial task holder describes the initial situation (including current tasks, resources, and constraints) to the other discourse participant. Then the initial task holder describes the plan so far. In these two phases the initial task holder typically holds the initiative, with the other discourse participant providing acknowledgments, backchannels, and clarification questions. The two discourse participants then collaborate to finish solving the task. In some dialogues, the initial task holder retains task initiative, but in others the other discourse participant takes initiative, and in some there is a good deal of competition for the initiative. Finally, one or both discourse participants summarize the plan. In those dialogues where a new task was given, the same process - task description, task solution, plan summary - is followed.

About this document ...

This document was generated using the LaTeX2HTML translator Version 98.1 release (February 19th, 1998)

Copyright © 1993, 1994, 1995, 1996, 1997, Nikos Drakos, Computer Based Learning Unit, University of Leeds.

The translation was initiated by Amanda Stent on 2001-12-11

Amanda Stent