Intent Recognition Evaluation

Miloš Kravčík ¹, Hong Li ¹, Tamar Arndt ^2
1 DFKI Berlin, ² Universität Leipzig

Zusammenfassung
Intelligent assistants provide a lot of potential in development of scalable mentoring solutions for higher education. Chatbots became popular also in this field, but they may cause users’ frustration if they do not understand the user correctly. Therefore, it is crucial to evaluate their performance in recognizing user messages, in order to know their weak points and improve them accordingly. In our experiments we took one chatbot used bystudents of educational sciences and evaluated it from various perspectives. The results indicate that this kind of validation can help to improve the usability of chatbots in the learning domain.