3T: The Teacher Talk Tool for Instructional Observation
University of Pittsburgh, University of Colorado Boulder, and Oregon State University researchers have developed a non-intrusive system to analyze the quality of teacher talk in the classroom (classroom talk) and provide feedback to educators. This novel system, 3T, uses automatic speech recognition, natural language processing and machine learning to accurately assess classroom talk in real-world settings. 3T provides feedback to educators that could improve engagement with students, promote discourse, improve and experiment with teaching styles and aid overall school improvement.
Description
Engaging students in the classroom is vital to improving educational outcomes and achievements. Classroom talk strongly influences student engagement by promoting meaningful dialogue. In particular, the use of open-ended or “authentic” questions by teachers increases student engagement, understanding, and interest in the taught subject. However, many teachers face challenges incorporating dialogic discourse into regular teaching and need useful feedback to improve. 3T is designed to provide feedback to teachers to improve their use of dialogic discourse in the classroom and provide crucial data to identify effective teaching techniques to improve engagement and retention in students.Applications
• Teacher education and assessment• Development and improvement of teaching methods
Advantages
In-person teacher observation is used to provide teachers with feedback on their practice and to improve performance. These observational methods are time consuming, expensive, and due to the clear presence of an observer in the classroom, may distract students and not reflect a “real” classroom experience. Previous attempts to develop computer-based observation tools have still required human input and do not scale due to technology costs.In this novel approach, only teacher audio is collected from an easy-to-use and affordable commercially available microphone. This approach is designed not to interfere with teaching allowing for more real-world assessment. Teacher-only recording allays concerns around student privacy. Audio is assessed in a web-based service for processing and analysis using various language models with actionable feedback provided to teachers directly, removing the need for external supervision and assessment.
Compared to some automated technologies relying on large language models, 3T is validated on lower-cost, more interpretable, computationally-efficient models.
Invention Readiness
Classroom testing of the 3T system has confirmed the system reliably produces results comparable to human assessment. High school teachers have used the Teacher Talk Tool in their classrooms, confirming the high usability and validity of 3T feedback.IP Status
https://patents.google.com/patent/US20230154465A1Related Publication(s)
Kelly, S., Guner, G., Hunkins, N., & D’Mello, S. K. (2024). High School English Teachers Reflect on Their Talk: A Study of Response to Automated Feedback with the Teacher Talk Tool. International Journal of Artificial Intelligence in Education, 35(2), 879–913. https://doi.org/10.1007/s40593-024-00417-x
Jensen, E., Dale, M., Donnelly, P. J., Stone, C., Kelly, S., Godley, A., Sidney K. D'Mello, S. K., (2020). Toward Automated Feedback on Teacher Discourse to Enhance Teacher Learning. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376418
Kelly, S., Olney, A. M., Donnelly, P., Nystrand, M., & D’Mello, S. K. (2018). Automatically Measuring Question Authenticity in Real-World Classrooms. Educational Researcher, 47(7), 451-464. https://doi.org/10.3102/0013189X18785613