The deep approach to AI-supported learning amplifies the benefits of AI's personalized guidance and strengthens the sense of human-AI learning community, while passive or mechanical approaches lead to poor outcomes despite high engagement levels.

Objective: This study aimed to investigate how students interact with artificial intelligence (AI) for English as a Foreign Language (EFL) learning and to identify what factors matter most in AI-supported language learning experiences.

Methods: The researchers conducted a three-month naturalistic study with 16 primary school students in China who used an AI coach for EFL learning. The researchers collected two types of data:

1. Students' usage data from the AI system (including frequencies of English shadowing, listening practice, and vocabulary learning, plus average scores)
2. Students' reflection essays about their interactions with the AI coach

The data was analyzed using:

• Two-step cluster analysis to identify distinct groups of students based on their engagement and performance
• Epistemic network analysis (ENA) to visualize and analyze the connections between different elements of students' learning experiences based on Community of Inquiry (CoI) and Students' Approaches to Learning (SAL) frameworks

Key Findings:

• Four distinct clusters of students emerged, each with different interaction patterns with the AI coach:
  • Cluster 1 (Effective Learners): Moderate engagement with highest performance scores, strong connections to deep learning approaches
  • Cluster 2 (Passive Learners): Lowest engagement and lowest scores, passive interaction with limited social presence
  • Cluster 3 (Well-Balanced Learners): High engagement and high scores, well-connected network of learning approaches
  • Cluster 4 (Inefficient Learners): Very high engagement but moderate scores, mechanically followed AI instructions
• The deep approach to learning played a crucial role in successful AI-supported learning, enhancing the benefits of AI's personalized guidance and strengthening the sense of human-AI learning community
• Teaching presence (especially feedback and guidance) was widely connected to other elements in successful students' learning networks
• Passively or mechanically following AI's instruction, even with high participation, decreased the sense of community and led to lower performance
• Social, cognitive, and teaching presences together created a more meaningful learning experience, but only when paired with deep learning approaches

Implications: The study demonstrates that:

1. Not all students automatically benefit from AI-supported learning; their approaches to learning significantly impact outcomes
2. The deep approach to learning is critical for maximizing AI benefits in language education
3. Human-AI learning should be viewed as a community with social, cognitive, and teaching presences
4. AI designers should focus on improving feedback algorithms to optimize student learning
5. Teachers should facilitate students' adoption of deep learning approaches rather than simply leaving them to interact with AI on their own

Limitations:

• Small sample size (16 students) from a single school in China
• Only relied on two data sources (system usage data and reflection essays)
• Unequal cluster sizes, with Cluster 2 having only two students
• Used only one type of AI agent (anthropomorphic virtual agent), limiting generalizability
• Analysis was confined by the combined CoI and SAL theoretical frameworks

Future Directions: Future research should:

• Include larger, more diverse participant samples across different disciplines and school levels
• Employ additional data collection techniques like follow-up interviews and questionnaires
• Experiment with different cluster analysis techniques and sample sizes
• Investigate different types of AI agents (mechanomorphic and cartoon-like)
• Explore alternative theoretical frameworks for understanding human-AI interactions

Title and Authors: "What Matters in AI-Supported Learning: A Study of Human-AI Interactions in Language Learning Using Cluster Analysis and Epistemic Network Analysis" by Xinghua Wang, Qian Liu, Hui Pang, Seng Chee Tan, Jun Lei, Matthew P. Wallace, and Linlin Li.

Published On: 2023

Published By: Computers & Education (Volume 194)