AI-Generated Responses to Sensitive Pesantren Cases for Musyrif/Musyrifah Counseling Communication Training: A Snapshot Study

Abstract

Pesantren increasingly face psychosocial issues such as bullying and harassment, requiring sensitive interventions. Musyrif/musyrifah often serve as first responders but lack formal counseling or Psychological First Aid (PFA) training. This snapshot study examines the feasibility of large language models (LLMs) as simulators for developing counseling communication skills in pesantren contexts. Five LLMs: Gemini, DeepSeek, ChatGPT, Claude, and Meta AI, were tested using a culturally relevant case scenario. Their counseling responses were evaluated through a seven-skill rubric: attending, empathy, reflection, clarification, paraphrasing, summarizing, and nonverbal sensitivity. Five raters independently scored the outputs. Gemini achieved the highest mean (M = 4.89), followed by DeepSeek (M = 4.71) and ChatGPT-5 (M = 4.57). Claude (M = 3.86) and Meta AI (M = 3.20) scored lower, especially in nonverbal sensitivity. The study concludes that advanced LLMs, particularly Gemini and DeepSeek, show strong potential as scalable, culturally adaptive tools to strengthen counseling readiness in pesantren.

Keywords: AI-assisted counseling, psychological first aid counseling communication, pesantren mental health, safe learning environment