COMPARATIVE ANALYSIS OF PERSONALIZED RESPONSE GENERATION METHODS BASED ON MESSENGER CORPUS

Abstract

This paper presents a comparative study of four approaches to automated personalized text response generation in the communication style of a specific individual: TF-IDF retrieval, Markov chains (bigram model), and two fine-tuned neural language models – DistilGPT-2 (82M parameters) and DialoGPT-medium (345M parameters). The training corpus consists of Ukrainian-language private Telegram chat logs (~200,000 messages), from which 10,000 query-response pairs were extracted in an 80/20 train/test split. Neural models were fine-tuned on the Kaggle platform using an NVIDIA T4 GPU; inference and evaluation were performed on CPU. Evaluation was conducted on 500 test pairs across five metrics: BLEU, ROUGE-L, BERTScore-F1, average inference time, and peak RAM consumption. DistilGPT-2 achieved the highest BLEU (0.0088) and ROUGE-L (0.0535) scores and the best BERTScore F1 (0.687), while DialoGPT-medium underperformed across quality metrics despite consuming significantly more memory (Net RAM: 1343 MB vs. 297 MB). TF-IDF retrieval is viable for minimal hardware, and Markov chains are the fastest method (70 µs per query), but both lag behind neural approaches in semantic coherence. The findings indicate that fine-tuned DistilGPT-2 offers the best quality-to-resource trade-off for style personalization in a local CPU deployment scenario.