Md Suhaimin, Mohd Suhairi and Arshad, Akashah and Sidek, Khairul Azami and Hamdan, Mohd Najib (2026) A bilingual fine-tuning framework for contextual LLM chatbots via multi-model synthetic data generation and selective grid search. In: 2026 10th International Conference on Mechatronics Engineering (ICOM), 8-9 September 2026, Kuala Lumpur.
![[img]](https://irep.iium.edu.my/style/images/fileicons/application_pdf.png) |
PDF
- Published Version
Restricted to Repository staff only Download (1MB) | Request a copy |
Abstract
Fine-tuning large language models (LLMs) for bilingual, domain-specific conversational tasks under constrained computational resources remain a significant practical challenge. This paper proposes the SynLoRA-SGS Framework, a four-stage pipeline that integrates: (1) multi-model synthetic data generation, (2) data aggregation and quality control, (3) Low-Rank Adaptation (LoRA)-based parameter-efficient fine-tuning, and (4) model merging and deployment. The framework leverages a teacher–student bilingual data generation strategy and prioritizes convergence stability through feedback-driven iterative refinement over exhaustive search, making it well suited for applied bilingual deployment environments. The framework is validated within the POLYCC LLM League 2025, a structured competition in which participants improved a provided lower-tier LLM under strict architectural and computational constraints. Experimental results across two evaluation stages demonstrate that bilingual dataset quality and targeted LoRA parameter selection (r and α) have a substantially greater impact on contextual response accuracy than dataset size alone. The final model achieved the highest overall ranking among six finalist teams under a multi-source evaluation. These findings suggest that the SynLoRA-SGS Framework offers a reproducible and efficient approach for bilingual contextual chatbot fine-tuning in constrained, real-world settings.
| Item Type: | Proceeding Paper (Invited Papers) |
|---|---|
| Additional Information: | 4698/132413 |
| Uncontrolled Keywords: | Large Language Model, Bilingual Chatbot, Synthetic Data Generation, Selective Grid Search, Fine-Tuning |
| Subjects: | T Technology > TK Electrical engineering. Electronics Nuclear engineering > TK7800 Electronics. Computer engineering. Computer hardware. Photoelectronic devices > TK7885 Computer engineering |
| Kulliyyahs/Centres/Divisions/Institutes (Can select more than one option. Press CONTROL button): | Kulliyyah of Engineering > Department of Electrical and Computer Engineering |
| Depositing User: | Assoc Prof Dr Khairul Azami Sidek |
| Date Deposited: | 04 Nov 2026 09:38 |
| Last Update: | 04 Nov 2026 09:38 |
| Queue Number: | 2026-10-Q5967 |
| URI: | http://irep.iium.edu.my/id/eprint/132413 |
| Indexed In: | Google Scholar |
Actions (login required)
 |
View Item |