This study investigates adapting vision-language models for medical image classification using MRI brain tumor images. CLIP shows promise in general image tasks but is limited in medical domains due to semantic gaps. Using Context Optimization (CoOp) on a dataset of 7,023 MRI images (glioma, meningioma, pituitary, and no tumor), we improved CLIP's accuracy from 26.3% to 94.8% by optimizing prompt tokens. This demonstrates that prompt tuning can bridge the domain gap and enhance medical diagnostic systems.

Ян Ц. (науч. рук. Жукова Н.А.) Enhancing brain tumor classification via prompt optimization in vision-language model // Сборник тезисов докладов конгресса молодых ученых. Электронное издание. – СПб: Университет ИТМО, [2025]. URL: https://kmu.itmo.ru/digests/article/14113