Feng, Jing (2025) Improved Crop Disease Detection Using Hybrid Vision Transformers And Knowledge Distillation. PhD thesis, Universiti Sains Malaysia.
![[img]](http://eprints.usm.my/style/images/fileicons/application_pdf.png) |
PDF
Download (596kB) |
Abstract
In the field of computer vision, transformer architectures have emerged as powerful alternatives to traditional convolutional neural networks, offering superior global modeling capabilities and remarkable performance in various visual recognition tasks. However, their large parameter size and reliance on extensive training data, coupled with a lack of efficient structures for fine-grained feature extraction, hinder their deployment in precise crop disease detection scenarios. In the domain of knowledge distillation, conventional teacher–student frameworks often suffer from mismatches between intermediate feature maps due to architectural depth differences. Moreover, the high dimensionality of intermediate feature maps results in quadratic growth in computational cost when directly performing feature map-level distillation, significantly reducing efficiency. To address these challenges, this work first improves the multi-head self-attention mechanism in ViT by introducing a Group-wise Attention design. The attention computation is partitioned into multiple sub-groups, within which parameters are shared and local attention is computed independently
| Item Type: | Thesis (PhD) |
|---|---|
| Subjects: | Q Science > QA Mathematics > QA1-939 Mathematics |
| Divisions: | Pusat Pengajian Sains Matematik (School of Mathematical Sciences) > Thesis |
| Depositing User: | Mr Aizat Asmawi Abdul Rahim |
| Date Deposited: | 26 Jun 2026 01:11 |
| Last Modified: | 26 Jun 2026 01:11 |
| URI: | http://eprints.usm.my/id/eprint/64458 |
Actions (login required)
 |
View Item |
