The utilization of chest X-ray (CXR) image data analysis for assisting in disease diagnosis is an important application of artificial intelligence. Supervised learning faces challenges due to a lack of large-scale labeled datasets and inaccuracies. Self-supervised learning offers a potential solution, but current research in this area is limited, and the diagnostic accuracy remains unsatisfactory. We propose an approach that integrates the self-supervised Bidirectional Encoder Representations from Image Transformers version 2 (BEiTv2) method with the vector quantization-based knowledge distillation (VQ-KD) strategy into CXR image data to enhance disease diagnosis accuracy. Our methodology demonstrates superior performance compared with existing self-supervised methods, showcasing its efficacy in improving diagnostic outcomes. Through transfer and ablation studies, we elucidate the benefits of the VQ-KD strategy in enhancing model performance and transferability to downstream tasks.