In this study, a polarization-maintaining fiber optical gyro (PMFOG) with orthogonal-polarization states is constructed, and the non-reciprocity error induced by varying temperatures has been studied. Compared with the traditional optical gyro system based on single-polarization, theory and experiments have shown that the non-reciprocity error induced by varying temperatures is more evident in the PMFOG with orthogonal-polarization states. It is approximately five times larger than that of the traditional PMFOG. To decrease the non-reciprocity error induced by varying temperatures, we proposed an innovative structure in this study, which enhances the symmetry of the light path without the addition of redundant optical devices. The clockwise and counterclockwise light beams experience the same change of refractive index along the fast and slow axes of the fiber, and the non-reciprocity error induced by varying temperatures can be reduced to zero when the whole gyro is heated up nearly uniformly. This research, which overcomes a design limitation in PMFOGs with orthogonal-polarization states, is significant for the miniaturization and optimization of the environmental adaptability of fiber optic gyroscopes.