The development of precision manufacturing puts forward higher requirements for the measurement accuracy of grating heterodyne interference displacement measurement, while its nonlinear error is one of the main factors that influence measurement accuracy. A synthetic model of nonlinear error including six error sources is established, and the influence of various nonlinear error sources is analyzed according to the phase expression of nonlinear error derived from the model. When the installation error γ  =  θ  =  1  deg and the performance of optical elements α  =  β  =  0.5  deg, dε₁  =  0.1  deg, dε₂  =    −  0.1  deg, T_p  =  0.95, R_p  =  0.05, R_s  =  0.98, T_s  =  0.02, η_p  =  0.4, η_s  =  0.35, nonlinear error will reach 2.32 nm. We provide a theoretical basis for compensating the nonlinear error of grating interferometers.