Near-infrared radiation (NIR) has been described as one of the highest-resolution tools for neuromodulation. However, the poor tissue penetration depth of NIR has limited its further application on some of the deeper layer neurons in vivo. A 980-nm short-wavelength NIR (SW-NIR) with high penetration depth was employed, and its inhibitory effect on neurons was investigated in vivo. In experiments, SW-NIR was implemented on the rat’s cochlear nucleus (CN), the auditory pathway was activated by pure-tones through the rat’s external auditory canal, and the neural responses were recorded in the inferior colliculus by a multichannel electrode array. Neural firing rate (FR) and the first spike latency (FSL) were analyzed to evaluate the optically induced neural inhibition. Meanwhile, a two-layered finite element, consisting of a fluid layer and a gray matter layer, was established to model the optically induced temperature changes in CN; different stimulation paradigms were used to compare the inhibitory efficiency of SW-NIR. Results showed that SW-NIR could reversibly inhibit acoustically induced CN neural activities: with the increase of laser radiant exposures energy, neural FR decreased significantly and FSL lengthened steadily. Significant inhibition occurred when the optical pulse stimulated prior to the acoustic stimulus. Results indicated that the inhibition relies on the establishment time of the temperature field. Moreover, our preliminary results suggest that short-wavelength infrared could regulate the activities of neurons beyond the neural tissues laser irradiated through neural networks and conduction in vivo. These findings may provide a method for accurate neuromodulation in vivo.