With the popularity of unmanned aerial vehicle (UAV) technology, remote sensing target tracking in aerial videos from UAVs has drawn much attention. However, various problems occur, such as obvious target scale changes and frequent target shape updates in UAV aerial videos, and typical tracking algorithms have difficulties in solving these challenges. Therefore, we propose a remote sensing target tracking method for UAV videos based on a multiscale antideformation network (MSADN). This method uses the fully convolutional Siamese network (SiameseFC) as its basic architecture. First, in the target feature extraction stage, we use a receptive field block module to change the single convolutional layer of the network. And the multibranch structure can improve the algorithm’s adaptability to target scale changes. Then, in the tracking stage, to solve the problem of the lack of template updates, we integrate a template dynamic update module into the SiameseFC architecture. This module uses long short-term memory to generate control signals to dynamically update target shape information, and it will improve the algorithm’s ability to cope with frequent target shape updates. Finally, compared to state-of-the-art trackers, experimental results show that the MSADN algorithm can obtain better performance (75.9% Prec, 56.4% Succ) while ensuring higher efficiency (60 fps).