Object detection plays an important role in the field of remote sensing (RS) images analysis. The advancement of object detection task for RS images is extremely challenging due to object scale variation and complex background. Almost all detection frameworks use the neck network to fuse the feature maps extracted by the backbone to obtain better features, among which feature pyramid network (FPN) is the most widely used. Although traditional FPN has shown great potential in multi-scale object detection based on deep learning, it has unsatisfactory detection accuracy for small objects and confusing objects in RS images because of the lack of rich semantic and contextual information. We propose an architecture, called semantics reused context FPN that is portable to any FPN-based detectors to boost the detection performance in RS images without parameters increasing significantly. It includes two blocks, namely, context feature enhanced block, which uses dense connection and a learnable branch structure to extract rich context features with multiple receptive fields, and semantic feature reused block, which enhances semantic information of shallow feature maps by reusing later-layer features. Comprehensive evaluations on three benchmark datasets of geospatial object detection demonstrate that our method is superior to the existing methods.