A decoupling algorithm, based on mode signal, is proposed for the control of double deformable mirrors (DMs) in a woofer–tweeter adaptive optics system. The algorithm operates as follows: first, based on the orthogonality property of Zernike polynomials within the continuous region of the unit circle, we construct the signal vectors of the constrained modes of woofer and tweeter. Subsequently, control matrices for both the woofer and tweeter are generated by employing singular value decomposition and considering the response matrix of DM signal to slope. Finally, the aberration is determined using the measurement results from the Shack–Hartmann wavefront sensor, and the mode signal coefficient vectors of the two DMs are obtained through integration with the leaky integrator. These coefficient vectors are then multiplied by a transition matrix to derive mode signal control vectors for both the woofer and tweeter. Simulation results demonstrate that the proposed algorithm outperforms both the conventional Zernike mode decomposition control algorithm and the projection-based decoupling algorithm. The experiments demonstrate that the algorithm effectively utilizes the compensation capability of the woofer, enabling it to release tweeter stroke for compensating high-order aberrations and achieving superior correction outcomes.