To meet the demands of 24-hour video surveillance, the person re-identification task must tackle the challenges of single-mode and cross-mode re-identification at the same time. However, current person re-identification tasks often treat single-mode and cross-mode as separate branches, neglecting their complementary application values. To address this challenge, this paper introduces a transferable network architecture that can collaboratively train datasets for both single-mode and cross-mode person re-identification. The network deeply excavates person information from each modality through a three-way branch, and thoroughly examines the issues of domain gap and imbalanced modal samples within the collaborative optimization architecture, effectively addressing multi-task optimization issues. To narrow the domain gap, the paper introduces a low-level feature alignment strategy that significantly reduces differences in cross-domain sample features, enabling the model to learn domain-invariant semantic features, thus enhancing generalization. Moreover, the paper devises a weak modality feature mining strategy to handle modality sample imbalances, dynamically adjusting training weights to prioritize optimizing the weaker modality. The experimental results show that the proposed framework can be transferred to a large number of mainstream methods that employ ResNet as the backbone network. Compared to the method AGW, Rank1 and mAP are increased by 23.79 % and 17.78 % , respectively.