Human brain is a complex system displaying modular characteristics in both structural and functional networks. Although coupled on a certain level, such two types of modular structures remain different in principle where the structural modules provide disciplines of anatomical organization, while the functional modules reflect the assembling principles of statistical association. Thus they must compromise with each other in supporting brain’s dynamic fluctuation. However there lacks community detection algorithm that can simultaneously identify modules on structural and functional brain networks thus leaves unknown how the brain makes trade-offs between the structural constraint and functional convenience. Here we present a novel framework of analyzing cross-modality community structure throughout applying the multi-object evolutionary algorithm (MOEA) to characterize the community partition on the Pareto-front where the modularity losses on both the structural and functional networks are concurrently optimized. We found that there were several primary community structures that supported the transition from modular structures of the structural network to that of the functional network. We found that regions with high participation coefficients in both structural and functional networks maintain high flexibility of facilitating the shift of structural community partition to the functional one. Further, the length of and area below the Pareto-front characterized the level of coupling between the two modality of community structure, fairly predicting the individual difference in executive performance. Together, we provide new analytical tools that make it possible to investigate the trade-offs among multiple community structures that frequently appeared when examining the topology reconfiguration of brain networks either across modality or temporal windows.