2024
Accurate structure prediction of immune proteins using parameter-efficient transfer learning
Tian Zhu*, Milong Ren*, Zaikai He, Siyuan Tao, Ming Li, Dongbo Bu, Haicang Zhang (* equal contribution)
Submitted to Nature Computational Science, Under review. 2024
We propose ImmuneFold, a transfer learning approach that fine-tunes ESMFold specifically for immune proteins. We leverage low-rank adaption (LoRA), a parameter-efficient fine-tuning technique that requires considerably less memory and substantially fewer parameters. Evaluations on various immune proteins, including T-cell receptors, antibodies, and nanobodies, demonstrate that ImmuneFold outperforms existing methods in prediction accuracy. Furthermore, we apply ImmuneFold to develop a zero-shot protocol for TCR-epitope binding prediction. Unlike previous supervised methods suffering from severe overfitting due to limited experimental binding data, our approach first predicts TCR-epitope structure using ImmuneFold and then directly estimates the binding affinity by calculating Rosseta energy.
Accurate structure prediction of immune proteins using parameter-efficient transfer learning
Tian Zhu*, Milong Ren*, Zaikai He, Siyuan Tao, Ming Li, Dongbo Bu, Haicang Zhang (* equal contribution)
Submitted to Nature Computational Science, Under review. 2024
We propose ImmuneFold, a transfer learning approach that fine-tunes ESMFold specifically for immune proteins. We leverage low-rank adaption (LoRA), a parameter-efficient fine-tuning technique that requires considerably less memory and substantially fewer parameters. Evaluations on various immune proteins, including T-cell receptors, antibodies, and nanobodies, demonstrate that ImmuneFold outperforms existing methods in prediction accuracy. Furthermore, we apply ImmuneFold to develop a zero-shot protocol for TCR-epitope binding prediction. Unlike previous supervised methods suffering from severe overfitting due to limited experimental binding data, our approach first predicts TCR-epitope structure using ImmuneFold and then directly estimates the binding affinity by calculating Rosseta energy.
Improving Molecular Graph Generation with Flow Matching and Optimal Transport
Tian Zhu*, Xiaoyang Hou*, Milong Ren, Dongbo Bu, Xin Gao, Chunming Zhang, Shiwei Sun (* equal contribution)
AIDrugX Workshop, Neural Information Processing Systems (NeurIPS) 2024
We propose GGFlow, a discrete flow matching generative model incorporating optimal transport for molecular graphs and it incorporates an edge-augmented graph transformer to enable the direct communications among chemical bounds. Additionally, GGFlow introduces a novel goal-guided generation framework to control the generative trajectory of our model, aiming to design novel molecular structures with the desired properties.
Improving Molecular Graph Generation with Flow Matching and Optimal Transport
Tian Zhu*, Xiaoyang Hou*, Milong Ren, Dongbo Bu, Xin Gao, Chunming Zhang, Shiwei Sun (* equal contribution)
AIDrugX Workshop, Neural Information Processing Systems (NeurIPS) 2024
We propose GGFlow, a discrete flow matching generative model incorporating optimal transport for molecular graphs and it incorporates an edge-augmented graph transformer to enable the direct communications among chemical bounds. Additionally, GGFlow introduces a novel goal-guided generation framework to control the generative trajectory of our model, aiming to design novel molecular structures with the desired properties.
GTAM: A Molecular Pretraining Model with Geometric Triangle Awareness
Xiaoyang Hou*, Tian Zhu*, Milong Ren*, Bo Duan, Chunming Zhang, Dongbo Bu, Shiwei Sun (* equal contribution)
Bioinformatics 2024
This method integrates innovative molecular encoders for both 2D graphs and 3D conformations, enabling the accurate capture of geometric dependencies among edges in graph-based molecular structures. Furthermore, GTAM is bolstered by the development of two contrastive training objectives designed to facilitate the direct transfer of edge information between 2D topological graphs and 3D geometric conformations, enhancing the functionality of the molecular encoders.
GTAM: A Molecular Pretraining Model with Geometric Triangle Awareness
Xiaoyang Hou*, Tian Zhu*, Milong Ren*, Bo Duan, Chunming Zhang, Dongbo Bu, Shiwei Sun (* equal contribution)
Bioinformatics 2024
This method integrates innovative molecular encoders for both 2D graphs and 3D conformations, enabling the accurate capture of geometric dependencies among edges in graph-based molecular structures. Furthermore, GTAM is bolstered by the development of two contrastive training objectives designed to facilitate the direct transfer of edge information between 2D topological graphs and 3D geometric conformations, enhancing the functionality of the molecular encoders.
CarbonNovo: Joint Design of Protein Structure and Sequence Using a Unified Energy-based Model
Milong Ren, Tian Zhu, Haicang Zhang
International Conference on Machine Learning (ICML) 2024
We propose CarbonNovo, a unified energy-based model for jointly generating protein structure and sequence. Specifically, we leverage a score-based generative model and Markov Random Fields for describing the energy landscape of protein structure and sequence. In CarbonNovo, the structure and sequence design module communicates at each diffusion step, encouraging the generation of more coherent structure-sequence pairs. Moreover, the unified framework allows for incorporating the protein language models as evolutionary constraints for generated proteins.
CarbonNovo: Joint Design of Protein Structure and Sequence Using a Unified Energy-based Model
Milong Ren, Tian Zhu, Haicang Zhang
International Conference on Machine Learning (ICML) 2024
We propose CarbonNovo, a unified energy-based model for jointly generating protein structure and sequence. Specifically, we leverage a score-based generative model and Markov Random Fields for describing the energy landscape of protein structure and sequence. In CarbonNovo, the structure and sequence design module communicates at each diffusion step, encouraging the generation of more coherent structure-sequence pairs. Moreover, the unified framework allows for incorporating the protein language models as evolutionary constraints for generated proteins.
Antibody Design Using a Score-based Diffusion Model Guided by Evolutionary, Physical and Geometric Constraints
Tian Zhu, Milong Ren, Haicang Zhang
International Conference on Machine Learning (ICML) 2024
We present AbX, a new score-based diffusion generative model guided by evolutionary, physical, and geometric constraints for antibody design. These constraints serve to narrow the search space and provide priors for plausible antibody sequences and structures. Specifically, we leverage a pre-trained protein language model as priors for evolutionary plausible antibodies and introduce additional training objectives for geometric and physical constraints like van der Waals forces. Furthermore, as far as we know, AbX is the first score-based diffusion model with continuous timesteps for antibody design, jointly modeling the discrete sequence space and the $SE(3)$ structure space.
Antibody Design Using a Score-based Diffusion Model Guided by Evolutionary, Physical and Geometric Constraints
Tian Zhu, Milong Ren, Haicang Zhang
International Conference on Machine Learning (ICML) 2024
We present AbX, a new score-based diffusion generative model guided by evolutionary, physical, and geometric constraints for antibody design. These constraints serve to narrow the search space and provide priors for plausible antibody sequences and structures. Specifically, we leverage a pre-trained protein language model as priors for evolutionary plausible antibodies and introduce additional training objectives for geometric and physical constraints like van der Waals forces. Furthermore, as far as we know, AbX is the first score-based diffusion model with continuous timesteps for antibody design, jointly modeling the discrete sequence space and the $SE(3)$ structure space.
2023
Predicting mutational effects on protein-protein binding via a side-chain diffusion probabilistic model
Shiwei Liu*, Tian Zhu*, Milong Ren, Chungong Yu, Dongbo Bu, Haicang Zhang (* equal contribution)
Neural Information Processing Systems (NeurIPS) 2023
In this work, we propose SidechainDiff, a representation learning-based approach that leverages unlabelled experimental protein structures. SidechainDiff utilizes a Riemannian diffusion model to learn the generative process of side-chain conformations and can also give the structural context representations of mutations on the protein-protein interface.
Predicting mutational effects on protein-protein binding via a side-chain diffusion probabilistic model
Shiwei Liu*, Tian Zhu*, Milong Ren, Chungong Yu, Dongbo Bu, Haicang Zhang (* equal contribution)
Neural Information Processing Systems (NeurIPS) 2023
In this work, we propose SidechainDiff, a representation learning-based approach that leverages unlabelled experimental protein structures. SidechainDiff utilizes a Riemannian diffusion model to learn the generative process of side-chain conformations and can also give the structural context representations of mutations on the protein-protein interface.