Designs to Improve Capability of Neural Networks to Make Structural Predictions

Tian-Yao Wang; Jian-Feng Li; Hong-Dong Zhang; Jeff Z. Y. Chen

doi:10.1007/s10118-023-2910-x

Your Location：

Home >

Browse articles >

Designs to Improve Capability of Neural Networks to Make Structural Predictions

RESEARCH ARTICLE | Updated：2023-01-31

- Designs to Improve Capability of Neural Networks to Make Structural Predictions
- Chinese Journal of Polymer Science Vol. 41, Pages: 1-9(2022)
- Affiliations：
  
  a.Department of Macromolecular Science, State Key Laboratory of Macromolecular Engineering of Polymers, Fudan University, Shanghai 200438, China
  b.Department of Physics and Astronomy, University of Waterloo, Waterloo N2L 3G1, Canada
- Author bio：
  
  lijf@fudan.edu.cn (J.F.L.)
  jeffchen@uwaterloo.ca (J.Z.Y.C.)
- Funds：
- DOI：10.1007/s10118-023-2910-x
  CLC：
Scan for full text
Tian-Yao Wang, Jian-Feng Li, Hong-Dong Zhang, et al. Designs to Improve Capability of Neural Networks to Make Structural Predictions. [J/OL]. Chinese Journal of Polymer Science 411-9(2022)
DOI：

Tian-Yao Wang, Jian-Feng Li, Hong-Dong Zhang, et al. Designs to Improve Capability of Neural Networks to Make Structural Predictions. [J/OL]. Chinese Journal of Polymer Science 411-9(2022) DOI： 10.1007/s10118-023-2910-x.

摘要

Abstract

A deep neural network model generally consists of different modules that play essential roles in performing a task. The optimal design of a module for use in modeling a physical problem is directly related to the success of the model. In this work, the effectiveness of a number of special modules, the self-attention mechanism for recognizing the importance of molecular sequence information in a polymer, as well as the big-stride representation and conditional random field for enhancing the network ability to produce desired local configurations, is numerically studied. Network models containing these modules are trained by using the well documented data of the native structures of the HP model and assessed according to their capability in making structural predictions of unseen data. The specific network design of self-attention mechanism adopted here is modified from a similar idea in natural language recognition. The big-stride representation module introduced in this work is shown to drastically improve network's capability to model polymer segments of strong lattice position correlations.

关键词

Keywords

Deep neural networkSelf-attention mechanismBig-stride representationConditional random methods

references

Yang, L.; Tan, X.; Wang, Z.; Zhang, X . Supramolecular polymers: historical development, preparation, characterization, and functions . Chem. Rev. , 2015 . 115 7196 -7239 . DOI:10.1021/cr500633bhttp://doi.org/10.1021/cr500633b .

Zhang, X.; Wang, C . Supramolecular amphiphiles . Chem. Soc. Rev. , 2011 . 40 94 -101 . DOI:10.1039/B919678Chttp://doi.org/10.1039/B919678C .

Elshire, R. J.; Glaubitz, J. C.; Sun, Q.; Poland, J. A.; Kawamoto, K. Buckler, E. S.; Mitchel, S. E . A robust, simple genotyping-by-sequencing (GBS) approach for high diversity species . PloS One , 2011 . 6 e19379 DOI:10.1371/journal.pone.0019379http://doi.org/10.1371/journal.pone.0019379 .

Saunders, M. G.; Voth, G. A . Coarse-graining methods for computational biology . Annual Rev. Biophys. , 2013 . 42 73 -93 . DOI:10.1146/annurev-biophys-083012-130348http://doi.org/10.1146/annurev-biophys-083012-130348 .

Perilla, J. R.; Goh, B. C.; Cassidy, C. K.; Liu, B.; Bernardi, R . C.; Rudack, T.; Yu, H.; Wu, Z.; Schulten, K. Molecular dynamics simulations of large macromolecular complexes . Curr. Opin. Struct. Biol. , 2015 . 31 64 -74 . DOI:10.1016/j.sbi.2015.03.007http://doi.org/10.1016/j.sbi.2015.03.007 .

Shakhnovich, E.; Farztdinov, G.; Gutin, A.; Karplus, M . Protein folding bottlenecks: A lattice Monte Carlo simulation . Phys. Rev. Lett. , 1991 . 67 1665 DOI:10.1103/PhysRevLett.67.1665http://doi.org/10.1103/PhysRevLett.67.1665 .

Scheraga, H . A.; Khalili, M.; Liwo, A. Protein-folding dynamics: overview of molecular simulation techniques . Annu. Rev. Phys. Chem. , 2007 . 58 57 -83 . DOI:10.1146/annurev.physchem.58.032806.104614http://doi.org/10.1146/annurev.physchem.58.032806.104614 .

Carrasquilla, J.; Melko, R. G . Machine learning phases of matter . Nat. Phys. , 2017 . 13 431 -434 . DOI:10.1038/nphys4035http://doi.org/10.1038/nphys4035 .

Smith, J. S.; Isayev, O.; Roitberg, A. E . ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost . Chem. Sci. , 2017 . 8 3192 -3203 . DOI:10.1039/C6SC05720Ahttp://doi.org/10.1039/C6SC05720A .

Schütt, K. T.; Arbabzadah, F.; Chmiela, S.; Muller; K. R.; Tkatchenko, A . Quantum-chemical insights from deep tensor neural networks . Nat. Commun. , 2017 . 8 1 -8 . DOI:10.1038/s41467-016-0009-6http://doi.org/10.1038/s41467-016-0009-6 .

Wei, Q.; Melko, R. G.; Chen, J. Z. Y . Identifying polymer states by machine learning . Phys. Rev. E , 2017 . 95 032504 .

Lau, K. F.; Dill, K. A . A lattice statistical mechanics model of the conformational and sequence spaces of proteins . Macromolecules , 1989 . 22 3986 -3997 . DOI:10.1021/ma00200a030http://doi.org/10.1021/ma00200a030 .

Dill, K. A.; MacCallum, J. L . The protein-folding problem, 50 years on . Science , 2012 . 338 1042 -1046 . DOI:10.1126/science.1219021http://doi.org/10.1126/science.1219021 .

Hossain, M. S.; Salam, A . Text-to-3D Scene Generation using Semantic Parsing and Spatial Knowledge with Rule Based System . Int. J. Comp. Sci. Issues (IJCSI) , 2017 . 14 37 -41. .

LeCun, Y.; Bengio, Y.; Hinton, G . Deep learning . Nature , 2015 . 521 (7553 ):436 -444 . DOI:10.1038/nature14539http://doi.org/10.1038/nature14539 .

Goodfellow, I.; Bengio, Y.; Courville, A. Deep learning. MIT Press, 2016.

Nielsen, M. A. Neural networks and deep learning. Determination Press San Francisco, CA, USA, 2015; Vol. 25.

Shalev-Shwartz, S.; Ben-David, S. Understanding machine learning: from theory to algorithms. Cambridge University Press, 2014.

Li, J.; Zhang, H.; Chen, J. Z. Y . Structural prediction and inverse design by a strongly correlated neural network . Phys. Rev. Lett. , 2019 . 123 108002 DOI:10.1103/PhysRevLett.123.108002http://doi.org/10.1103/PhysRevLett.123.108002 .

Cheng, J.; Dong, L.; Lapata, M. Long short-term memory-networks for machine reading. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, 2016, pp. 551–561.

Parikh, A. P.; Täckström, O.; Das, D.; Uszkoreit, J. A decomposable attention model for natural language inference. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, 2016, pp. 2249–2255.

Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L.; Polosukhin, I. Attention is all you need. Adv. Neural Inf. Process. Syst., 2017.

Flory, P. J. Principles of polymer chemistry. Cornell University Press, 1953.

Landau, D.; Binder, K. A guide to Monte Carlo simulations in statistical physics. Cambridge University Press, 2021.

Allen, M. P.; Tildesley, D. J. Computer simulation of liquids. Oxford University Press, 2017.

Wang, S.; Peng, J.; Ma, J.; Xu; J. B . Protein secondary structure prediction using deep convolutional neural fields . Scient. Rep. , 2016 . 6 1 -11 . DOI:10.1038/s41598-016-0001-8http://doi.org/10.1038/s41598-016-0001-8 .

Chan, H. S.; Dill, K. A . Transition states and folding dynamics of proteins and heteropolymers . J. Chem. Phys. , 1994 . 100 9238 -9257 . DOI:10.1063/1.466677http://doi.org/10.1063/1.466677 .

Please see https://github.com/vvoelz/HPSandbox.

Wüst, T.; Landau, D. P . Versatile approach to access the low temperature thermodynamics of lattice polymers and proteins . Phys. Rev. Lett. , 2009 . 102 178101 DOI:10.1103/PhysRevLett.102.178101http://doi.org/10.1103/PhysRevLett.102.178101 .

Bošković, B.; Brest, J . Genetic algorithm with advanced mechanisms applied to the protein structure prediction in a hydrophobic-polar model and cubic lattice . Appl. Soft Comput. , 2016 . 45 61 -70 . DOI:10.1016/j.asoc.2016.04.001http://doi.org/10.1016/j.asoc.2016.04.001 .

Yang, C. H.; Wu, K. C.; Lin, Y. S.; Chuang; L. Y.; Chang H. W . Protein folding prediction in the HP model using ions motion optimization with a greedy algorithm . BioData Mining , 2018 . 11 1 -14 . DOI:10.1186/s13040-018-0176-6http://doi.org/10.1186/s13040-018-0176-6 .

Li, Y. W.; Wuest, T.; Landau, D. P . Generic folding and transition hierarchies for surface adsorption of hydrophobic-polar lattice model proteins . Phys. Rev. E , 2013 . 87 012706 DOI:10.1103/PhysRevE.87.012706http://doi.org/10.1103/PhysRevE.87.012706 .

Wu, H.; Yang, R.; Fu, Q.; Chen, J. P.; Lu, W. Z.; Li, H. O . Research on predicting 2D-HP protein folding using reinforcement learning with full state space . Bmc Bioinformatics , 2019 . 20 DOI:10.1186/s12859-019-3259-6http://doi.org/10.1186/s12859-019-3259-6 .

Please see https://github.com/Titanium-ALarx7/HP-ProteinPrediction-SCN.

He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition, Proceedings of the IEEE conference on computer vision and pattern recognition, 2016; pp. 770−778.

Hochreiter, S.; Schmidhuber, J . Long short-term memory . Neural Comput. , 1997 . 9 1735 -1780 . DOI:10.1162/neco.1997.9.8.1735http://doi.org/10.1162/neco.1997.9.8.1735 .

Devlin, J.; Chang, M. W.; Lee, K.; Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. “NAACL-HLT2019”, Minneapolis, Minnesota, 2018, pp. 4171–4186.

Lafferty, J.; McCallum, A.; Pereira, F. C. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. 2001.

Frauenkron, H.; Bastolla, U.; Gerstner, E.; Grassberger, P.; Nadler, W . New Monte Carlo algorithm for protein folding . Phys. Rev. Lett. , 1998 . 80 3149 DOI:10.1103/PhysRevLett.80.3149http://doi.org/10.1103/PhysRevLett.80.3149 .

Thachuk, C.; Shmygelska, A.; Hoos, H. H . A replica exchange Monte Carlo algorithm for protein folding in the HP model . BMC bioinformatics , 2007 . 8 1 -20 . DOI:10.1186/1471-2105-8-1http://doi.org/10.1186/1471-2105-8-1 .

Wüst, T.; Landau, D . The HP model of protein folding: A challenging testing ground for Wang-Landau sampling . Comp. Phys. Commun. , 2008 . 179 124 -127 . DOI:10.1016/j.cpc.2008.01.028http://doi.org/10.1016/j.cpc.2008.01.028 .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Designs to Improve Capability of Neural Networks to Make Structural Predictions

Related Author

No data

Related Institution

No data

⁰