Designs to Improve Capability of Neural Networks to Make Structural Predictions

Tian-Yao Wang; Jian-Feng Li; Hong-Dong Zhang; Jeff Z. Y. Chen

doi:10.1007/s10118-023-2910-x

Your Location：

Home >

Browse articles >

Designs to Improve Capability of Neural Networks to Make Structural Predictions

RESEARCH ARTICLE | Updated：2023-08-24

- Designs to Improve Capability of Neural Networks to Make Structural Predictions
- Chinese Journal of Polymer Science Vol. 41, Issue 9, Pages: 1477-1485(2023)
- Affiliations：
  
  a.Department of Macromolecular Science, State Key Laboratory of Macromolecular Engineering of Polymers, Fudan University, Shanghai 200438, China
  b.Department of Physics and Astronomy, University of Waterloo, Waterloo N2L 3G1, Canada
- Author bio：
  
  lijf@fudan.edu.cn (J.F.L.)
  jeffchen@uwaterloo.ca (J.Z.Y.C.)
- Funds：
- DOI：10.1007/s10118-023-2910-x
  CLC：
- Received：03 November 2022，
  
  Revised：2022-11-22，
  
  Accepted：24 November 2022，
  
  Online First：31 January 2023，
  
  Published：01 September 2023
- Accepted：
Scan QR Code
Wang, T. Y.; Li, J. F.; Zhang, H. D.; Chen, J. Z. Y. Designs to improve capability of neural networks to make structural predictions. Chinese J. Polym. Sci. 2023, 41, 1477–1485

Tian-Yao Wang, Jian-Feng Li, Hong-Dong Zhang, et al. Designs to Improve Capability of Neural Networks to Make Structural Predictions[J]. Chinese Journal of Polymer Science, 2023, 41(9): 1477-1485.
Wang, T. Y.; Li, J. F.; Zhang, H. D.; Chen, J. Z. Y. Designs to improve capability of neural networks to make structural predictions. Chinese J. Polym. Sci. 2023, 41, 1477–1485 DOI： 10.1007/s10118-023-2910-x.

Tian-Yao Wang, Jian-Feng Li, Hong-Dong Zhang, et al. Designs to Improve Capability of Neural Networks to Make Structural Predictions[J]. Chinese Journal of Polymer Science, 2023, 41(9): 1477-1485. DOI： 10.1007/s10118-023-2910-x.

摘要

A number of specially-designed modules have been employed to improve the performance of the neural network on structural predictions.

Abstract

A deep neural network model generally consists of different modules that play essential roles in performing a task. The optimal design of a module for use in modeling a physical problem is directly related to the success of the model. In this work

the effectiveness of a number of special modules

the self-attention mechanism for recognizing the importance of molecular sequence information in a polymer

as well as the big-stride representation and conditional random field for enhancing the network ability to produce desired local configurations

is numerically studied. Network models containing these modules are trained by using the well documented data of the native structures of the HP model and assessed according to their capability in making structural predictions of unseen data. The specific network design of self-attention mechanism adopted here is modified from a similar idea in natural language recognition. The big-stride representation module introduced in this work is shown to drastically improve network's capability to model polymer segments of strong lattice position correlations.

关键词

Keywords

references

Yang,L.;Tan,X.;Wang,Z.;Zhang,X.Supramolecularpolymers:historicaldevelopment,preparation,characterization,andfunctions. Chem. Rev. 2015 , 115 ,7196−7239..

Zhang,X.;Wang,C.Supramolecularamphiphiles. Chem. Soc. Rev. 2011 , 40 ,94−101..

Elshire,R.J.;Glaubitz,J.C.;Sun,Q.;Poland,J.A.;Kawamoto,K.Buckler,E.S.;Mitchel,S.E.Arobust,simplegenotyping-by-sequencing(GBS)approachforhighdiversityspecies. PloS One 2011 , 6 ,e19379..

Saunders,M.G.;Voth,G.A.Coarse-grainingmethodsforcomputationalbiology. Annual Rev. Biophys. 2013 , 42 ,73−93..

Perilla,J.R.;Goh,B.C.;Cassidy,C.K.;Liu,B.;Bernardi,R.C.;Rudack,T.;Yu,H.;Wu,Z.;Schulten,K.Moleculardynamicssimulationsoflargemacromolecularcomplexes. Curr. Opin. Struct. Biol. 2015 , 31 ,64−74..

Shakhnovich,E.;Farztdinov,G.;Gutin,A.;Karplus,M.Proteinfoldingbottlenecks:AlatticeMonteCarlosimulation. Phys. Rev. Lett. 1991 , 67 ,1665..

Scheraga,H.A.;Khalili,M.;Liwo,A.Protein-foldingdynamics:overviewofmolecularsimulationtechniques. Annu. Rev. Phys. Chem. 2007 , 58 ,57−83..

Carrasquilla,J.;Melko,R.G.Machinelearningphasesofmatter. Nat. Phys. 2017 , 13 ,431−434..

Smith,J.S.;Isayev,O.;Roitberg,A.E.ANI-1:anextensibleneuralnetworkpotentialwithDFTaccuracyatforcefieldcomputationalcost. Chem. Sci. 2017 , 8 ,3192−3203..

Schütt,K.T.;Arbabzadah,F.;Chmiela,S.;Muller;K.R.;Tkatchenko,A.Quantum-chemicalinsightsfromdeeptensorneuralnetworks. Nat. Commun. 2017 , 8 ,1−8..

Wei,Q.;Melko,R.G.;Chen,J.Z.Y.Identifyingpolymerstatesbymachinelearning. Phys. Rev. E 2017 , 95 ,032504..

Lau,K.F.;Dill,K.A.Alatticestatisticalmechanicsmodeloftheconformationalandsequencespacesofproteins. Macromolecules 1989 , 22 ,3986−3997..

Dill,K.A.;MacCallum,J.L.Theprotein-foldingproblem,50yearson. Science 2012 , 338 ,1042−1046..

Hossain,M.S.;Salam,A.Text-to-3DSceneGenerationusingSemanticParsingandSpatialKnowledgewithRuleBasedSystem. Int. J. Comp. Sci. Issues (IJCSI) 2017 , 14 ,37−41..

LeCun,Y.;Bengio,Y.;Hinton,G.Deeplearning. Nature 2015 , 521 (7553),436−444..

Goodfellow,I.;Bengio,Y.;Courville,A. Deep learning .MITPress, 2016 ..

Nielsen,M.A. Neural networks and deep learning .DeterminationPressSanFrancisco,CA,USA, 2015 ;Vol.25..

Shalev-Shwartz,S.;Ben-David,S. Understanding machine learning: from theory to algorithms .CambridgeUniversityPress, 2014 ..

Li,J.;Zhang,H.;Chen,J.Z.Y.Structuralpredictionandinversedesignbyastronglycorrelatedneuralnetwork. Phys. Rev. Lett. 2019 , 123 ,108002..

Cheng,J.;Dong,L.;Lapata,M.Longshort-termmemory-networksformachinereading. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing ,Austin,Texas, 2016 ,pp.551–561..

Parikh,A.P.;Täckström,O.;Das,D.;Uszkoreit,J.Adecomposableattentionmodelfornaturallanguageinference. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing ,Austin,Texas, 2016 ,pp.2249–2255..

Vaswani,A.;Shazeer,N.;Parmar,N.;Uszkoreit,J.;Jones,L.;Gomez,A.N.;Kaiser,L.;Polosukhin,I.Attentionisallyouneed. Adv. Neural Inf. Process. Syst., 2017 . .

Flory,P.J. Principles of polymer chemistry .CornellUniversityPress, 1953 ..

Landau,D.;Binder,K. A guide to Monte Carlo simulations in statistical physics .CambridgeUniversityPress, 2021 ..

Allen,M.P.;Tildesley,D.J. Computer simulation of liquids .OxfordUniversityPress, 2017 ..

Wang,S.;Peng,J.;Ma,J.;Xu;J.B.Proteinsecondarystructurepredictionusingdeepconvolutionalneuralfields. Scient. Rep. 2016 , 6 ,1−11..

Chan,H.S.;Dill,K.A.Transitionstatesandfoldingdynamicsofproteinsandheteropolymers. J. Chem. Phys. 1994 , 100 ,9238−9257..

Please see https://github.com/vvoelz/HPSandbox .

Wüst,T.;Landau,D.P.Versatileapproachtoaccessthelowtemperaturethermodynamicsoflatticepolymersandproteins. Phys. Rev. Lett. 2009 , 102 ,178101..

Bošković,B.;Brest,J.Geneticalgorithmwithadvancedmechanismsappliedtotheproteinstructurepredictioninahydrophobic-polarmodelandcubiclattice. Appl. Soft Comput. 2016 , 45 ,61−70..

Yang,C.H.;Wu,K.C.;Lin,Y.S.;Chuang;L.Y.;ChangH.W.ProteinfoldingpredictionintheHPmodelusingionsmotionoptimizationwithagreedyalgorithm. BioData Mining 2018 , 11 ,1−14..

Li,Y.W.;Wuest,T.;Landau,D.P.Genericfoldingandtransitionhierarchiesforsurfaceadsorptionofhydrophobic-polarlatticemodelproteins. Phys. Rev. E 2013 , 87 ,012706..

Wu,H.;Yang,R.;Fu,Q.;Chen,J.P.;Lu,W.Z.;Li,H.O.Researchonpredicting2D-HPproteinfoldingusingreinforcementlearningwithfullstatespace. Bmc Bioinformatics 2019 ,20..

Pleasesee https://github.com/Titanium-ALarx7/HP-ProteinPrediction-SCN .

He,K.;Zhang,X.;Ren,S.;Sun,J.Deepresiduallearningforimagerecognition, Proceedings of the IEEE conference on computer vision and pattern recognition , 2016 ;pp.770−778..

Hochreiter,S.;Schmidhuber,J.Longshort-termmemory. Neural Comput. 1997 , 9 ,1735−1780..

Devlin,J.;Chang,M.W.;Lee,K.;Toutanova,K.Bert:Pre-trainingofdeepbidirectionaltransformersforlanguageunderstanding. “NAACL-HLT2019” ,Minneapolis,Minnesota, 2018 ,pp.4171–4186..

Lafferty,J.;McCallum,A.;Pereira,F.C.Conditionalrandomfields:Probabilisticmodelsforsegmentingandlabelingsequencedata. 2001 .

Frauenkron,H.;Bastolla,U.;Gerstner,E.;Grassberger,P.;Nadler,W.NewMonteCarloalgorithmforproteinfolding. Phys. Rev. Lett. 1998 , 80 ,3149..

Thachuk,C.;Shmygelska,A.;Hoos,H.H.AreplicaexchangeMonteCarloalgorithmforproteinfoldingintheHPmodel. BMC bioinformatics 2007 , 8 ,1−20..

Wüst,T.;Landau,D.TheHPmodelofproteinfolding:AchallengingtestinggroundforWang-Landausampling. Comp. Phys. Commun. 2008 , 179 ,124−127..

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰