Designs to Improve Capability of Neural Networks to Make Structural Predictions

Tian-Yao Wang; Jian-Feng Li; Hong-Dong Zhang; Jeff Z. Y. Chen

doi:10.1007/s10118-023-2910-x

Your Location：

Home >

Browse articles >

Designs to Improve Capability of Neural Networks to Make Structural Predictions

RESEARCH ARTICLE | Updated：2023-08-24

- Designs to Improve Capability of Neural Networks to Make Structural Predictions
- Designs to Improve Capability of Neural Networks to Make Structural Predictions
- 高分子科学（英文版） 2023年41卷第9期页码：1477-1485
- Affiliations：
  
  a.Department of Macromolecular Science, State Key Laboratory of Macromolecular Engineering of Polymers, Fudan University, Shanghai 200438, China
  b.Department of Physics and Astronomy, University of Waterloo, Waterloo N2L 3G1, Canada
- Author bio：
  
  lijf@fudan.edu.cn (J.F.L.)
  jeffchen@uwaterloo.ca (J.Z.Y.C.)
- Funds：
  
  This work was financially supported by the National Natural Science Foundation of China (Nos. 21973018 and 21534002) and the Natural Sciences and Engineering Research Council (NSERC) of Canada.;Electronic supplementary information (ESI) is available free of charge in the online version of this article at http://doi.org/10.1007/s10118-023-2910-x.
- DOI：10.1007/s10118-023-2910-x
  中图分类号：
- 收稿：2022-11-03，
  
  修回：2022-11-22，
  
  录用：2022-11-24，
  
  网络首发：2023-01-31，
  
  纸质出版：2023-09-01
- Accepted：
Scan QR Code
Wang, T. Y.; Li, J. F.; Zhang, H. D.; Chen, J. Z. Y. Designs to improve capability of neural networks to make structural predictions. Chinese J. Polym. Sci. 2023, 41, 1477–1485

Tian-Yao Wang, Jian-Feng Li, Hong-Dong Zhang, et al. Designs to Improve Capability of Neural Networks to Make Structural Predictions[J]. Chinese Journal of Polymer Science, 2023, 41(9): 1477-1485.
Wang, T. Y.; Li, J. F.; Zhang, H. D.; Chen, J. Z. Y. Designs to improve capability of neural networks to make structural predictions. Chinese J. Polym. Sci. 2023, 41, 1477–1485 DOI： 10.1007/s10118-023-2910-x.

Tian-Yao Wang, Jian-Feng Li, Hong-Dong Zhang, et al. Designs to Improve Capability of Neural Networks to Make Structural Predictions[J]. Chinese Journal of Polymer Science, 2023, 41(9): 1477-1485. DOI： 10.1007/s10118-023-2910-x.

摘要

A number of specially-designed modules have been employed to improve the performance of the neural network on structural predictions.

Abstract

A deep neural network model generally consists of different modules that play essential roles in performing a task. The optimal design of a module for use in modeling a physical problem is directly related to the success of the model. In this work

the effectiveness of a number of special modules

the self-attention mechanism for recognizing the importance of molecular sequence information in a polymer

as well as the big-stride representation and conditional random field for enhancing the network ability to produce desired local configurations

is numerically studied. Network models containing these modules are trained by using the well documented data of the native structures of the HP model and assessed according to their capability in making structural predictions of unseen data. The specific network design of self-attention mechanism adopted here is modified from a similar idea in natural language recognition. The big-stride representation module introduced in this work is shown to drastically improve network's capability to model polymer segments of strong lattice position correlations.

关键词

Keywords

references

Yang,L.;Tan,X.;Wang,Z.;Zhang,X.Supramolecularpolymers:historicaldevelopment,preparation,characterization,andfunctions. Chem. Rev. 2015 , 115 ,7196−7239..

Zhang,X.;Wang,C.Supramolecularamphiphiles. Chem. Soc. Rev. 2011 , 40 ,94−101..

Elshire,R.J.;Glaubitz,J.C.;Sun,Q.;Poland,J.A.;Kawamoto,K.Buckler,E.S.;Mitchel,S.E.Arobust,simplegenotyping-by-sequencing(GBS)approachforhighdiversityspecies. PloS One 2011 , 6 ,e19379..

Saunders,M.G.;Voth,G.A.Coarse-grainingmethodsforcomputationalbiology. Annual Rev. Biophys. 2013 , 42 ,73−93..

Perilla,J.R.;Goh,B.C.;Cassidy,C.K.;Liu,B.;Bernardi,R.C.;Rudack,T.;Yu,H.;Wu,Z.;Schulten,K.Moleculardynamicssimulationsoflargemacromolecularcomplexes. Curr. Opin. Struct. Biol. 2015 , 31 ,64−74..

Shakhnovich,E.;Farztdinov,G.;Gutin,A.;Karplus,M.Proteinfoldingbottlenecks:AlatticeMonteCarlosimulation. Phys. Rev. Lett. 1991 , 67 ,1665..

Scheraga,H.A.;Khalili,M.;Liwo,A.Protein-foldingdynamics:overviewofmolecularsimulationtechniques. Annu. Rev. Phys. Chem. 2007 , 58 ,57−83..

Carrasquilla,J.;Melko,R.G.Machinelearningphasesofmatter. Nat. Phys. 2017 , 13 ,431−434..

Smith,J.S.;Isayev,O.;Roitberg,A.E.ANI-1:anextensibleneuralnetworkpotentialwithDFTaccuracyatforcefieldcomputationalcost. Chem. Sci. 2017 , 8 ,3192−3203..

Schütt,K.T.;Arbabzadah,F.;Chmiela,S.;Muller;K.R.;Tkatchenko,A.Quantum-chemicalinsightsfromdeeptensorneuralnetworks. Nat. Commun. 2017 , 8 ,1−8..

Wei,Q.;Melko,R.G.;Chen,J.Z.Y.Identifyingpolymerstatesbymachinelearning. Phys. Rev. E 2017 , 95 ,032504..

Lau,K.F.;Dill,K.A.Alatticestatisticalmechanicsmodeloftheconformationalandsequencespacesofproteins. Macromolecules 1989 , 22 ,3986−3997..

Dill,K.A.;MacCallum,J.L.Theprotein-foldingproblem,50yearson. Science 2012 , 338 ,1042−1046..

Hossain,M.S.;Salam,A.Text-to-3DSceneGenerationusingSemanticParsingandSpatialKnowledgewithRuleBasedSystem. Int. J. Comp. Sci. Issues (IJCSI) 2017 , 14 ,37−41..

LeCun,Y.;Bengio,Y.;Hinton,G.Deeplearning. Nature 2015 , 521 (7553),436−444..

Goodfellow,I.;Bengio,Y.;Courville,A. Deep learning .MITPress, 2016 ..

Nielsen,M.A. Neural networks and deep learning .DeterminationPressSanFrancisco,CA,USA, 2015 ;Vol.25..

Shalev-Shwartz,S.;Ben-David,S. Understanding machine learning: from theory to algorithms .CambridgeUniversityPress, 2014 ..

Li,J.;Zhang,H.;Chen,J.Z.Y.Structuralpredictionandinversedesignbyastronglycorrelatedneuralnetwork. Phys. Rev. Lett. 2019 , 123 ,108002..

Cheng,J.;Dong,L.;Lapata,M.Longshort-termmemory-networksformachinereading. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing ,Austin,Texas, 2016 ,pp.551–561..

Parikh,A.P.;Täckström,O.;Das,D.;Uszkoreit,J.Adecomposableattentionmodelfornaturallanguageinference. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing ,Austin,Texas, 2016 ,pp.2249–2255..

Vaswani,A.;Shazeer,N.;Parmar,N.;Uszkoreit,J.;Jones,L.;Gomez,A.N.;Kaiser,L.;Polosukhin,I.Attentionisallyouneed. Adv. Neural Inf. Process. Syst., 2017 . .

Flory,P.J. Principles of polymer chemistry .CornellUniversityPress, 1953 ..

Landau,D.;Binder,K. A guide to Monte Carlo simulations in statistical physics .CambridgeUniversityPress, 2021 ..

Allen,M.P.;Tildesley,D.J. Computer simulation of liquids .OxfordUniversityPress, 2017 ..

Wang,S.;Peng,J.;Ma,J.;Xu;J.B.Proteinsecondarystructurepredictionusingdeepconvolutionalneuralfields. Scient. Rep. 2016 , 6 ,1−11..

Chan,H.S.;Dill,K.A.Transitionstatesandfoldingdynamicsofproteinsandheteropolymers. J. Chem. Phys. 1994 , 100 ,9238−9257..

Please see https://github.com/vvoelz/HPSandbox .

Wüst,T.;Landau,D.P.Versatileapproachtoaccessthelowtemperaturethermodynamicsoflatticepolymersandproteins. Phys. Rev. Lett. 2009 , 102 ,178101..

Bošković,B.;Brest,J.Geneticalgorithmwithadvancedmechanismsappliedtotheproteinstructurepredictioninahydrophobic-polarmodelandcubiclattice. Appl. Soft Comput. 2016 , 45 ,61−70..

Yang,C.H.;Wu,K.C.;Lin,Y.S.;Chuang;L.Y.;ChangH.W.ProteinfoldingpredictionintheHPmodelusingionsmotionoptimizationwithagreedyalgorithm. BioData Mining 2018 , 11 ,1−14..

Li,Y.W.;Wuest,T.;Landau,D.P.Genericfoldingandtransitionhierarchiesforsurfaceadsorptionofhydrophobic-polarlatticemodelproteins. Phys. Rev. E 2013 , 87 ,012706..

Wu,H.;Yang,R.;Fu,Q.;Chen,J.P.;Lu,W.Z.;Li,H.O.Researchonpredicting2D-HPproteinfoldingusingreinforcementlearningwithfullstatespace. Bmc Bioinformatics 2019 ,20..

Pleasesee https://github.com/Titanium-ALarx7/HP-ProteinPrediction-SCN .

He,K.;Zhang,X.;Ren,S.;Sun,J.Deepresiduallearningforimagerecognition, Proceedings of the IEEE conference on computer vision and pattern recognition , 2016 ;pp.770−778..

Hochreiter,S.;Schmidhuber,J.Longshort-termmemory. Neural Comput. 1997 , 9 ,1735−1780..

Devlin,J.;Chang,M.W.;Lee,K.;Toutanova,K.Bert:Pre-trainingofdeepbidirectionaltransformersforlanguageunderstanding. “NAACL-HLT2019” ,Minneapolis,Minnesota, 2018 ,pp.4171–4186..

Lafferty,J.;McCallum,A.;Pereira,F.C.Conditionalrandomfields:Probabilisticmodelsforsegmentingandlabelingsequencedata. 2001 .

Frauenkron,H.;Bastolla,U.;Gerstner,E.;Grassberger,P.;Nadler,W.NewMonteCarloalgorithmforproteinfolding. Phys. Rev. Lett. 1998 , 80 ,3149..

Thachuk,C.;Shmygelska,A.;Hoos,H.H.AreplicaexchangeMonteCarloalgorithmforproteinfoldingintheHPmodel. BMC bioinformatics 2007 , 8 ,1−20..

Wüst,T.;Landau,D.TheHPmodelofproteinfolding:AchallengingtestinggroundforWang-Landausampling. Comp. Phys. Commun. 2008 , 179 ,124−127..

浏览量

163

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

Topology-regulated Co-assembly of Single Chain Polymer Nanoparticles and Block Copolymers into Hierarchical Micelle@Vesicle Composite Structures

A Paradigm for BF3 Co-initiated Cationic Polymerization with Low Dispersity: Introducing Acetonitrile and Lowering the Reaction Temperature

Upgrading Commercially Available Polyethylene to High-value Materials with Strong Mechanical Performance and Hydrophilicity by Blending with Ethylene/meta-Methoxystyrene Copolymers

Porous Organic Polymers for Photocatalytic Hydrogen Peroxide Production: Recent Advances and Future Perspectives

End-to-End Neuron-inspired Signal Transduction Using Organic Electrochemical Transistors