build based on 355ebbc

dev/.documenter-siteinfo.json

@@ -1 +1 @@
-{"documenter":{"julia_version":"1.11.0","generation_timestamp":"2024-10-14T13:07:13","documenter_version":"1.7.0"}}
+{"documenter":{"julia_version":"1.11.1","generation_timestamp":"2024-10-17T13:20:09","documenter_version":"1.7.0"}}

dev/differentiable/index.html

@@ -297,4 +297,4 @@
 Epoch 17  |  Dist end  1.041  |  Loss  0.041
 Epoch 18  |  Dist end  1.030  |  Loss  0.030
 Epoch 19  |  Dist end  1.017  |  Loss  0.017
-Epoch 20  |  Dist end  1.003  |  Loss  0.003</code></pre><p>After training it looks much better: <img src="../images/logo_after.gif" alt="Logo after"/> You could replace the simple network here with a much more complicated model and it would theoretically be able to train, even if it might prove practically difficult (see discussion below).</p><h2 id="Biomolecular-force-fields"><a class="docs-heading-anchor" href="#Biomolecular-force-fields">Biomolecular force fields</a><a id="Biomolecular-force-fields-1"></a><a class="docs-heading-anchor-permalink" href="#Biomolecular-force-fields" title="Permalink"></a></h2><p>Molly was used to train the <a href="https://doi.org/10.1039/D3SC05230C">GB99dms force field</a> for implicit solvent molecular dynamics of proteins. This involved doing differentiable simulations of one million steps with a loss function based on the residue-residue distance match to explicit solvent simulations. The <a href="https://github.com/greener-group/GB99dms">code is available</a>.</p><h2 id="Molecular-loss-functions"><a class="docs-heading-anchor" href="#Molecular-loss-functions">Molecular loss functions</a><a id="Molecular-loss-functions-1"></a><a class="docs-heading-anchor-permalink" href="#Molecular-loss-functions" title="Permalink"></a></h2><p>Ultimately, you need some objective function in order to calculate the gradient for each parameter. Here are some ideas for loss functions suitable for differentiable molecular simulations:</p><ul><li>The distance between atoms at the end of the simulation compared to some reference state. This loss is used in the examples given here, is physically reasonable, and has obvious bounds.</li><li>The distance between atoms throughout the simulation.</li><li>The radial distribution function of atoms.</li><li>RMSD between atoms and a reference state - this would be suitable for macromolecules.</li><li>dRMSD, the distance between a distance map and a reference distance map.</li><li>The radius of gyration of a molecule.</li><li>The flexibility of a set of atoms over the simulation.</li><li>Supramolecular geometry, for example assembly of molecules into straight fibres.</li><li>The correlation of different velocities over the simulation.</li><li>The energy of the system.</li><li>The temperature of the system.</li><li>Some measure of phase change or a critical point.</li><li>A combination of the above, for example to obtain a force field relevant to both ordered and disordered proteins.</li></ul><p>Some of these are currently not possible in Molly as the loggers are ignored for gradient purposes, but this will hopefully change in future.</p><h2 id="Tips-and-tricks"><a class="docs-heading-anchor" href="#Tips-and-tricks">Tips and tricks</a><a id="Tips-and-tricks-1"></a><a class="docs-heading-anchor-permalink" href="#Tips-and-tricks" title="Permalink"></a></h2><ul><li>The magnitude of gradients may be less important than the sign. Consider sampling gradients across different sources of stochasticity, such as starting velocities and conformations.</li><li>Exploding gradients prove a problem when using the velocity Verlet integrator in the NVE ensemble. This is why the velocity rescaling and Berendsen thermostats were used in the above examples. Langevin dynamics also seems to work. It is likely that the development of suitable simulation strategies and thermostats will be necessary to unlock the potential of differentiable simulation.</li><li>Do you <em>really</em> need a neural network to describe your potential? Think about learning a smaller number of physically-meaningful parameters before you put in a large neural network and expect it to learn. Whilst it is true that neural networks are universal function approximators, it does not follow that you will be able to train one by differentiating through  a long simulation. A 1,000-step simulation with a 10-layer network at each step is analogous to training a 10,000 layer network (with shared weights).</li><li>Forward mode AD holds much promise for differentiable simulation, provided that the number of parameters is small, because the memory requirement is constant in the number of simulation steps. However, if the code runs slower than non-differentiable alternatives then the best approach is likely to use finite differencing with the simulation as a black box. Adjoint sensitivity is another approach to getting gradients which is not yet available in Molly.jl.</li></ul></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../documentation/">« Documentation</a><a class="docs-footer-nextpage" href="../examples/">Examples »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="auto">Automatic (OS)</option><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option><option value="catppuccin-latte">catppuccin-latte</option><option value="catppuccin-frappe">catppuccin-frappe</option><option value="catppuccin-macchiato">catppuccin-macchiato</option><option value="catppuccin-mocha">catppuccin-mocha</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 1.7.0 on <span class="colophon-date" title="Monday 14 October 2024 13:07">Monday 14 October 2024</span>. Using Julia version 1.11.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+Epoch 20  |  Dist end  1.003  |  Loss  0.003</code></pre><p>After training it looks much better: <img src="../images/logo_after.gif" alt="Logo after"/> You could replace the simple network here with a much more complicated model and it would theoretically be able to train, even if it might prove practically difficult (see discussion below).</p><h2 id="Biomolecular-force-fields"><a class="docs-heading-anchor" href="#Biomolecular-force-fields">Biomolecular force fields</a><a id="Biomolecular-force-fields-1"></a><a class="docs-heading-anchor-permalink" href="#Biomolecular-force-fields" title="Permalink"></a></h2><p>Molly was used to train the <a href="https://doi.org/10.1039/D3SC05230C">GB99dms force field</a> for implicit solvent molecular dynamics of proteins. This involved doing differentiable simulations of one million steps with a loss function based on the residue-residue distance match to explicit solvent simulations. The <a href="https://github.com/greener-group/GB99dms">code is available</a>.</p><h2 id="Molecular-loss-functions"><a class="docs-heading-anchor" href="#Molecular-loss-functions">Molecular loss functions</a><a id="Molecular-loss-functions-1"></a><a class="docs-heading-anchor-permalink" href="#Molecular-loss-functions" title="Permalink"></a></h2><p>Ultimately, you need some objective function in order to calculate the gradient for each parameter. Here are some ideas for loss functions suitable for differentiable molecular simulations:</p><ul><li>The distance between atoms at the end of the simulation compared to some reference state. This loss is used in the examples given here, is physically reasonable, and has obvious bounds.</li><li>The distance between atoms throughout the simulation.</li><li>The radial distribution function of atoms.</li><li>RMSD between atoms and a reference state - this would be suitable for macromolecules.</li><li>dRMSD, the distance between a distance map and a reference distance map.</li><li>The radius of gyration of a molecule.</li><li>The flexibility of a set of atoms over the simulation.</li><li>Supramolecular geometry, for example assembly of molecules into straight fibres.</li><li>The correlation of different velocities over the simulation.</li><li>The energy of the system.</li><li>The temperature of the system.</li><li>Some measure of phase change or a critical point.</li><li>A combination of the above, for example to obtain a force field relevant to both ordered and disordered proteins.</li></ul><p>Some of these are currently not possible in Molly as the loggers are ignored for gradient purposes, but this will hopefully change in future.</p><h2 id="Tips-and-tricks"><a class="docs-heading-anchor" href="#Tips-and-tricks">Tips and tricks</a><a id="Tips-and-tricks-1"></a><a class="docs-heading-anchor-permalink" href="#Tips-and-tricks" title="Permalink"></a></h2><ul><li>The magnitude of gradients may be less important than the sign. Consider sampling gradients across different sources of stochasticity, such as starting velocities and conformations.</li><li>Exploding gradients prove a problem when using the velocity Verlet integrator in the NVE ensemble. This is why the velocity rescaling and Berendsen thermostats were used in the above examples. Langevin dynamics also seems to work. It is likely that the development of suitable simulation strategies and thermostats will be necessary to unlock the potential of differentiable simulation.</li><li>Do you <em>really</em> need a neural network to describe your potential? Think about learning a smaller number of physically-meaningful parameters before you put in a large neural network and expect it to learn. Whilst it is true that neural networks are universal function approximators, it does not follow that you will be able to train one by differentiating through  a long simulation. A 1,000-step simulation with a 10-layer network at each step is analogous to training a 10,000 layer network (with shared weights).</li><li>Forward mode AD holds much promise for differentiable simulation, provided that the number of parameters is small, because the memory requirement is constant in the number of simulation steps. However, if the code runs slower than non-differentiable alternatives then the best approach is likely to use finite differencing with the simulation as a black box. Adjoint sensitivity is another approach to getting gradients which is not yet available in Molly.jl.</li></ul></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../documentation/">« Documentation</a><a class="docs-footer-nextpage" href="../examples/">Examples »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="auto">Automatic (OS)</option><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option><option value="catppuccin-latte">catppuccin-latte</option><option value="catppuccin-frappe">catppuccin-frappe</option><option value="catppuccin-macchiato">catppuccin-macchiato</option><option value="catppuccin-mocha">catppuccin-mocha</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 1.7.0 on <span class="colophon-date" title="Thursday 17 October 2024 13:20">Thursday 17 October 2024</span>. Using Julia version 1.11.1.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>