Fixes and improvements related to random number generation. Improvements for data set filtering.#887

drroe · 2021-03-15T15:13:32Z

This PR contains a lot of improvements.

Random number generation is greatly improved under the hood, and after some more testing the default random number generator should be changed (lead candidate right now is Xoshiro128++). Added a gauss keyword to the random command.
The algorithm for the permutedihedrals keyword with the random command is greatly improved. In general, fewer rotations are required to get much higher quality structures (with no clashes so far in testing proteins up to 268 a.a.!).
The algorithm for randomizeions has been sped up an order of magnitude. In addition, users are not required to specify an around mask, and in conjunction with allowoverlaps eliminates all distance checking, allowing for pure random ion placement.
The code for datafilter and the filter action has been centralized under the hood. In addition, the final counts of passed/filtered frames are now stored in data sets, and 2d/3d sets can be counted (but not yet actively filtered).

My hope is that this is the version (5.1) that ends up in AmberTools 21.

…0 and 1

…est.

change results.

…e). Make max_factor_ a double, and make the default 10.

max rotations be on a per-dihedral basis. Make rotation counts unsigned in old routine. Fix printf formatting.

!multi

testing.

…ndomizeions improvements, and permutedihedrals improvements.

drroe · 2021-03-15T15:14:56Z

FML

E: The repository 'https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_20.04  Release' no longer has a Release file.

I just want stable CI....

swails · 2021-03-15T15:17:29Z

What package is it trying to install?

swails · 2021-03-15T15:18:38Z

.github/workflows/merge-gate.yml

 jobs:
  linux_build:
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest


Same image ATM.

AmberJenkins · 2021-03-15T15:19:57Z

The PGI build in Jenkins failed.

drroe · 2021-03-15T15:24:04Z

@swails its failing on the apt update command, which I had to add previously because I was getting "Unable to fetch some archives" messages without it (see #885).

command instance

drroe · 2021-03-15T16:49:20Z

@hainm note the pytraj failure is expected; the algorithm for the randomizeions command has been improved, but now produces different output. The originalalgorithm keyword can be used to recover original behavior, or the test output will need to be updated. I can do that after I merge this.

drroe · 2021-03-15T16:52:00Z

@swails taking out apt update seems to have fixed it this time. I guess actions is still smoothing out the rough edges.

cpptraj PR 887: Amber-MD/cpptraj#887

drroe added 30 commits February 27, 2021 07:01

Avoid modulo bias by using bit shifting

76c20d9

Fix Makefile; source changed to cpp

7f143fe

Change function name, results can include max

4f4d820

Override Number_UpTo for Marsaglia since it natively returns between …

c82d5db

…0 and 1

Use Number_UpTo() for interval functions

d6e5596

Use rn_num_interval instead of rn_gen

028dc7d

Set RNG to Marsaglia to recover earlier test results.

dcce87c

Try generating numbers between 0 and 1 using method of Doornik

73b810c

Use the shuffle from Random_Number. Set Marsaglia for Kmeans random t…

cf122ce

…est.

Use rn_num_interval for cluster sieve. Set Marsaglia gen for test.

1ce14c4

Try to avoid range and modulo bias

70b7cd8

Change number of bits reported to dieharder based on RNG. Does not

67429ab

change results.

Merge branch 'master' into update.rng

9b400fd

Merge remote-tracking branch 'origin/update.rng' into update.rng

fd1113a

Merge branch 'master' into update.rng

1749ba7

Merge branch 'master' into update.rng

d7aa09b

use rng_gen() times two PI instead of integer values for angle

b44bdac

Try a simpler random algorithm.

6d5a880

Use new simpler algorithm unless back-tracking is specified

5b40809

Allow other names besides RunTest.sh (more flexible)

3fcdbd5

Add test for new method, update test for old method

421f203

Increase the residue cutoff for the new method.

a92f8f1

Add code docs

cbfd744

Only set up CheckResidue() stuff if needed (not needed for new routin…

ea94c75

…e). Make max_factor_ a double, and make the default 10.

Get rid of duplicate check

ac606e5

Only need to convert to degrees for debug

79ef368

Change max factor default back to 2. In new random rotate routine, have

5a894b3

max rotations be on a per-dihedral basis. Make rotation counts unsigned in old routine. Fix printf formatting.

Hide some debug info. Fix cleanup

a9eab21

Set default RNG to Marsaglia for tests

84cfc84

Set marsaglia for test

0e0012b

drroe added 13 commits March 15, 2021 10:00

Clean up keyword help functions

5d61615

Fix up help

11f9ab3

Add count sets and countout keyword

b501cea

Add 2d matrix filter and count test.

7100537

Hide debug info.

8f1c134

Move final printout to individual functions; ensure it only happens when

1872c0b

!multi

Fix up filter action and datafilter command entries

0c4efce

Add link to Emmett Leddin's cpptraj vim syntax file

341b1d8

DRR - Go back to Marsaglia for now. New generators still need more

64b25f9

testing.

Minor version bump for RNG improvements, data filter improvements, ra…

44e0352

…ndomizeions improvements, and permutedihedrals improvements.

Fix up and improve randomizeions manual entry.

162402e

Fix up help for permutedihedrals

28e07df

maxfactor option was listed twice

0a3f85b

drroe added the enhancement label Mar 15, 2021

drroe self-assigned this Mar 15, 2021

Hail mary to fix github actions...

fe648bc

Merge branch 'master' into update.rng

42f5895

swails reviewed Mar 15, 2021

View reviewed changes

.github/workflows/merge-gate.yml

jobs:

linux_build:

runs-on: ubuntu-20.04

runs-on: ubuntu-latest

Copy link

Contributor

swails Mar 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same image ATM.

drroe added 2 commits March 15, 2021 12:02

Try to fix GitHub Actions by putting update and install in the same

eb90e38

command instance

Well that was a bust. Try removing the update command...

bd8b3fd

drroe added a commit to drroe/pytraj that referenced this pull request Mar 15, 2021

Change test input/target save to match changes to cpptraj made in

13d229c

cpptraj PR 887: Amber-MD/cpptraj#887

drroe merged commit 9933f8c into Amber-MD:master Mar 15, 2021

drroe deleted the update.rng branch March 15, 2021 18:48

drroe mentioned this pull request Mar 15, 2021

Fix randomizeions test Amber-MD/pytraj#1577

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixes and improvements related to random number generation. Improvements for data set filtering.#887

Fixes and improvements related to random number generation. Improvements for data set filtering.#887
drroe merged 81 commits intoAmber-MD:masterfrom
drroe:update.rng

drroe commented Mar 15, 2021

Uh oh!

drroe commented Mar 15, 2021

Uh oh!

swails commented Mar 15, 2021

Uh oh!

swails Mar 15, 2021

Uh oh!

AmberJenkins commented Mar 15, 2021

Uh oh!

drroe commented Mar 15, 2021

Uh oh!

drroe commented Mar 15, 2021

Uh oh!

drroe commented Mar 15, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Comments

Conversation

drroe commented Mar 15, 2021

Uh oh!

drroe commented Mar 15, 2021

Uh oh!

swails commented Mar 15, 2021

Uh oh!

swails Mar 15, 2021

Choose a reason for hiding this comment

Uh oh!

AmberJenkins commented Mar 15, 2021

Uh oh!

drroe commented Mar 15, 2021

Uh oh!

drroe commented Mar 15, 2021

Uh oh!

drroe commented Mar 15, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Comments