Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmdstanpy yields an error when sampling is completed "CmdStan error: terminated by signal 6 Unknown error -6" but from_csv reconstructs the fit #754

Open
Garren-H opened this issue May 22, 2024 · 9 comments
Open

cmdstanpy yields an error when sampling is completed "CmdStan error: terminated by signal 6 Unknown error -6" but from_csv reconstructs the fit #754

Garren-H opened this issue May 22, 2024 · 9 comments

Comments

@Garren-H
Copy link

Description:

Hi all, I have ran into another issue (on a different cluster) when running my stan code. The sampling executes as expected but afterwards an error is displayed

The stan .txt output file yields: 
method = sample (Default)
  sample
    num_samples = 1000 (Default)
    num_warmup = 5000
    save_warmup = 0 (Default)
    thin = 1 (Default)
    adapt
      engaged = 1 (Default)
      gamma = 0.050000 (Default)
      delta = 0.800000 (Default)
      kappa = 0.750000 (Default)
      t0 = 10.000000 (Default)
      init_buffer = 75 (Default)
      term_buffer = 50 (Default)
      window = 25 (Default)
      save_metric = 0 (Default)
    algorithm = hmc (Default)
      hmc
        engine = nuts (Default)
          nuts
            max_depth = 5
        metric = dense_e
        metric_file =  (Default)
        stepsize = 1.000000 (Default)
        stepsize_jitter = 0.000000 (Default)
    num_chains = 8
id = 1 (Default)
data
  file = Subsets/Alkane_Primary alcohol/Include_clusters_False/Variance_known_True/Variance_MC_known_True/rank_2/data.json
init = 2 (Default)
random
  seed = 24898
output
  file = /home/22796002/Pure RK PMF/Subsets/Alkane_Primary alcohol/Include_clusters_False/Variance_known_True/Variance_MC_known_True/rank_2/Step1/Pure_PMF-20240522025118.csv
  diagnostic_file =  (Default)
  refresh = 100 (Default)
  sig_figs = 18
  profile_file = /home/22796002/Pure RK PMF/Subsets/Alkane_Primary alcohol/Include_clusters_False/Variance_known_True/Variance_MC_known_True/rank_2/Step1/Pure_PMF-20240522025118-profile.csv
  save_cmdstan_config = 0 (Default)
num_threads = 32 (Default)


Gradient evaluation took 0.021501 seconds
1000 transitions using 10 leapfrog steps per transition would take 215.01 seconds.
Adjust your expectations accordingly!



Gradient evaluation took 0.017354 seconds
1000 transitions using 10 leapfrog steps per transition would take 173.54 seconds.
Adjust your expectations accordingly!



Gradient evaluation took 0.015889 seconds
1000 transitions using 10 leapfrog steps per transition would take 158.89 seconds.
Adjust your expectations accordingly!



Gradient evaluation took 0.016126 seconds
1000 transitions using 10 leapfrog steps per transition would take 161.26 seconds.
Adjust your expectations accordingly!



Gradient evaluation took 0.027377 seconds
1000 transitions using 10 leapfrog steps per transition would take 273.77 seconds.
Adjust your expectations accordingly!



Gradient evaluation took 0.015305 seconds
1000 transitions using 10 leapfrog steps per transition would take 153.05 seconds.
Adjust your expectations accordingly!



Gradient evaluation took 0.014925 seconds
1000 transitions using 10 leapfrog steps per transition would take 149.25 seconds.
Adjust your expectations accordingly!



Gradient evaluation took 0.013077 seconds
1000 transitions using 10 leapfrog steps per transition would take 130.77 seconds.
Adjust your expectations accordingly!


Chain [1] Iteration:    1 / 6000 [  0%]  (Warmup)
Chain [3] Iteration:    1 / 6000 [  0%]  (Warmup)
Chain [4] Iteration:    1 / 6000 [  0%]  (Warmup)
Chain [2] Iteration:    1 / 6000 [  0%]  (Warmup)
Chain [5] Iteration:    1 / 6000 [  0%]  (Warmup)
Chain [6] Iteration:    1 / 6000 [  0%]  (Warmup)
Chain [8] Iteration:    1 / 6000 [  0%]  (Warmup)
Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Exception: Exception: multi_normal_lpdf: Location parameter[1] is -nan, but must be finite! (in 'Pure_PMF.stan', line 62, column 20 to column 112) (in 'Pure_PMF.stan', line 192, column 12 to line 195, column 39)
If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

Chain [7] Iteration:    1 / 6000 [  0%]  (Warmup)
Chain [4] Iteration:  100 / 6000 [  1%]  (Warmup)
Chain [7] Iteration:  100 / 6000 [  1%]  (Warmup)
Chain [1] Iteration:  100 / 6000 [  1%]  (Warmup)
Chain [3] Iteration:  100 / 6000 [  1%]  (Warmup)
Chain [5] Iteration:  100 / 6000 [  1%]  (Warmup)
Chain [6] Iteration:  100 / 6000 [  1%]  (Warmup)
Chain [8] Iteration:  100 / 6000 [  1%]  (Warmup)
Chain [2] Iteration:  100 / 6000 [  1%]  (Warmup)
Chain [5] Iteration:  200 / 6000 [  3%]  (Warmup)
Chain [4] Iteration:  200 / 6000 [  3%]  (Warmup)
Chain [6] Iteration:  200 / 6000 [  3%]  (Warmup)
Chain [8] Iteration:  200 / 6000 [  3%]  (Warmup)
Chain [7] Iteration:  200 / 6000 [  3%]  (Warmup)
Chain [2] Iteration:  200 / 6000 [  3%]  (Warmup)
Chain [1] Iteration:  200 / 6000 [  3%]  (Warmup)
Chain [3] Iteration:  200 / 6000 [  3%]  (Warmup)
Chain [5] Iteration:  300 / 6000 [  5%]  (Warmup)
Chain [4] Iteration:  300 / 6000 [  5%]  (Warmup)
Chain [6] Iteration:  300 / 6000 [  5%]  (Warmup)
Chain [2] Iteration:  300 / 6000 [  5%]  (Warmup)
Chain [8] Iteration:  300 / 6000 [  5%]  (Warmup)
Chain [3] Iteration:  300 / 6000 [  5%]  (Warmup)
Chain [7] Iteration:  300 / 6000 [  5%]  (Warmup)
Chain [1] Iteration:  300 / 6000 [  5%]  (Warmup)
Chain [5] Iteration:  400 / 6000 [  6%]  (Warmup)
Chain [4] Iteration:  400 / 6000 [  6%]  (Warmup)
Chain [6] Iteration:  400 / 6000 [  6%]  (Warmup)
Chain [3] Iteration:  400 / 6000 [  6%]  (Warmup)
Chain [2] Iteration:  400 / 6000 [  6%]  (Warmup)
Chain [8] Iteration:  400 / 6000 [  6%]  (Warmup)
Chain [1] Iteration:  400 / 6000 [  6%]  (Warmup)
Chain [7] Iteration:  400 / 6000 [  6%]  (Warmup)
Chain [5] Iteration:  500 / 6000 [  8%]  (Warmup)
Chain [4] Iteration:  500 / 6000 [  8%]  (Warmup)
Chain [2] Iteration:  500 / 6000 [  8%]  (Warmup)
Chain [6] Iteration:  500 / 6000 [  8%]  (Warmup)
Chain [8] Iteration:  500 / 6000 [  8%]  (Warmup)
Chain [1] Iteration:  500 / 6000 [  8%]  (Warmup)
Chain [3] Iteration:  500 / 6000 [  8%]  (Warmup)
Chain [7] Iteration:  500 / 6000 [  8%]  (Warmup)
Chain [5] Iteration:  600 / 6000 [ 10%]  (Warmup)
Chain [4] Iteration:  600 / 6000 [ 10%]  (Warmup)
Chain [2] Iteration:  600 / 6000 [ 10%]  (Warmup)
Chain [6] Iteration:  600 / 6000 [ 10%]  (Warmup)
Chain [1] Iteration:  600 / 6000 [ 10%]  (Warmup)
Chain [8] Iteration:  600 / 6000 [ 10%]  (Warmup)
Chain [3] Iteration:  600 / 6000 [ 10%]  (Warmup)
Chain [7] Iteration:  600 / 6000 [ 10%]  (Warmup)
Chain [4] Iteration:  700 / 6000 [ 11%]  (Warmup)
Chain [5] Iteration:  700 / 6000 [ 11%]  (Warmup)
Chain [2] Iteration:  700 / 6000 [ 11%]  (Warmup)
Chain [6] Iteration:  700 / 6000 [ 11%]  (Warmup)
Chain [3] Iteration:  700 / 6000 [ 11%]  (Warmup)
Chain [1] Iteration:  700 / 6000 [ 11%]  (Warmup)
Chain [8] Iteration:  700 / 6000 [ 11%]  (Warmup)
Chain [7] Iteration:  700 / 6000 [ 11%]  (Warmup)
Chain [5] Iteration:  800 / 6000 [ 13%]  (Warmup)
Chain [4] Iteration:  800 / 6000 [ 13%]  (Warmup)
Chain [2] Iteration:  800 / 6000 [ 13%]  (Warmup)
Chain [6] Iteration:  800 / 6000 [ 13%]  (Warmup)
Chain [3] Iteration:  800 / 6000 [ 13%]  (Warmup)
Chain [1] Iteration:  800 / 6000 [ 13%]  (Warmup)
Chain [8] Iteration:  800 / 6000 [ 13%]  (Warmup)
Chain [7] Iteration:  800 / 6000 [ 13%]  (Warmup)
Chain [4] Iteration:  900 / 6000 [ 15%]  (Warmup)
Chain [5] Iteration:  900 / 6000 [ 15%]  (Warmup)
Chain [2] Iteration:  900 / 6000 [ 15%]  (Warmup)
Chain [6] Iteration:  900 / 6000 [ 15%]  (Warmup)
Chain [3] Iteration:  900 / 6000 [ 15%]  (Warmup)
Chain [8] Iteration:  900 / 6000 [ 15%]  (Warmup)
Chain [1] Iteration:  900 / 6000 [ 15%]  (Warmup)
Chain [7] Iteration:  900 / 6000 [ 15%]  (Warmup)
Chain [4] Iteration: 1000 / 6000 [ 16%]  (Warmup)
Chain [5] Iteration: 1000 / 6000 [ 16%]  (Warmup)
Chain [2] Iteration: 1000 / 6000 [ 16%]  (Warmup)
Chain [8] Iteration: 1000 / 6000 [ 16%]  (Warmup)
Chain [6] Iteration: 1000 / 6000 [ 16%]  (Warmup)
Chain [3] Iteration: 1000 / 6000 [ 16%]  (Warmup)
Chain [7] Iteration: 1000 / 6000 [ 16%]  (Warmup)
Chain [1] Iteration: 1000 / 6000 [ 16%]  (Warmup)
Chain [4] Iteration: 1100 / 6000 [ 18%]  (Warmup)
Chain [5] Iteration: 1100 / 6000 [ 18%]  (Warmup)
Chain [2] Iteration: 1100 / 6000 [ 18%]  (Warmup)
Chain [8] Iteration: 1100 / 6000 [ 18%]  (Warmup)
Chain [3] Iteration: 1100 / 6000 [ 18%]  (Warmup)
Chain [7] Iteration: 1100 / 6000 [ 18%]  (Warmup)
Chain [6] Iteration: 1100 / 6000 [ 18%]  (Warmup)
Chain [1] Iteration: 1100 / 6000 [ 18%]  (Warmup)
Chain [2] Iteration: 1200 / 6000 [ 20%]  (Warmup)
Chain [4] Iteration: 1200 / 6000 [ 20%]  (Warmup)
Chain [5] Iteration: 1200 / 6000 [ 20%]  (Warmup)
Chain [8] Iteration: 1200 / 6000 [ 20%]  (Warmup)
Chain [7] Iteration: 1200 / 6000 [ 20%]  (Warmup)
Chain [3] Iteration: 1200 / 6000 [ 20%]  (Warmup)
Chain [1] Iteration: 1200 / 6000 [ 20%]  (Warmup)
Chain [6] Iteration: 1200 / 6000 [ 20%]  (Warmup)
Chain [4] Iteration: 1300 / 6000 [ 21%]  (Warmup)
Chain [2] Iteration: 1300 / 6000 [ 21%]  (Warmup)
Chain [5] Iteration: 1300 / 6000 [ 21%]  (Warmup)
Chain [8] Iteration: 1300 / 6000 [ 21%]  (Warmup)
Chain [7] Iteration: 1300 / 6000 [ 21%]  (Warmup)
Chain [3] Iteration: 1300 / 6000 [ 21%]  (Warmup)
Chain [6] Iteration: 1300 / 6000 [ 21%]  (Warmup)
Chain [1] Iteration: 1300 / 6000 [ 21%]  (Warmup)
Chain [4] Iteration: 1400 / 6000 [ 23%]  (Warmup)
Chain [2] Iteration: 1400 / 6000 [ 23%]  (Warmup)
Chain [5] Iteration: 1400 / 6000 [ 23%]  (Warmup)
Chain [8] Iteration: 1400 / 6000 [ 23%]  (Warmup)
Chain [7] Iteration: 1400 / 6000 [ 23%]  (Warmup)
Chain [3] Iteration: 1400 / 6000 [ 23%]  (Warmup)
Chain [6] Iteration: 1400 / 6000 [ 23%]  (Warmup)
Chain [1] Iteration: 1400 / 6000 [ 23%]  (Warmup)
Chain [4] Iteration: 1500 / 6000 [ 25%]  (Warmup)
Chain [2] Iteration: 1500 / 6000 [ 25%]  (Warmup)
Chain [5] Iteration: 1500 / 6000 [ 25%]  (Warmup)
Chain [8] Iteration: 1500 / 6000 [ 25%]  (Warmup)
Chain [7] Iteration: 1500 / 6000 [ 25%]  (Warmup)
Chain [3] Iteration: 1500 / 6000 [ 25%]  (Warmup)
Chain [6] Iteration: 1500 / 6000 [ 25%]  (Warmup)
Chain [1] Iteration: 1500 / 6000 [ 25%]  (Warmup)
Chain [4] Iteration: 1600 / 6000 [ 26%]  (Warmup)
Chain [2] Iteration: 1600 / 6000 [ 26%]  (Warmup)
Chain [5] Iteration: 1600 / 6000 [ 26%]  (Warmup)
Chain [8] Iteration: 1600 / 6000 [ 26%]  (Warmup)
Chain [7] Iteration: 1600 / 6000 [ 26%]  (Warmup)
Chain [3] Iteration: 1600 / 6000 [ 26%]  (Warmup)
Chain [6] Iteration: 1600 / 6000 [ 26%]  (Warmup)
Chain [1] Iteration: 1600 / 6000 [ 26%]  (Warmup)
Chain [4] Iteration: 1700 / 6000 [ 28%]  (Warmup)
Chain [2] Iteration: 1700 / 6000 [ 28%]  (Warmup)
Chain [5] Iteration: 1700 / 6000 [ 28%]  (Warmup)
Chain [8] Iteration: 1700 / 6000 [ 28%]  (Warmup)
Chain [7] Iteration: 1700 / 6000 [ 28%]  (Warmup)
Chain [3] Iteration: 1700 / 6000 [ 28%]  (Warmup)
Chain [6] Iteration: 1700 / 6000 [ 28%]  (Warmup)
Chain [1] Iteration: 1700 / 6000 [ 28%]  (Warmup)
Chain [4] Iteration: 1800 / 6000 [ 30%]  (Warmup)
Chain [2] Iteration: 1800 / 6000 [ 30%]  (Warmup)
Chain [5] Iteration: 1800 / 6000 [ 30%]  (Warmup)
Chain [8] Iteration: 1800 / 6000 [ 30%]  (Warmup)
Chain [7] Iteration: 1800 / 6000 [ 30%]  (Warmup)
Chain [3] Iteration: 1800 / 6000 [ 30%]  (Warmup)
Chain [6] Iteration: 1800 / 6000 [ 30%]  (Warmup)
Chain [1] Iteration: 1800 / 6000 [ 30%]  (Warmup)
Chain [4] Iteration: 1900 / 6000 [ 31%]  (Warmup)
Chain [2] Iteration: 1900 / 6000 [ 31%]  (Warmup)
Chain [5] Iteration: 1900 / 6000 [ 31%]  (Warmup)
Chain [8] Iteration: 1900 / 6000 [ 31%]  (Warmup)
Chain [7] Iteration: 1900 / 6000 [ 31%]  (Warmup)
Chain [3] Iteration: 1900 / 6000 [ 31%]  (Warmup)
Chain [6] Iteration: 1900 / 6000 [ 31%]  (Warmup)
Chain [1] Iteration: 1900 / 6000 [ 31%]  (Warmup)
Chain [4] Iteration: 2000 / 6000 [ 33%]  (Warmup)
Chain [2] Iteration: 2000 / 6000 [ 33%]  (Warmup)
Chain [5] Iteration: 2000 / 6000 [ 33%]  (Warmup)
Chain [8] Iteration: 2000 / 6000 [ 33%]  (Warmup)
Chain [7] Iteration: 2000 / 6000 [ 33%]  (Warmup)
Chain [3] Iteration: 2000 / 6000 [ 33%]  (Warmup)
Chain [6] Iteration: 2000 / 6000 [ 33%]  (Warmup)
Chain [1] Iteration: 2000 / 6000 [ 33%]  (Warmup)
Chain [4] Iteration: 2100 / 6000 [ 35%]  (Warmup)
Chain [2] Iteration: 2100 / 6000 [ 35%]  (Warmup)
Chain [5] Iteration: 2100 / 6000 [ 35%]  (Warmup)
Chain [7] Iteration: 2100 / 6000 [ 35%]  (Warmup)
Chain [8] Iteration: 2100 / 6000 [ 35%]  (Warmup)
Chain [3] Iteration: 2100 / 6000 [ 35%]  (Warmup)
Chain [6] Iteration: 2100 / 6000 [ 35%]  (Warmup)
Chain [1] Iteration: 2100 / 6000 [ 35%]  (Warmup)
Chain [4] Iteration: 2200 / 6000 [ 36%]  (Warmup)
Chain [2] Iteration: 2200 / 6000 [ 36%]  (Warmup)
Chain [5] Iteration: 2200 / 6000 [ 36%]  (Warmup)
Chain [7] Iteration: 2200 / 6000 [ 36%]  (Warmup)
Chain [8] Iteration: 2200 / 6000 [ 36%]  (Warmup)
Chain [3] Iteration: 2200 / 6000 [ 36%]  (Warmup)
Chain [6] Iteration: 2200 / 6000 [ 36%]  (Warmup)
Chain [1] Iteration: 2200 / 6000 [ 36%]  (Warmup)
Chain [4] Iteration: 2300 / 6000 [ 38%]  (Warmup)
Chain [2] Iteration: 2300 / 6000 [ 38%]  (Warmup)
Chain [5] Iteration: 2300 / 6000 [ 38%]  (Warmup)
Chain [7] Iteration: 2300 / 6000 [ 38%]  (Warmup)
Chain [8] Iteration: 2300 / 6000 [ 38%]  (Warmup)
Chain [3] Iteration: 2300 / 6000 [ 38%]  (Warmup)
Chain [6] Iteration: 2300 / 6000 [ 38%]  (Warmup)
Chain [1] Iteration: 2300 / 6000 [ 38%]  (Warmup)
Chain [4] Iteration: 2400 / 6000 [ 40%]  (Warmup)
Chain [2] Iteration: 2400 / 6000 [ 40%]  (Warmup)
Chain [5] Iteration: 2400 / 6000 [ 40%]  (Warmup)
Chain [7] Iteration: 2400 / 6000 [ 40%]  (Warmup)
Chain [8] Iteration: 2400 / 6000 [ 40%]  (Warmup)
Chain [3] Iteration: 2400 / 6000 [ 40%]  (Warmup)
Chain [6] Iteration: 2400 / 6000 [ 40%]  (Warmup)
Chain [1] Iteration: 2400 / 6000 [ 40%]  (Warmup)
Chain [4] Iteration: 2500 / 6000 [ 41%]  (Warmup)
Chain [2] Iteration: 2500 / 6000 [ 41%]  (Warmup)
Chain [5] Iteration: 2500 / 6000 [ 41%]  (Warmup)
Chain [7] Iteration: 2500 / 6000 [ 41%]  (Warmup)
Chain [8] Iteration: 2500 / 6000 [ 41%]  (Warmup)
Chain [3] Iteration: 2500 / 6000 [ 41%]  (Warmup)
Chain [6] Iteration: 2500 / 6000 [ 41%]  (Warmup)
Chain [1] Iteration: 2500 / 6000 [ 41%]  (Warmup)
Chain [4] Iteration: 2600 / 6000 [ 43%]  (Warmup)
Chain [2] Iteration: 2600 / 6000 [ 43%]  (Warmup)
Chain [5] Iteration: 2600 / 6000 [ 43%]  (Warmup)
Chain [7] Iteration: 2600 / 6000 [ 43%]  (Warmup)
Chain [8] Iteration: 2600 / 6000 [ 43%]  (Warmup)
Chain [3] Iteration: 2600 / 6000 [ 43%]  (Warmup)
Chain [6] Iteration: 2600 / 6000 [ 43%]  (Warmup)
Chain [1] Iteration: 2600 / 6000 [ 43%]  (Warmup)
Chain [4] Iteration: 2700 / 6000 [ 45%]  (Warmup)
Chain [2] Iteration: 2700 / 6000 [ 45%]  (Warmup)
Chain [5] Iteration: 2700 / 6000 [ 45%]  (Warmup)
Chain [7] Iteration: 2700 / 6000 [ 45%]  (Warmup)
Chain [8] Iteration: 2700 / 6000 [ 45%]  (Warmup)
Chain [3] Iteration: 2700 / 6000 [ 45%]  (Warmup)
Chain [6] Iteration: 2700 / 6000 [ 45%]  (Warmup)
Chain [1] Iteration: 2700 / 6000 [ 45%]  (Warmup)
Chain [4] Iteration: 2800 / 6000 [ 46%]  (Warmup)
Chain [2] Iteration: 2800 / 6000 [ 46%]  (Warmup)
Chain [5] Iteration: 2800 / 6000 [ 46%]  (Warmup)
Chain [7] Iteration: 2800 / 6000 [ 46%]  (Warmup)
Chain [8] Iteration: 2800 / 6000 [ 46%]  (Warmup)
Chain [3] Iteration: 2800 / 6000 [ 46%]  (Warmup)
Chain [6] Iteration: 2800 / 6000 [ 46%]  (Warmup)
Chain [1] Iteration: 2800 / 6000 [ 46%]  (Warmup)
Chain [4] Iteration: 2900 / 6000 [ 48%]  (Warmup)
Chain [2] Iteration: 2900 / 6000 [ 48%]  (Warmup)
Chain [5] Iteration: 2900 / 6000 [ 48%]  (Warmup)
Chain [7] Iteration: 2900 / 6000 [ 48%]  (Warmup)
Chain [8] Iteration: 2900 / 6000 [ 48%]  (Warmup)
Chain [3] Iteration: 2900 / 6000 [ 48%]  (Warmup)
Chain [6] Iteration: 2900 / 6000 [ 48%]  (Warmup)
Chain [1] Iteration: 2900 / 6000 [ 48%]  (Warmup)
Chain [4] Iteration: 3000 / 6000 [ 50%]  (Warmup)
Chain [2] Iteration: 3000 / 6000 [ 50%]  (Warmup)
Chain [5] Iteration: 3000 / 6000 [ 50%]  (Warmup)
Chain [7] Iteration: 3000 / 6000 [ 50%]  (Warmup)
Chain [8] Iteration: 3000 / 6000 [ 50%]  (Warmup)
Chain [3] Iteration: 3000 / 6000 [ 50%]  (Warmup)
Chain [6] Iteration: 3000 / 6000 [ 50%]  (Warmup)
Chain [1] Iteration: 3000 / 6000 [ 50%]  (Warmup)
Chain [4] Iteration: 3100 / 6000 [ 51%]  (Warmup)
Chain [2] Iteration: 3100 / 6000 [ 51%]  (Warmup)
Chain [5] Iteration: 3100 / 6000 [ 51%]  (Warmup)
Chain [7] Iteration: 3100 / 6000 [ 51%]  (Warmup)
Chain [8] Iteration: 3100 / 6000 [ 51%]  (Warmup)
Chain [3] Iteration: 3100 / 6000 [ 51%]  (Warmup)
Chain [6] Iteration: 3100 / 6000 [ 51%]  (Warmup)
Chain [1] Iteration: 3100 / 6000 [ 51%]  (Warmup)
Chain [4] Iteration: 3200 / 6000 [ 53%]  (Warmup)
Chain [2] Iteration: 3200 / 6000 [ 53%]  (Warmup)
Chain [5] Iteration: 3200 / 6000 [ 53%]  (Warmup)
Chain [7] Iteration: 3200 / 6000 [ 53%]  (Warmup)
Chain [8] Iteration: 3200 / 6000 [ 53%]  (Warmup)
Chain [3] Iteration: 3200 / 6000 [ 53%]  (Warmup)
Chain [6] Iteration: 3200 / 6000 [ 53%]  (Warmup)
Chain [1] Iteration: 3200 / 6000 [ 53%]  (Warmup)
Chain [4] Iteration: 3300 / 6000 [ 55%]  (Warmup)
Chain [2] Iteration: 3300 / 6000 [ 55%]  (Warmup)
Chain [5] Iteration: 3300 / 6000 [ 55%]  (Warmup)
Chain [8] Iteration: 3300 / 6000 [ 55%]  (Warmup)
Chain [7] Iteration: 3300 / 6000 [ 55%]  (Warmup)
Chain [3] Iteration: 3300 / 6000 [ 55%]  (Warmup)
Chain [6] Iteration: 3300 / 6000 [ 55%]  (Warmup)
Chain [1] Iteration: 3300 / 6000 [ 55%]  (Warmup)
Chain [4] Iteration: 3400 / 6000 [ 56%]  (Warmup)
Chain [2] Iteration: 3400 / 6000 [ 56%]  (Warmup)
Chain [5] Iteration: 3400 / 6000 [ 56%]  (Warmup)
Chain [8] Iteration: 3400 / 6000 [ 56%]  (Warmup)
Chain [7] Iteration: 3400 / 6000 [ 56%]  (Warmup)
Chain [3] Iteration: 3400 / 6000 [ 56%]  (Warmup)
Chain [6] Iteration: 3400 / 6000 [ 56%]  (Warmup)
Chain [1] Iteration: 3400 / 6000 [ 56%]  (Warmup)
Chain [4] Iteration: 3500 / 6000 [ 58%]  (Warmup)
Chain [2] Iteration: 3500 / 6000 [ 58%]  (Warmup)
Chain [5] Iteration: 3500 / 6000 [ 58%]  (Warmup)
Chain [8] Iteration: 3500 / 6000 [ 58%]  (Warmup)
Chain [7] Iteration: 3500 / 6000 [ 58%]  (Warmup)
Chain [3] Iteration: 3500 / 6000 [ 58%]  (Warmup)
Chain [6] Iteration: 3500 / 6000 [ 58%]  (Warmup)
Chain [1] Iteration: 3500 / 6000 [ 58%]  (Warmup)
Chain [4] Iteration: 3600 / 6000 [ 60%]  (Warmup)
Chain [2] Iteration: 3600 / 6000 [ 60%]  (Warmup)
Chain [5] Iteration: 3600 / 6000 [ 60%]  (Warmup)
Chain [8] Iteration: 3600 / 6000 [ 60%]  (Warmup)
Chain [7] Iteration: 3600 / 6000 [ 60%]  (Warmup)
Chain [3] Iteration: 3600 / 6000 [ 60%]  (Warmup)
Chain [6] Iteration: 3600 / 6000 [ 60%]  (Warmup)
Chain [1] Iteration: 3600 / 6000 [ 60%]  (Warmup)
Chain [4] Iteration: 3700 / 6000 [ 61%]  (Warmup)
Chain [2] Iteration: 3700 / 6000 [ 61%]  (Warmup)
Chain [5] Iteration: 3700 / 6000 [ 61%]  (Warmup)
Chain [8] Iteration: 3700 / 6000 [ 61%]  (Warmup)
Chain [7] Iteration: 3700 / 6000 [ 61%]  (Warmup)
Chain [3] Iteration: 3700 / 6000 [ 61%]  (Warmup)
Chain [6] Iteration: 3700 / 6000 [ 61%]  (Warmup)
Chain [1] Iteration: 3700 / 6000 [ 61%]  (Warmup)
Chain [4] Iteration: 3800 / 6000 [ 63%]  (Warmup)
Chain [2] Iteration: 3800 / 6000 [ 63%]  (Warmup)
Chain [5] Iteration: 3800 / 6000 [ 63%]  (Warmup)
Chain [8] Iteration: 3800 / 6000 [ 63%]  (Warmup)
Chain [7] Iteration: 3800 / 6000 [ 63%]  (Warmup)
Chain [3] Iteration: 3800 / 6000 [ 63%]  (Warmup)
Chain [6] Iteration: 3800 / 6000 [ 63%]  (Warmup)
Chain [1] Iteration: 3800 / 6000 [ 63%]  (Warmup)
Chain [4] Iteration: 3900 / 6000 [ 65%]  (Warmup)
Chain [2] Iteration: 3900 / 6000 [ 65%]  (Warmup)
Chain [5] Iteration: 3900 / 6000 [ 65%]  (Warmup)
Chain [8] Iteration: 3900 / 6000 [ 65%]  (Warmup)
Chain [7] Iteration: 3900 / 6000 [ 65%]  (Warmup)
Chain [3] Iteration: 3900 / 6000 [ 65%]  (Warmup)
Chain [6] Iteration: 3900 / 6000 [ 65%]  (Warmup)
Chain [1] Iteration: 3900 / 6000 [ 65%]  (Warmup)
Chain [4] Iteration: 4000 / 6000 [ 66%]  (Warmup)
Chain [5] Iteration: 4000 / 6000 [ 66%]  (Warmup)
Chain [2] Iteration: 4000 / 6000 [ 66%]  (Warmup)
Chain [8] Iteration: 4000 / 6000 [ 66%]  (Warmup)
Chain [7] Iteration: 4000 / 6000 [ 66%]  (Warmup)
Chain [3] Iteration: 4000 / 6000 [ 66%]  (Warmup)
Chain [6] Iteration: 4000 / 6000 [ 66%]  (Warmup)
Chain [1] Iteration: 4000 / 6000 [ 66%]  (Warmup)
Chain [4] Iteration: 4100 / 6000 [ 68%]  (Warmup)
Chain [5] Iteration: 4100 / 6000 [ 68%]  (Warmup)
Chain [2] Iteration: 4100 / 6000 [ 68%]  (Warmup)
Chain [8] Iteration: 4100 / 6000 [ 68%]  (Warmup)
Chain [7] Iteration: 4100 / 6000 [ 68%]  (Warmup)
Chain [3] Iteration: 4100 / 6000 [ 68%]  (Warmup)
Chain [6] Iteration: 4100 / 6000 [ 68%]  (Warmup)
Chain [1] Iteration: 4100 / 6000 [ 68%]  (Warmup)
Chain [4] Iteration: 4200 / 6000 [ 70%]  (Warmup)
Chain [5] Iteration: 4200 / 6000 [ 70%]  (Warmup)
Chain [2] Iteration: 4200 / 6000 [ 70%]  (Warmup)
Chain [8] Iteration: 4200 / 6000 [ 70%]  (Warmup)
Chain [7] Iteration: 4200 / 6000 [ 70%]  (Warmup)
Chain [3] Iteration: 4200 / 6000 [ 70%]  (Warmup)
Chain [6] Iteration: 4200 / 6000 [ 70%]  (Warmup)
Chain [1] Iteration: 4200 / 6000 [ 70%]  (Warmup)
Chain [4] Iteration: 4300 / 6000 [ 71%]  (Warmup)
Chain [2] Iteration: 4300 / 6000 [ 71%]  (Warmup)
Chain [5] Iteration: 4300 / 6000 [ 71%]  (Warmup)
Chain [8] Iteration: 4300 / 6000 [ 71%]  (Warmup)
Chain [7] Iteration: 4300 / 6000 [ 71%]  (Warmup)
Chain [3] Iteration: 4300 / 6000 [ 71%]  (Warmup)
Chain [6] Iteration: 4300 / 6000 [ 71%]  (Warmup)
Chain [1] Iteration: 4300 / 6000 [ 71%]  (Warmup)
Chain [4] Iteration: 4400 / 6000 [ 73%]  (Warmup)
Chain [8] Iteration: 4400 / 6000 [ 73%]  (Warmup)
Chain [5] Iteration: 4400 / 6000 [ 73%]  (Warmup)
Chain [2] Iteration: 4400 / 6000 [ 73%]  (Warmup)
Chain [7] Iteration: 4400 / 6000 [ 73%]  (Warmup)
Chain [3] Iteration: 4400 / 6000 [ 73%]  (Warmup)
Chain [6] Iteration: 4400 / 6000 [ 73%]  (Warmup)
Chain [1] Iteration: 4400 / 6000 [ 73%]  (Warmup)
Chain [4] Iteration: 4500 / 6000 [ 75%]  (Warmup)
Chain [8] Iteration: 4500 / 6000 [ 75%]  (Warmup)
Chain [2] Iteration: 4500 / 6000 [ 75%]  (Warmup)
Chain [5] Iteration: 4500 / 6000 [ 75%]  (Warmup)
Chain [7] Iteration: 4500 / 6000 [ 75%]  (Warmup)
Chain [3] Iteration: 4500 / 6000 [ 75%]  (Warmup)
Chain [6] Iteration: 4500 / 6000 [ 75%]  (Warmup)
Chain [1] Iteration: 4500 / 6000 [ 75%]  (Warmup)
Chain [4] Iteration: 4600 / 6000 [ 76%]  (Warmup)
Chain [8] Iteration: 4600 / 6000 [ 76%]  (Warmup)
Chain [5] Iteration: 4600 / 6000 [ 76%]  (Warmup)
Chain [2] Iteration: 4600 / 6000 [ 76%]  (Warmup)
Chain [7] Iteration: 4600 / 6000 [ 76%]  (Warmup)
Chain [3] Iteration: 4600 / 6000 [ 76%]  (Warmup)
Chain [6] Iteration: 4600 / 6000 [ 76%]  (Warmup)
Chain [1] Iteration: 4600 / 6000 [ 76%]  (Warmup)
Chain [4] Iteration: 4700 / 6000 [ 78%]  (Warmup)
Chain [8] Iteration: 4700 / 6000 [ 78%]  (Warmup)
Chain [5] Iteration: 4700 / 6000 [ 78%]  (Warmup)
Chain [2] Iteration: 4700 / 6000 [ 78%]  (Warmup)
Chain [7] Iteration: 4700 / 6000 [ 78%]  (Warmup)
Chain [3] Iteration: 4700 / 6000 [ 78%]  (Warmup)
Chain [6] Iteration: 4700 / 6000 [ 78%]  (Warmup)
Chain [1] Iteration: 4700 / 6000 [ 78%]  (Warmup)
Chain [4] Iteration: 4800 / 6000 [ 80%]  (Warmup)
Chain [8] Iteration: 4800 / 6000 [ 80%]  (Warmup)
Chain [5] Iteration: 4800 / 6000 [ 80%]  (Warmup)
Chain [2] Iteration: 4800 / 6000 [ 80%]  (Warmup)
Chain [7] Iteration: 4800 / 6000 [ 80%]  (Warmup)
Chain [3] Iteration: 4800 / 6000 [ 80%]  (Warmup)
Chain [6] Iteration: 4800 / 6000 [ 80%]  (Warmup)
Chain [1] Iteration: 4800 / 6000 [ 80%]  (Warmup)
Chain [4] Iteration: 4900 / 6000 [ 81%]  (Warmup)
Chain [8] Iteration: 4900 / 6000 [ 81%]  (Warmup)
Chain [2] Iteration: 4900 / 6000 [ 81%]  (Warmup)
Chain [5] Iteration: 4900 / 6000 [ 81%]  (Warmup)
Chain [7] Iteration: 4900 / 6000 [ 81%]  (Warmup)
Chain [3] Iteration: 4900 / 6000 [ 81%]  (Warmup)
Chain [6] Iteration: 4900 / 6000 [ 81%]  (Warmup)
Chain [1] Iteration: 4900 / 6000 [ 81%]  (Warmup)
Chain [4] Iteration: 5000 / 6000 [ 83%]  (Warmup)
Chain [4] Iteration: 5001 / 6000 [ 83%]  (Sampling)
Chain [2] Iteration: 5000 / 6000 [ 83%]  (Warmup)
Chain [2] Iteration: 5001 / 6000 [ 83%]  (Sampling)
Chain [5] Iteration: 5000 / 6000 [ 83%]  (Warmup)
Chain [8] Iteration: 5000 / 6000 [ 83%]  (Warmup)
Chain [5] Iteration: 5001 / 6000 [ 83%]  (Sampling)
Chain [8] Iteration: 5001 / 6000 [ 83%]  (Sampling)
Chain [7] Iteration: 5000 / 6000 [ 83%]  (Warmup)
Chain [3] Iteration: 5000 / 6000 [ 83%]  (Warmup)
Chain [7] Iteration: 5001 / 6000 [ 83%]  (Sampling)
Chain [3] Iteration: 5001 / 6000 [ 83%]  (Sampling)
Chain [6] Iteration: 5000 / 6000 [ 83%]  (Warmup)
Chain [6] Iteration: 5001 / 6000 [ 83%]  (Sampling)
Chain [1] Iteration: 5000 / 6000 [ 83%]  (Warmup)
Chain [1] Iteration: 5001 / 6000 [ 83%]  (Sampling)
Chain [4] Iteration: 5100 / 6000 [ 85%]  (Sampling)
Chain [2] Iteration: 5100 / 6000 [ 85%]  (Sampling)
Chain [5] Iteration: 5100 / 6000 [ 85%]  (Sampling)
Chain [8] Iteration: 5100 / 6000 [ 85%]  (Sampling)
Chain [7] Iteration: 5100 / 6000 [ 85%]  (Sampling)
Chain [3] Iteration: 5100 / 6000 [ 85%]  (Sampling)
Chain [6] Iteration: 5100 / 6000 [ 85%]  (Sampling)
Chain [1] Iteration: 5100 / 6000 [ 85%]  (Sampling)
Chain [4] Iteration: 5200 / 6000 [ 86%]  (Sampling)
Chain [2] Iteration: 5200 / 6000 [ 86%]  (Sampling)
Chain [5] Iteration: 5200 / 6000 [ 86%]  (Sampling)
Chain [8] Iteration: 5200 / 6000 [ 86%]  (Sampling)
Chain [7] Iteration: 5200 / 6000 [ 86%]  (Sampling)
Chain [3] Iteration: 5200 / 6000 [ 86%]  (Sampling)
Chain [6] Iteration: 5200 / 6000 [ 86%]  (Sampling)
Chain [1] Iteration: 5200 / 6000 [ 86%]  (Sampling)
Chain [4] Iteration: 5300 / 6000 [ 88%]  (Sampling)
Chain [2] Iteration: 5300 / 6000 [ 88%]  (Sampling)
Chain [8] Iteration: 5300 / 6000 [ 88%]  (Sampling)
Chain [5] Iteration: 5300 / 6000 [ 88%]  (Sampling)
Chain [7] Iteration: 5300 / 6000 [ 88%]  (Sampling)
Chain [3] Iteration: 5300 / 6000 [ 88%]  (Sampling)
Chain [6] Iteration: 5300 / 6000 [ 88%]  (Sampling)
Chain [1] Iteration: 5300 / 6000 [ 88%]  (Sampling)
Chain [4] Iteration: 5400 / 6000 [ 90%]  (Sampling)
Chain [2] Iteration: 5400 / 6000 [ 90%]  (Sampling)
Chain [8] Iteration: 5400 / 6000 [ 90%]  (Sampling)
Chain [5] Iteration: 5400 / 6000 [ 90%]  (Sampling)
Chain [7] Iteration: 5400 / 6000 [ 90%]  (Sampling)
Chain [3] Iteration: 5400 / 6000 [ 90%]  (Sampling)
Chain [6] Iteration: 5400 / 6000 [ 90%]  (Sampling)
Chain [1] Iteration: 5400 / 6000 [ 90%]  (Sampling)
Chain [4] Iteration: 5500 / 6000 [ 91%]  (Sampling)
Chain [2] Iteration: 5500 / 6000 [ 91%]  (Sampling)
Chain [8] Iteration: 5500 / 6000 [ 91%]  (Sampling)
Chain [5] Iteration: 5500 / 6000 [ 91%]  (Sampling)
Chain [7] Iteration: 5500 / 6000 [ 91%]  (Sampling)
Chain [3] Iteration: 5500 / 6000 [ 91%]  (Sampling)
Chain [6] Iteration: 5500 / 6000 [ 91%]  (Sampling)
Chain [1] Iteration: 5500 / 6000 [ 91%]  (Sampling)
Chain [4] Iteration: 5600 / 6000 [ 93%]  (Sampling)
Chain [2] Iteration: 5600 / 6000 [ 93%]  (Sampling)
Chain [5] Iteration: 5600 / 6000 [ 93%]  (Sampling)
Chain [8] Iteration: 5600 / 6000 [ 93%]  (Sampling)
Chain [7] Iteration: 5600 / 6000 [ 93%]  (Sampling)
Chain [3] Iteration: 5600 / 6000 [ 93%]  (Sampling)
Chain [6] Iteration: 5600 / 6000 [ 93%]  (Sampling)
Chain [1] Iteration: 5600 / 6000 [ 93%]  (Sampling)
Chain [4] Iteration: 5700 / 6000 [ 95%]  (Sampling)
Chain [2] Iteration: 5700 / 6000 [ 95%]  (Sampling)
Chain [5] Iteration: 5700 / 6000 [ 95%]  (Sampling)
Chain [8] Iteration: 5700 / 6000 [ 95%]  (Sampling)
Chain [7] Iteration: 5700 / 6000 [ 95%]  (Sampling)
Chain [3] Iteration: 5700 / 6000 [ 95%]  (Sampling)
Chain [6] Iteration: 5700 / 6000 [ 95%]  (Sampling)
Chain [1] Iteration: 5700 / 6000 [ 95%]  (Sampling)
Chain [4] Iteration: 5800 / 6000 [ 96%]  (Sampling)
Chain [2] Iteration: 5800 / 6000 [ 96%]  (Sampling)
Chain [5] Iteration: 5800 / 6000 [ 96%]  (Sampling)
Chain [8] Iteration: 5800 / 6000 [ 96%]  (Sampling)
Chain [7] Iteration: 5800 / 6000 [ 96%]  (Sampling)
Chain [3] Iteration: 5800 / 6000 [ 96%]  (Sampling)
Chain [6] Iteration: 5800 / 6000 [ 96%]  (Sampling)
Chain [1] Iteration: 5800 / 6000 [ 96%]  (Sampling)
Chain [4] Iteration: 5900 / 6000 [ 98%]  (Sampling)
Chain [2] Iteration: 5900 / 6000 [ 98%]  (Sampling)
Chain [5] Iteration: 5900 / 6000 [ 98%]  (Sampling)
Chain [8] Iteration: 5900 / 6000 [ 98%]  (Sampling)
Chain [7] Iteration: 5900 / 6000 [ 98%]  (Sampling)
Chain [3] Iteration: 5900 / 6000 [ 98%]  (Sampling)
Chain [6] Iteration: 5900 / 6000 [ 98%]  (Sampling)
Chain [1] Iteration: 5900 / 6000 [ 98%]  (Sampling)
Chain [4] Iteration: 6000 / 6000 [100%]  (Sampling)

 Elapsed Time: 5448.98 seconds (Warm-up)
               1134.54 seconds (Sampling)
               6583.52 seconds (Total)

Chain [2] Iteration: 6000 / 6000 [100%]  (Sampling)

 Elapsed Time: 5453.06 seconds (Warm-up)
               1134.51 seconds (Sampling)
               6587.57 seconds (Total)

Chain [5] Iteration: 6000 / 6000 [100%]  (Sampling)

 Elapsed Time: 5458.26 seconds (Warm-up)
               1132.12 seconds (Sampling)
               6590.38 seconds (Total)

Chain [8] Iteration: 6000 / 6000 [100%]  (Sampling)

 Elapsed Time: 5458.49 seconds (Warm-up)
               1133.37 seconds (Sampling)
               6591.87 seconds (Total)

Chain [7] Iteration: 6000 / 6000 [100%]  (Sampling)

 Elapsed Time: 5463.18 seconds (Warm-up)
               1131.45 seconds (Sampling)
               6594.64 seconds (Total)

Chain [3] Iteration: 6000 / 6000 [100%]  (Sampling)

 Elapsed Time: 5466.54 seconds (Warm-up)
               1131.53 seconds (Sampling)
               6598.07 seconds (Total)

Chain [6] Iteration: 6000 / 6000 [100%]  (Sampling)

 Elapsed Time: 5475.86 seconds (Warm-up)
               1127.31 seconds (Sampling)
               6603.17 seconds (Total)

Chain [1] Iteration: 6000 / 6000 [100%]  (Sampling)

 Elapsed Time: 5485.22 seconds (Warm-up)
               1123.69 seconds (Sampling)
               6608.91 seconds (Total)

double free or corruption (!prev)

The python error indicates (only relevant parts included)

fit = model.sample(data=f'{path}/data.json', output_dir=output_dir1,
                            refresh=100, iter_warmup=5000, 
                            iter_sampling=1000, chains=chains, parallel_chains=chains, 
                            threads_per_chain=threads_per_chain, max_treedepth=5,
                            metric='dense_e', save_profile=True, sig_figs=18)

CmdStan error: terminated by signal 6 Unknown error -6
RuntimeError: Error during sampling:
Exception: Exception: multi_normal_lpdf: Location parameter[1] is -nan, but must be finite! (in 'Pure_PMF.stan', line 62, column 20 to column 112) (in 'Pure_PMF.stan', line 192, column 12 to line 195, column 39)
Command and output files:
RunSet: chains=8, chain_ids=[1, 2, 3, 4, 5, 6, 7, 8], num_processes=1
 cmd (chain 1):
	['/home/22796002/Pure RK PMF/Subsets/Alkane_Primary alcohol/Include_clusters_False/Variance_known_True/Variance_MC_known_True/rank_2/Pure_PMF', 'id=1', 'random', 'seed=24898', 'data', 'file=Subsets/Alkane_Primary alcohol/Include_clusters_False/Variance_known_True/Variance_MC_known_True/rank_2/data.json', 'output', 'file=/home/22796002/Pure RK PMF/Subsets/Alkane_Primary alcohol/Include_clusters_False/Variance_known_True/Variance_MC_known_True/rank_2/Step1/Pure_PMF-20240522025118.csv', 'profile_file=/home/22796002/Pure RK PMF/Subsets/Alkane_Primary alcohol/Include_clusters_False/Variance_known_True/Variance_MC_known_True/rank_2/Step1/Pure_PMF-20240522025118-profile.csv', 'refresh=100', 'sig_figs=18', 'method=sample', 'num_samples=1000', 'num_warmup=5000', 'algorithm=hmc', 'engine=nuts', 'max_depth=5', 'metric=dense_e', 'adapt', 'engaged=1', 'num_chains=8']
 retcodes=[-6]

Reconstructing the fit from .csv files does not produce any errors

Stan code for reference 

    functions {
        // kernel for composition
        matrix Kx(vector x1, vector x2, int order) {
            int N = rows(x1);
            int M = rows(x2);
            matrix[N, order+1] X1;
            matrix[M, order+1] X2;
            for (i in 1:order) {
                X1[:,i] = x1 .^(order+2-i) - x1;
                X2[:,i] = x2 .^(order+2-i) - x2;
            }
            X1[:,order+1] = 1e-1 * x1 .* sqrt(1-x1) .* exp(x1);
            X2[:,order+1] = 1e-1 * x2 .* sqrt(1-x2) .* exp(x2);
            return X1 * X2';
        }

        // kernel for Temperature
        matrix KT(vector T1, vector T2) {
            int N = rows(T1);
            int M = rows(T2);
            matrix[N, 4] TT1 = append_col(append_col(append_col(rep_vector(1.0, N), T1), 1e-2* T1.^2), 1e-4* T1.^3);
            matrix[M, 4] TT2 = append_col(append_col(append_col(rep_vector(1.0, M), T2), 1e-2* T2.^2), 1e-4* T2.^3);

            return TT1 * TT2';
        }

        // Combined kernel
        matrix Kernel(vector x1, vector x2, vector T1, vector T2, int order) {
            return Kx(x1, x2, order) .* KT(T1, T2); 
        }
        // functions for the priors of the feature matrices
        real ps_feature_matrices(array[] int M_slice, int start, int end, array[] matrix U_raw, 
          array[] matrix V_raw) {
            real all_target = 0;
            for (m in start:end) {
                all_target += std_normal_lpdf(to_vector(U_raw[m]));
                all_target += std_normal_lpdf(to_vector(V_raw[m]));
            }
            return all_target;
          }
        // function for the likelihood and prior on smoothed values
        real ps_like(array[] int N_slice, int start, int end, vector y, matrix cov_f2_f1,
          matrix mu_y_y_MC, matrix L_MC_inv, vector v,
          array[,] int Idx_all, int M, int N_T, int N_MC, int N_C, int N_known, array[] int N_points,
          vector v_ARD, array[,] matrix U_raw, array[,] matrix V_raw, real v_MC) {
            real all_target = 0;
            for (i in start:end) {
                vector[N_MC] y_MC_pred;
                for(t in 1:N_T) {
                    for (m in 1:M-1) {
                        y_MC_pred[m+N_C*(t-1)] = dot_product(U_raw[t,m,:,Idx_all[i,1]] .* v_ARD, V_raw[t,m,:,Idx_all[i,2]]);
                        y_MC_pred[N_C-m+1+N_C*(t-1)] = dot_product(U_raw[t,m,:,Idx_all[i,2]] .* v_ARD, V_raw[t,m,:,Idx_all[i,1]]); 
                    }
                    y_MC_pred[M+N_C*(t-1)] = dot_product(U_raw[t,M,:,Idx_all[i,1]] .* v_ARD, U_raw[t,M,:,Idx_all[i,2]]);
                }
                all_target += -0.5 * dot_self(L_MC_inv * y_MC_pred); // GP prior

                if (i <= N_known) {
                    vector[N_points[i]] y_pred = mu_y_y_MC[sum(N_points[:i-1])+1:sum(N_points[:i]),:] * y_MC_pred;
                    matrix[N_points[i], N_points[i]] cov_y = add_diag(cov_f2_f1[sum(N_points[:i-1])+1:sum(N_points[:i]), :N_points[i]], v[i]);
                    all_target += multi_normal_lpdf(y[sum(N_points[:i-1])+1:sum(N_points[:i])] | y_pred, cov_y);
                }
            }
            return all_target;
          }
    }

    data {
        int N_known;                        // number of known mixtures
        int N_unknown;                      // number of unknown mixtures
        array[N_known] int N_points;        // number of experimental points per known dataset
        int order;                          // order of the compositional polynomial
        vector[sum(N_points)] x1;           // experimental composition
        vector[sum(N_points)] T1;           // experimental temperatures
        vector[sum(N_points)] y1;           // experimental excess enthalpy
        int N_T;                            // number of interpolated temperatures
        int N_C;                            // number of interpolated compositions
        vector[N_T] T2_int;                 // unique temperatures to interpolate
        vector[N_C] x2_int;                 // unique compositions to interpolate
        real<lower=0> v_MC;                 // error between of y_MC and (U,V)
        int grainsize;                      // number of grainsizes

        int N;                              // number of components
        int D;                              // rank of feature matrices
        array[N_known, 2] int Idx_known;    // indices (row, column) of known datasets
        array[N_unknown, 2] int Idx_unknown;// indices (row, column) of unknown datasets
        
        real<lower=0> jitter;               // jitter for stability of covariances
        vector<lower=0>[N_known] v;         // known data-model variance
    }

    transformed data {
        real error = 0.01;                                      // error in the data (fraction of experimental data)
        vector[sum(N_points)] var_data = square(error*y1);      // variance of the data
        int M = (N_C + 1) %/% 2;                                // interger division to get the number of U matrices
        int N_MC = N_C*N_T;                                     // overall number of interpolated datapoints per dataset
        array[M-1] int M_slice;                                 // array of integers to be used as indices in parallel computations
        array[N_known+N_unknown] int N_slice;                   // array of integers to be used as indices in parallel computations
        array[N_known+N_unknown,2] int Idx_all = append_array(Idx_known, Idx_unknown); // indices of all datasets
        vector[N_MC] x2;                                        // concatnated vector of x2_int
        vector[N_MC] T2;                                        // concatenated vector of T2_int
        matrix[N_MC, N_MC] K_MC;                                // kernel for the interpolated data
        matrix[N_MC, N_MC] y_MC_prec;                            // inverse of kernel
        matrix[sum(N_points), N_MC] mu_y_y_MC;                  // Mapping from y_MC to y (predictive GP mean)
        matrix[sum(N_points), max(N_points)] cov_f2_f1;         // covariance of y smmooth|y_MC
        matrix[N_MC, N_MC] L_y_MC_inv_cov;                            // Inverse of cholesky decomposition of K_MC
        matrix[sum(N_points), max(N_points)] cov_y_yMC;         // conditional covariance matrix y|y_MC
        matrix[N_MC, N_MC] cov_y_MC;                            // conditional covariance of y_MC|(U,V)
        // Assign MC input vectors for temperature and composition
        for (i in 1:N_T) {
            x2[(i-1)*N_C+1:i*N_C] = x2_int;
            T2[(i-1)*N_C+1:i*N_C] = rep_vector(T2_int[i],N_C);
        } 

        // Assign MC kernel
        K_MC = add_diag(Kernel(x2, x2, T2, T2, order), jitter);
        // Compute inverse using cholesky
        {
            L_y_MC_inv_cov = inverse(cholesky_decompose(add_diag(K_MC, v_MC)));
            y_MC_prec = crossprod(L_y_MC_inv_cov);         // L_MC_inv' * L_MC_inv
        }

        {
            // Loop through data sets to compute the relevant matrix, mapping from y_MC to y and covariance
            for (i in 1:N_known) {
                // obtain kernel matrix for cross covariance
                matrix[N_points[i], N_MC] K_y_yMC = Kernel(x1[sum(N_points[:i-1])+1:sum(N_points[:i])], 
                                                        x2, 
                                                        T1[sum(N_points[:i-1])+1:sum(N_points[:i])], 
                                                        T2, order); 

                // Stable implementation of A*B*A^T where B is a positive semi-definite matrix
                // This is decomposed into A*(L^-1)^T*L^-1*A^T where L is the cholesky factor of B
                // A*B*A^T = C*C^T where C = A*(L^-1)^T

                // Computation of above mentioned K12*K22^-1*K12^T
                matrix[N_points[i], N_MC] stable_inv = K_y_yMC * L_y_MC_inv_cov';
                // obtain kernel for the data
                matrix[N_points[i], N_points[i]] K_y = Kernel(x1[sum(N_points[:i-1])+1:sum(N_points[:i])], 
                                                            x1[sum(N_points[:i-1])+1:sum(N_points[:i])], 
                                                            T1[sum(N_points[:i-1])+1:sum(N_points[:i])], 
                                                            T1[sum(N_points[:i-1])+1:sum(N_points[:i])], order);
                // Computation of K12*K22^-1*K12^T
                cov_f2_f1[sum(N_points[:i-1])+1:sum(N_points[:i]), :N_points[i]] = add_diag(K_y - stable_inv * stable_inv', var_data[sum(N_points[:i-1])+1:sum(N_points[:i])]);
                // Mapping from y_MC to y
                mu_y_y_MC[sum(N_points[:i-1])+1:sum(N_points[:i]), :] = K_y_yMC * y_MC_prec;

                // slice variable for parallel computations
                N_slice[i] = i;
            }
        }
        
        // Continue to assign the slice variable for unknown datasets
        for (i in 1:N_unknown) {
            N_slice[N_known+i] = N_known+i;
        }

        // Slice variables for feature matrices
        M_slice = N_slice[:M-1];
    }

    parameters {
        array[N_T, M] matrix[D,N] U_raw;            // feature matrices U
        array[N_T, M-1] matrix[D,N] V_raw;          // feature matrices V
        real<lower=0> scale;                        // scale dictating the strenght of ARD effect
        vector<lower=0>[D] v_ARD;                   // ARD variances aranged in increasing order with lower bound zero
    }
    
    model {
        // Gamma prior for scale
        profile("Scale Prior"){
            scale ~ gamma(1e-9, 1e-9);
        }

        // Exponential prior on ARD variances
        profile("ARD Prior"){
            v_ARD ~ exponential(0.1);
        }
    
        // priors for feature matrices
        profile("Feature Matrices"){
            for (t in 1:N_T) {
                target += reduce_sum(ps_feature_matrices, M_slice, grainsize, U_raw[t,:M-1], 
                            V_raw[t,:]);
                to_vector(U_raw[t,M,:,:]) ~ std_normal();
            }
        }
        
        // Likelihood function
        profile("Likelihood"){
            target += reduce_sum(ps_like, N_slice, grainsize, y1, cov_f2_f1,
            mu_y_y_MC, L_y_MC_inv_cov, v,
            Idx_all, M, N_T, N_MC, N_C, N_known, N_points,
            v_ARD, U_raw, V_raw, v_MC);
        }
    }

I do not know if the error might be related to the previous issue raised

Versions

cmdstan 2.34.0 hff4ab46_0 conda-forge
cmdstanpy 1.2.1 pyhd8ed1ab_0 conda-forge

OS: Scientific Linux 7.9 (Nitrogen)
@ahartikainen
Copy link
Contributor

I bet it is a problem with paths containing space char. Can you move your location and retest?

Weird thing is that there should be automatic fix for this.

@Garren-H
Copy link
Author

I am running the same code by only changing the value of D in the data file. For the case when D=1 the code did not result in any errors, the above was for when D=2. Surely it would've given the same error for when D=1 if this were the case?

@ahartikainen
Copy link
Contributor

yeah, true it would not make any sense

Have you tried add prints to debug what can be seen and when it crashes (or at what step)?

@Garren-H
Copy link
Author

Surely it has to be post-processing as the fit can be reconstructed using the from_csv method from the csv files. The thing that bugs me is the final line in the .txt output from stan, which I have not previously encountered.

double free or corruption (!prev)

@WardBrian
Copy link
Member

Can you run the model under gdb to get a backtrace from that double free?

I saw your previous issue saying it wasn’t recreatable on another machine, is this one?

@Garren-H
Copy link
Author

I have no idea what gdb is?

I will see if I can reproduce the error on my personal device. I do however have less cores on my personal device, so it will take longer. I will report back once this has completed.

@WardBrian
Copy link
Member

Do these issues still occur if you remove the use of profile from the model code? We recently fixed some issues with that functionality that could be related

@Garren-H
Copy link
Author

I am running the model with and without profiling on the cluster (changed the stan code to remove profiles) to see if the error can be reproduced on the cluster. Will take a few hours to complete

@Garren-H
Copy link
Author

I have ran these models and it seems that this error is not reproducible either. Both the model with and without profiling completed without errors this time around

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ahartikainen @WardBrian @Garren-H