Stan crashes (segfaults?) if a data tuple containing a vector gets "deconstructed", "reconstructed" in nested functions without `data` annotations which also take `parameter` arguments and then the values of the vector get accessed.

[nsiccha]

While working around stan-dev/math#3041, I've encountered the above issue.

My workaround for reduce_sum's inability to take tuple arguments was to first "deconstruct" (flatten) them before passing them to reduce_sum, and then to "reconstruct" (restore the old non-flat structure) them before passing them to the original function.

This however lead to segfaults. The following is AFAICT a minimal reproducer:

R code using cmdstanr 2.36.0

library(cmdstanr)
model = cmdstan_model(
    stan_file=paste(getwd(), "stan", "tuples.stan", sep="/"),
)
post = model$sample(
    data=list(),
    chains=1,
)

Stan code

functions {
    void do_something_reconstruct_deconstruct(tuple(array[] int, vector) combined, real x){
        do_something_reconstruct(combined.1, combined.2, x);
    }
    void do_something_reconstruct(array[] int indices, vector mem, real x){
        do_something((indices, mem), x);
    }
    void do_something(tuple(array[] int, vector) combined, real x){
        print(combined.1); // This is fine
        print(size(combined.2)); // This is fine
        print(combined); // This crashes
        // print(combined.2); // This crashes
        // print(sum(combined.2)); // This crashes
    }
}
transformed data {
    int n = 2;
    array[n] int indices = rep_array(0, n);
    vector[n] mem = rep_vector(0., n);
    tuple(array[n] int, vector[n]) combined = (indices, mem);
    print(combined);
    do_something_reconstruct_deconstruct(combined, 1.);
}
parameters {
    real x;
}
model {
    x ~ std_normal();
    print(2);
    // This is fine
    do_something_reconstruct_deconstruct(combined, 1.);
    print(3);
    // This is fine
    do_something_reconstruct_deconstruct(combined, 1.);
    print(4);
    // This crashes
    do_something_reconstruct_deconstruct(combined, x);
    print(5);
    // This crashes
    do_something_reconstruct_deconstruct(combined, x);
}

[nsiccha]

Funnily enough, if you provide all data annotations that Stan can handle (i.e. excluding the one mentioned in #3225), things work. That is, the following stan file happily samples (noisily because of all the print statements):

functions {
    void do_something_reconstruct_deconstruct(data tuple(array[] int, vector) combined, real x){
        do_something_reconstruct(combined.1, combined.2, x);
    }
    void do_something_reconstruct(data array[] int indices, data vector mem, real x){
        do_something((indices, mem), x);
    }
    void do_something(tuple(array[] int, vector) combined, real x){
        print(combined.1); // This is fine
        print(size(combined.2)); // This is fine
        print(combined); // This is NOW (with the data annotations above) fine
    }
}
transformed data {
    int n = 2;
    array[n] int indices = rep_array(0, n);
    vector[n] mem = rep_vector(0., n);
    tuple(array[n] int, vector[n]) combined = (indices, mem);
    print(combined);
    do_something_reconstruct_deconstruct(combined, 1.);
}
parameters {
    real x;
}
model {
    x ~ std_normal();
    print(2);
    // This is fine
    do_something_reconstruct_deconstruct(combined, 1.);
    print(3);
    // This is fine
    do_something_reconstruct_deconstruct(combined, 1.);
    print(4);
    // This is NOW (with the data annotations above) fine
    do_something_reconstruct_deconstruct(combined, x);
    print(5);
    // This is NOW (with the data annotations above) fine
    do_something_reconstruct_deconstruct(combined, x);
}

[nsiccha]

And if this helps to diagnose things, if I slice into the vector element, Stan sometimes/often prints this runtime error (but still crashes):

Chain 1 Exception: Exception: Exception: Exception: vector[min_max] min indexing: accessing element out of range. index 1 out of range; expecting index to be between 1 and -853419584 (in '/tmp/Rtmp17MqPt/model-cbef45c02bfd.stan', line 12, column 8 to column 36) (in '/tmp/Rtmp17MqPt/model-cbef45c02bfd.stan', line 6, column 8 to column 47) (in '/tmp/Rtmp17MqPt/model-cbef45c02bfd.stan', line 3, column 8 to column 67) (in '/tmp/Rtmp17MqPt/model-cbef45c02bfd.stan', line 27, column 4 to column 64)
Chain 1 Exception: Exception: Exception: Exception: vector[min_max] min indexing: accessing element out of range. index 1 out of range; expecting index to be between 1 and -853419584 (in '/tmp/Rtmp17MqPt/model-cbef45c02bfd.stan', line 12, column 8 to column 36) (in '/tmp/Rtmp17MqPt/model-cbef45c02bfd.stan', line 6, column 8 to column 47) (in '/tmp/Rtmp17MqPt/model-cbef45c02bfd.stan', line 3, column 8 to column 67) (in '/tmp/Rtmp17MqPt/model-cbef45c02bfd.stan', line 27, column 4 to column 64)
Warning: Chain 1 finished unexpectedly!

Stan file

functions {
    real do_something_reconstruct_deconstruct(tuple(array[] int, vector) combined, real x){
        return do_something_reconstruct(combined.1, combined.2, x);
    }
    real do_something_reconstruct(array[] int indices, vector mem, real x){
        return do_something((indices, mem), x);
    }
    real do_something(tuple(array[] int, vector) combined, real x){
        // This DOES NOT print a runtime error (but still crashes).
        // return combined.2[1];
        // This DOES sometimes print a runtime error (but still crashes).
        return sum(combined.2[1:1]);
    }
}
transformed data {
    int n = 2;
    array[n] int indices = rep_array(0, n);
    vector[n] mem = rep_vector(0., n);
    tuple(array[n] int, vector[n]) combined = (indices, mem);
}
parameters {
    real x;
}
model {
    x ~ std_normal();
    // This crashes
    target += do_something_reconstruct_deconstruct(combined, x);
}

[WardBrian]

And if this helps to diagnose things, if I slice into the vector element, Stan sometimes/often prints this runtime error (but still crashes):

That error to me makes it seem like the vector is falling out of scope, so the size being printed is garbage now. Not sure if that would be a bug in the code generation or in rvalue (@SteveBronder would probably know better).

Thanks for reporting @nsiccha!

[nsiccha]

Right, I forgot to mention one thing! Printing actually works, an arbitrary number of times within a single evaluation of the log density gradient, before it then crashes. Let me actually attach the output of the above combination.

[nsiccha]

This is the output of the combination in the first issue description:

Running MCMC with 1 chain...

Chain 1 ([0,0],[0,0]) 
Chain 1 [0,0] 
Chain 1 2 
Chain 1 ([0,0],[0,0]) 
Chain 1 2 
Chain 1 [0,0] 
Chain 1 2 
Chain 1 ([0,0],[0,0]) 
Chain 1 3 
Chain 1 [0,0] 
Chain 1 2 
Chain 1 ([0,0],[0,0]) 
Chain 1 4 
Chain 1 [0,0] 
Chain 1 2 
Chain 1 ([0,0],[0,0]) 
Chain 1 5 
Chain 1 [0,0] 
Chain 1 2 
Chain 1 ([0,0],[0,0]) 
Warning: Chain 1 finished unexpectedly!

[WardBrian]

Replacing the code generation of do_something_reconstruct with a call to forward_as_tuple instead of building our own std::tuple seems to work. @SteveBronder and I cannot entirely remember why we stopped code generating it that way during the original tuples PR, we know we did at one point and them moved to our current strategy.

[WardBrian]

3/3 on these issues really being compiler problems 🥶

This turns out to be another issue due to 599774d, one of the original sins of the tuple implementation, technically speaking.

The correct solution seems to be properly using forward_as_tuple.