@jeremylt

Can you point me to the line in Ratel? The Ratel test suite passed for me

ratel/src/materials/ratel-mpm.c:3467

    // Apply composite operator
    RatelCallCeed(ratel, CeedOperatorApply(op_state, CEED_VECTOR_NONE, state_ceed, CEED_REQUEST_IMMEDIATE));

[0]PETSC ERROR: #1 RatelMPMAcceptState() at /home/zatkins/project/micromorph/ratel/src/materials/ratel-mpm.c:3467
[0]PETSC ERROR: #2 RatelTSPostEvaluate() at /home/zatkins/project/micromorph/ratel/src/internal/ratel-petsc-ops.c:594
[0]PETSC ERROR: #3 TSPostEvaluate() at /home/zatkins/project/micromorph/petsc/src/ts/interface/ts.c:3250
[0]PETSC ERROR: #4 TSSolve() at /home/zatkins/project/micromorph/petsc/src/ts/interface/ts.c:4085
[0]PETSC ERROR: #5 main() at /home/zatkins/project/micromorph/ratel/examples/ex02-quasistatic.c:71

And is there a reason we can't veccopy?

You know, I thought there was, but now I'm not so sure. I'll try swapping over to copying the CeedVector

Okay, so the issue is that we need to use CeedVectorCopyStrided, as even with CEED_COPY_VALUES, CeedVectorCopy will get rid of the borrowed array and allocate a new owned array.

huh, that shouldn't be happening

I figured as much -- that's how the logic is baked in for some reason. I also think there may be some weird host/device sync issues related to copying, but I can't say for sure what they are.

Ack, we're using CeedVectorSetArray internally which drops the old array. Probably we need a new method on CeedVectors that is vec->CopyArray(vec, mem_type, array). That method would need to be set for GPU backends, and if its not present, then we can fallback to memcpy in CeedVectorCopy().

I can whip it up tomorrow morning pretty quick unless you'd like to take a crack at it.

I probably won't get to it today or tomorrow, so go ahead! Thanks

I'm not really following the need for vec->CopyArray. Where would that memory come from? Is there a reason not to wrap it in a CeedVector and then CeedVectorCopy instead of introducing a new way to use a raw pointer (without specifying its length, but tacitly promising that it's long enough)?

That's exactly what we should do, but the internal logic on how to do that throws away the wrapped array when it should not

"throws away" something that is not array_owned?

The user facing interface is correct - my impl is bad. I am fixing it