@jling is right — doing exactly what you ask for is impossible so long as the struct fields are immutable. But what you ask for is the default behavior for mutable fields (such as arrays or a number in a Ref), in which case the struct itself need not be mutable.
I just wanted to add, maybe you will find StructArrays.jl useful. It lets you deal with arrays-of-structs, while internally being a struct-of-arrays. Used in conjunction with Accessors.jl, this can be pretty handy. See this answer.