Thanks for the reference! It is nice that this problem is already solved by a package. However, I feel that adding a package dependency seems a bit much for such a simple task, especially if it is only needed a single time.
The following example uses the Base function setdiff, which I found to be an improvement on my initial attempt:
julia> set2 = myvector[setdiff(eachindex(myvector), inds)]
7-element Vector{Int64}:
1
3
4
7
8
9
10