You are not doing it wrong. The way select! is implemented now is that it creates a new DataFrame and then overwrites the old one.
What you ask for is doable as an optimization (i.e. we would need to check that the user does only simple column selection) but it was considered that microseconds are fast enough. Do you have any specific use case where you would need this?
Note though that even non-trivial selections should not be expected to be fast, e.g. select!(df, r"x", Not(:y), Between(:y, :z)) is quite complex to process and will be more expensive than nanoseconds.