Hello all. I’ve recently taken to removing pointers from some code and investigating whether there is anything which would not be possible to do efficiently in Julia without them.
First exhibit, the behavior of reinterpret. Consider this function
function wrap(::Type{T}, A::Vector) where T
ptr = convert(Ptr{T}, pointer(A))
unsafe_wrap(Array, ptr, length(A))
end
const A = rand(Int64, 10^6)
(yes, I know this isn’t safe because of the length, it’s just for demonstration purposes) then
So, it appears that reinterpret causes an extra allocation that results in it being a bit slow. Any ideas as to why this is the case and what, if anything, can be done to make it as efficient as using a pointer?
I just realized that I may have answered my own question, perhaps the extra allocation has something to do with ensuring that the data sizes are compatible?
I don’t consider it slow, I consider it slower than it possibly could be (only because the pointers are faster). The case in which this type of performance might matter is if performing a huge number of such operations on a large in-memory data buffer.
So I guess I’m just missing something fundamental about what goes on here. My understanding was that both of these operations create Julia arrays that point to the same underlying data. In either case the only thing being allocated is the Julia array (just a reference) that is passed as a return value. I would have thought that there’d be a new Julia array allocated on the heap in both cases, what am I missing?
The correct way of reinterpreting an array, is by using reinterpret. The reinterpreted array keeps the original array safe from being garbage collected for example.
The size of the buffer doesn’t matter so not sure what your point is here. Also why would you do a huge numbers if reinterprets? Presumably you do it because you want to operate on the data…
As a s general remark, there is a recent trend on discourse to quite quickly grab after unsafe methods, pointers, C-style casting of types etc. This is quite unfortunate. It is not how you program in Julia and by adopting that kind of workflow one is throwing away many of the things that make julia nice to program in.
I didn’t think there were rules telling people just how they should program in Julia.
There used to be a (poorly named) philosophy that Julia programmers should be considered responsible adults, and not be prevented from doing what they needed to do.
I imagine that in many cases the allocation for the wrapper will be elided if it doesn’t escape — and someday there will be a general solution there, too.
What are you trying to reinterpret? Going between Vectors of static vectors to a matrix is the common case I am thinking about, that any operation on the matrix is huge compared to 3.5ns.
Legacy, special-case, temporary, implementation details. It’s not uncommon for languages implementations to need to include small amounts of code that violates recommended practice in order to provide a stable, public API. So, yeah, it happens, but it usually doesn’t belong on a mailing list. You have to know your audience. The group of people who need to work on these internal features is small. Most users shouldn’t need to know about these details: there’s an enormous difference between being permitted to do unsafe operations, and providing general guidance on how to use the language well. </rant>
Note too that we don’t do anything like this for Arrays, just for the very limited case of Strings. The standard library makes a strong and clear distinction between the two (whenever possible). Benchmarks like ExpandingMan and mbauman are useful for confirming that the expected optimizations are being done on the preferred-style code which avoids unsafe functions.
In this case, ExpandingMan started off stating that he was already using pointers.
I’ve also needed to use them, as nothing else has been fast enough for what I’ve been doing.
(My other option would be to simply stop using Julia, which I’d really hate to do!)
If at some wonderful future, the compiler is made smart enough that there is zero difference in performance,
not 10%, 44%, 2x, etc., then I’d start using “safer” alternatives.
So, to make things perfectly clear, for people who haven’t been dealing with low-level programming with pointers in C/C++/assembly language, they probably should not be messing around with them in Julia either
(if you have, then it’s great that the capability is there, to really solve the “two-language” problem).
ExpandingMan started by stating that he was trying to rid his code of uses of pointer. Not the same.
This is the promise of C++ (“guaranteed zero cost abstractions”). It’s here, available now, if that is your goal. It is not a goal of Julia. As mbauman’s benchmarks are intended to show, this can lead to optimizing the wrong part of the program for a problem that does not exist.
And if so, they hopefully should have quickly learned what a terrible idea it is to mess with them and why assembly languages have completely lost to C and why there are many modern attempts now to eliminate C in turn, such as, for different target audiences, Rust, Julia, Swift, Javascript, etc., and each with different allocation schemes. But all start from a common premise that pointers should always be managed references.
To be clear, I’m not claiming that the use of pointers in my code was appropriate. In fact, I think it was probably inappropriate, but my starting point was some existing code that made extensive use of them.
I agree. The other reason I was using pointers in the first place (apart from basing what I was doing on some existing code) was ignorance on my part about when exactly data gets copied in Julia. I was extremely paranoid that if I was not extremely careful, at some point some huge data buffer would get copied as an unfortunate side effect of some other code. I’ve done some testing and reading and I think I’m a bit less ignorant about this now. I don’t think it was ever a real problem in the first place.
Yes, I shouldn’t have mentioned the size of the buffer, that was clearly silly. The huge number of reinterprets is because potentially lots of computations would be done with this data, which shouldn’t necessarily involve it getting copied. For instance, if I have two vectors x::Vector{Float64} and y::Vector{Float64} both of which come from some underlying data buffer B::Vector{UInt8}, I don’t want to have to allocate two new arrays for x and y to do x .+ y. This is for a Julia implementation of Apache Arrow for which it is very hard to predict use cases, so I don’t want to bake some fundamental flaw or performance limitation into my code. I’m not going crazy optimizing it yet, but I want to make sure that optimizing it is at least possible. Looking into this stuff is part of the process, and it part of that has been a learning process for me (paranoia is often my downfall).
The good news is, it’s looking more and more to me like there is no need for pointers in any of this code. That is impressive for Julia’s sake!
As for the more general discussion of deprecating pointer, I at least partly agree with @ScottPJones sentiment in that I’d be extremely nervous that there would be some things which could not, even in principle, be done efficiently in Julia without them. In my opinion there ought to be some extremely solid reason for believing that is not the case before deprecating them. However, as should be apparent from the above discussion, I am certainly not qualified to make this determination, and at present I’m not able to come up with a specific example that demonstrates the necessity of pointer (which seems like a good thing).
Wow, ok I was starting to think this was all solved an I could just use reinterpret, but now I am confused. I guess I will have to do more testing with more conversions.