Conversion of CxxWrap.StdLib.StdVectorAllocated{UInt8} to Julian Vector{UInt8} faster

Hey guys, issue I am facing is the conversion of the arrays and the efficiency of the conversion

julia> res = screenshot_rgb() # image of size 1920*1080 with 4 bytes per pixel returned
8294400-element CxxWrap.StdLib.StdVectorAllocated{UInt8}:
 0x1d # B
 0x1d # G
 0x1d # R
 0x00 # A
 0x1d # B
    ⋮ # 
 0xcb # B
 0x5a # G
 0x23 # R
 0x00 # A

### below methods take something like 500ms which make whole function
### very very slow for my usecase, they all do the same thing
 res = convert(Array, res)
 res = collect(res)
 res = res[:]
 res = res .* 0x01

To elaborate, my function wants to convert c++ 4bytes per pixel vector uint8 style image data(which is easy and fast to convert to BGRA format) to julian image(RGB and Gray) fast, problematic part for me is the alpha channel data and the format of data xgetimage provides. So to tackle that I wanted to do some operations like a deleteat(to remove alpha channel) but it doesn’t work on StdVector and needs a julian vector

function screenshot_rgb(Width= 1920, Height= 1080; originx= 0, originy= 0)
    res = GrabScreen.screenshot(Width, Height, originx, originy); # 14ms
    res = collect(res) # THIS IS EMBARRASINGLY SLOW 
    deleteat!(res, 4:4:length(res)) #46.541ns
    res = reverse(res) # 7 ms # these reverses also take time, would try remove them

    res = reinterpret(RGB{N0f8}, res) #144ns
    res = reshape(res , Width, Height) #24ns
    res = reverse(res') #9ms
end

Also, I am not too fixated on converting it to julian vector, so to handle these teeny tiny format issues, I can do the processing back in c++ only to provide it in better format.
I would need to think of the methods I used if that conversion is faster(then might reinterpret to BGRA and use RGB.()(which is not that slow), so yeah any help is much appreciated

Do you have an MWE to create the 8294400-element CxxWrap.StdLib.StdVectorAllocated{UInt8}
even in C++ I can compile ? (on Linux)

Hey @lawless-m , I have been able to fix the performance issue by changing the format in c++ only(commit link)according to need but I am still interested in learning a faster method of conversion with collect or something similiar. The MWE to get something like me would be:

git clone https://github.com/ashwani-rathee/GrabScreen.jl
cd GrabScreen.jl
code .
# after that in repl
]activate .
using Revise
using GrabScreen
GrabScreen.screenshot(1920,1080, 0,0,0) # width, height, originx, originy, typeofdata(0->bgra,1->rgb,2->gray) 
# This will throw the c++ array

This will have to wait until later.
I was so close, I’m remote atm. and X is fussy about security :slight_smile:

$ DISPLAY=:0 julia --project=.
julia> using GrabScreen                                                                                                                                                                                                                                           
julia> GrabScreen.screenshot(1920,1080, 0,0,0) 
Invalid MIT-MAGIC-COOKIE-1 key                                                                                                   
signal (11): Segmentation fault                                                                                                  
in expression starting at REPL[3]:1

I am relatively new to X, so I didn’t think there was something like a key that would be specific to a computer. This is an issue that needs a fix(idk how though yet), thanks for finding it @lawless-m

Well, if it’s bug reports you want :slight_smile:
It also aborts if you try and use it on a VNC connection

julia> GrabScreen.screenshot(800,600,0,0,0)
julia: /home/ashwani/OpenCV.jl/libcxxwrap-julia/include/jlcxx/type_conversion.hpp:640: jlcxx::BoxedValue<T> jlcxx::boxed_cpp_pointer(T*, jl_datatype_t*, bool) [with T = std::vector<unsigned char>; jl_datatype_t = _jl_datatype_t]: Assertion `jl_is_concrete_type((jl_value_t*)dt)' failed.

signal (6): Aborted
in expression starting at none:1
gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
1 Like

and in VSCode ssh-Terminal to the remote Linux machine slightly different

julia> GrabScreen.screenshot(1920,1080, 0,0,0)

signal (11): Segmentation fault
in expression starting at REPL[2]:1
_Z10screenshotiiiii at ./src/libgrabscreen.so (unknown line)
1 Like

The MIT magic cookie is to stop people grabbing each other’s screens. That X session is the Login screen as owned by root. I don’t know what you’d check for either to make sure there is a screen available at all.

After all, I might be using X-forwarding (although I don’t have a Server here to try it with)

1 Like

I don’t want anyone to be grabbing anyone else’s screens :joy:, I think I need to find a way to kind of compile the shared library for the target system when they try to install GrabScreen.jl for their own system. It might be time to learn about binarybuilers and yggdrasil I believe.