Module fails at first run, but works at second run

General Usage
, , ,
1

Hello,

I have a very strange behaviour with my program. When I start it for the first time, then it fails with an error message that is not very helpful. When I start it for the second time, then it completes without any problems. This happens always when I have a new Julia session. Therefore I think that it could be a problem with the compiler or with the scope of variables, the dependencies or other parts of the program.

The program runs always, even at first run, without any problems when I have it as a script and not as a module. I had to transform it to a module because of the PackageCompiler that I use to create an EXE file. However, it is no problem of the PackageCompiler because the problem occurs even when starting my program manually in VSCode.

My program has this structure:

module BlaApp
using Dates
using DataFrames
using Nettle
using ODBC
using Tables
using TOML

import DBInterface as dbi
import SMTPClient as smtp

function program_main()
# Here I have hundreds of lines of code. This is my program which works without any problems as a script.
    
    df_1 = DBInterface.execute(conn_1, sql_query) |> DataFrame   # At this line I get the error message at first run!
    println("LZ: ", nrow(df_1)) # VSCode marks this line with a blue underscore and seems not to recognize the dataframe!
    
    # More lines of code.
end

function julia_main()::Cint  # The main function used to start the program. A must have because of PackageCompiler.
    try 
        program_main()	# This one fails. First run.
    catch
        program_main()	# This one completees withouut any problems.
    end
    return 0
end

end

This is the error message when running the program for the first time:
ERROR:
Stacktrace:
[1] error(s::String)
@ Base .\error.jl:35
[2] iterate(x::ODBC.Cursor{false, false}, st::Int64)
@ ODBC C:\Users\miro.julia\packages\ODBC\9VZTC\src\dbinterface.jl:398
[3] iterate
@ C:\Users\miro.julia\packages\ODBC\9VZTC\src\dbinterface.jl:387 [inlined]
[4] iterate
@ .\iterators.jl:167 [inlined]
[5] iterate
@ .\iterators.jl:166 [inlined]
[6] buildcolumns(schema::Tables.Schema{(:BookingFachlicheID, :Export_id, :ChangedWhen), Tuple{Union{Missing, String}, Int32, Dates.DateTime}}, rowitr::ODBC.Cursor{false, false})
@ Tables C:\Users\miro.julia\packages\Tables\T7rHm\src\fallbacks.jl:139
[7] columns
@ C:\Users\miro.julia\packages\Tables\T7rHm\src\fallbacks.jl:256 [inlined]
[8] DataFrames.DataFrame(x::ODBC.Cursor{false, false}; copycols::Nothing)
@ DataFrames C:\Users\miro.julia\packages\DataFrames\LteEl\src\other\tables.jl:57
[9] DataFrame
@ C:\Users\miro.julia\packages\DataFrames\LteEl\src\other\tables.jl:48 [inlined]
[10] |>(x::ODBC.Cursor{false, false}, f::Type{DataFrames.DataFrame})
@ Base .\operators.jl:911
[11] program_main()
@ Main.BlaApp c:\JuliaWS\Bla\BlaApp\src\BlaApp.jl:133
[12] julia_main()
@ Main.BlaApp c:\JuliaWS\Bla\BlaApp\src\BlaApp.jl:235
[13] top-level scope
@ REPL[1]:1

The error message points to this line:
df_1 = DBInterface.execute(conn_1, sql_query) |> DataFrame

It is the only line in my program that uses this syntax “|> DataFrame” for converting the result of the DBInterface.execute, a cursor, to a DataFrame. But, as I mentioned, it works as a script always and when running the function in the module for the second time without any problems.

I already tried to do import DataFrames as df and referencing by df instead of using DataFrames, but it did not help.

I am using Julia version 1.8.3, DataFrames version 1.5.0 and Windows 10 64-Bit.

What is the problem?

Thanks for any answers in advance!

Kind regards,
Miroslav

2

It is very unlikely that DataFrames.jl causes the issue. Most likely it is query execution. Maybe @quinnj can help?

3

Hello,

I had a similar problem a few days ago because I used an include somewhere in the middle of my code. The program only worked from the second run, because on the first run the file was not yet included.

It may be something along those lines?

1 Like
4

The error you’re seeing is due to an invalid db handle (ODBC.jl/dbinterface.jl at 27e02ab10c3e3632640b7edc31c989a8554ba36a · JuliaDatabases/ODBC.jl · GitHub), but it’s unclear why exactly you’d be seeing different behavior between runs. My guess is that this is specific to the database odbc driver you’re using where they need some kind of warm up or something?

5

Hi quinnj,

It is very strange.

df_1 = DBInterface.execute(conn_1, sql_query) |> DataFrame # At this line I get the error message at first run!

The cursor is created and it contains data!
But the data frame is not created or is not OK. The |> DataFrame seems to be the problem.

I tested it by adding additional lines of code. First I create a dummy cursor, print all its contents via a loop and then create a dummy dataframe. The dataframe is empty, but it should not be empty!
Then I create the second dataframe with the same sql query and it contains data. Now there is no error message anymore! It seems that the creation of the dummy cursor, looping it and creating the dummy dataframe helped to run the code to create the second, correct dataframe.

    dummy_cursor = DBInterface.execute(conn_1, sql_query)
    for dummy_row in dummy_cursor
        println(dummy_row)
    end

# OUTPUT: The cursor contains data and rows are printed. OK!
    
    dummy_df = dummy_cursor |> DataFrame
    println(describe(dummy_df))

# OUTPUT: The result is an empty dataframe! NOT OK!
    
    df_1 = DBInterface.execute(conn_1, sql_query) |> DataFrame
    println(describe(df_1))

# OUTPUT: The result is a dataframe containing data! OK!

I am using ODBC Driver 17 for SQL Server as an ODBC driver for Microsoft SQL Server and ODBC version 1.1.2.

If I do not have the dummy code lines, then the error occurs and I have to start my program for a second time in the same Julia session in order to run it successfully. The dummy code lines solve the problem and no error message occurs, but I do not understand why this is so. The dummy dataframe should not be empty. Therefore I think that something goes wrong when using |> DataFrame for the first time. As I mentioned, it happens only if my source code is a part of a module. There is no problem if it is just a part of a script!