This tells me somehow during the cmdstan build step it pulled in (or tried to) the TBB stuff. TBB is a threads library from Intel that definitely can help with performance but I thought it was disabled by default in the cmdstan makefiles. But maybe not on Linux. I’m assuming you didn’t enable them in make/local?
Not sure how to debug this issue without the output of the make build step. To capture that output, do:
make clean-all
make build
Rob
The weird thing is it only seems to be pulling that in when I call it through julia. I had no problem making and executing the bernoulli example through the terminal.
$ make examples/bernoulli/bernoulli
$ ./examples/bernoulli/bernoulli sample data file=examples/bernoulli/bernoulli.data.R
I try to find a convenient way to capture the output and get back to you!
Good observation. It happens during compilation. I’m a bit surprised by the .linuxbrew loader that shows up in the error message, but I don’t know Linux that well. Also I wonder if the make build output shows the .linuxbrew ld.
One thing we could try is to disable C++ threads altogether, i.e. change the make/local file to set STAN_THREADS=false (or remove the file) and see if the compilation then succeeds.
In the sample phase we would have to update rc = stan_sample(sm, data=observed_data) to rc = stan_sample(sm, use_cpp_chains=false, data=observed_data). That’s not a long term solution, but might help for what you need.
So I did some digging this morning to see if I could fin the issue with the .linuxbrew and I found a few interesting things. first of all the version of gcc and g++ I was using was pretty old (version 5.5).
I updated those and can no pick which versions of the compiler I want to use, using sudo update-alternatives --config gcc. It gives me a nice list like follows:
Selection Path Priority Status
------------------------------------------------------------
* 0 /usr/bin/gcc-9 9 auto mode
1 /usr/bin/gcc-7 7 manual mode
2 /usr/bin/gcc-8 8 manual mode
3 /usr/bin/gcc-9 9 manual mode
Press <enter> to keep the current choice[*], or type selection number: 0
The interesting issue is that when I select g++ to use , all goes as expected and when i try to check the g++ version I get:
(base) charper@barrel:~/test$ g++ --version
g++ (Ubuntu 9.4.0-1ubuntu1~18.04) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
HOWEVER, regardless of what I select for gcc I get the following:
(base) charper@barrel:~/test$ gcc --version
bash: /home/linuxbrew/.linuxbrew/bin/gcc: No such file or directory
So maybe something is weird with how my system is pointing to gcc? We see this .linuxbrewer show up in the error thrown when I try to run the julia script?
I got it to work! I just manually sett my gcc with update-alternatives --set gcc "/usr/bin/gcc-9" and it worked perfectly! I guess that last issue was with my version and location of gcc? Thanks so much for all the help and I hope you have a good weekend!
Thanks for sticking with it Chris. Have a great weekend.
Hello,
I am having the same issue. The difference in my case is that I am using a Mac M1.
Details:
Mac M1
OS: macOS Monterey 12.0.1
CmdStan 2.29
Julia 1.7
StanSample 6.2.3
Any suggestions to fix it would be appreciated.
Not sure if it matters, but here is the stacktrace I get (just in case anything is different):
/var/folders/r1/t2s9jjkj4k56jy6_29_s3svm0000gn/T/jl_K5WQCR/test_model.stan updated.
ERROR: IOError: cd(""): no such file or directory (ENOENT)
Stacktrace:
[1] uv_error
@ ./libuv.jl:97 [inlined]
[2] cd(dir::String)
@ Base.Filesystem ./file.jl:89
[3] cd(f::StanSample.var"#25#26"{IOBuffer, String, String}, dir::String)
@ Base.Filesystem ./file.jl:109
[4] SampleModel(name::String, model::String, tmpdir::String)
@ StanSample ~/.julia/packages/StanSample/gacEa/src/stanmodel/SampleModel.jl:110
[5] SampleModel(name::String, model::String)
@ StanSample ~/.julia/packages/StanSample/gacEa/src/stanmodel/SampleModel.jl:101
[6] top-level scope
@ REPL[6]:1
The error message tells us that there probably is an issue compiling the Stan program.
You can get a more informative error message by running CmdStan by hand. To do this, navigate to your CmdStan directory (the one where your CMDSTAN environment variable points to, and type make examples/bernoulli/bernoulli (or the path to any other .stan file).
As Daniel suggested, this usually indicates cmdstan-2.29.0 is not properly installed. Running the default example in the CMDSTAN directory might provide pointers. Would you also mind sending me your Julia script and the test_model.stan? A lot has changed with the support for C++ threads in StanSample.jl v9, hence my question.
Do you run Julia in ARM mode or under Rosetta?
Thanks for the answers. I should have clarified that I have tried most of the steps listed above, including providing values for CMDSTAN_HOME. No idea how to update gcc on a Mac (as charperflow did) as my Google searches have not been helpful in this area.
Before posting, I ran the bernoulli example in CmdStan on its own, so CmdStan should be installed correctly.
I am running julia in ARM mode, although I get the same message when I use Julia 1.6.5, which runs under Rosetta.
Apologies for failing to mention some of this before.
Do you have a make/local file in the CMDSTAN folder?
Something like:
# To use this template, make a copy from make/local.example to make/local and uncomment options as needed.
# Be sure to run `make clean-all` before compiling a model to make sure everything gets rebuilt.
# Change the C++ compiler
CXX=clang++
# Enable threading
STAN_THREADS=true
# Enable the MPI backend (requires also setting (replace gcc with clang on Mac)
# STAN_MPI=true
# CXX=mpicxx
# TBB_CXX_TYPE=gcc
Just for completeness, these are the steps I use on my M1 to install cmdstan:
git clone https://github.com/stan-dev/cmdstan.git --recursive cmdstan
cd cmdstan
# create a ./make/local file as listed above in this Discourse post
make -j9 build # or make clean-all
# 9 is the number of threads used for compiling cmdstan
# Test the Bernoulli example
make examples/bernoulli/bernoulli
# Run the example
./examples/bernoulli/bernoulli num_threads=6 sample num_chains=4 data
file=examples/bernoulli/bernoulli.data.json
bin/stansummary output_1.csv
Once Steve gets it up and running I’ll add that to the StanSample.jl and Stan.jl README docs.
Daniel, I did see your post on Conda install and worked through that a while ago (for someone on Windows). I haven’t had time to work through that for C++ threads.
So after having some issues, I removed and reinstalled CmdStan using your instructions above. Here is the code I used in the terminal in case it helps updating the README.
git clone https://github.com/stan-dev/cmdstan.git --recursive cmdstan
cd cmdstan
ls -lia ./make/local
#ls: ./make/local: No such file or directory
ls -lia ./make/local.example
cp -R ./make/local.example ./make/local
ls -lia ./make/local
touch ./make/local
echo "STAN_THREADS=true" > ./make/local
make -j4 build
# run model
#cd cmdstan
make examples/bernoulli/bernoulli
./examples/bernoulli/bernoulli num_threads=4 sample num_chains=4 data file=examples/bernoulli/bernoulli.data.json
ls -l output_1.csv
bin/stansummary output_1.csv
And with that, it now works with StanSample. Thank you for the help!
Thanks for the update Steve and glad it works. Will use a (slightly) simplified version of this.
Looking at above steps in the terminal I believe the echo ... step creates a new ./make/local file (thus the previous cp ... is undone) with only a single line in it (STAN_THREADS=true). Thus you have compiled using gcc, not clang++ I guess.
If you ever would like to switch to clang++ you can use:
echo "CXX=clang++\nSTAN_THREADS=true" > ./make/local
but either compiler toolset works.