Issues implementing a simple Stan example

This tells me somehow during the cmdstan build step it pulled in (or tried to) the TBB stuff. TBB is a threads library from Intel that definitely can help with performance but I thought it was disabled by default in the cmdstan makefiles. But maybe not on Linux. I’m assuming you didn’t enable them in make/local?

Not sure how to debug this issue without the output of the make build step. To capture that output, do:

make clean-all
make build

Rob

The weird thing is it only seems to be pulling that in when I call it through julia. I had no problem making and executing the bernoulli example through the terminal.

$ make examples/bernoulli/bernoulli
$ ./examples/bernoulli/bernoulli sample data file=examples/bernoulli/bernoulli.data.R

I try to find a convenient way to capture the output and get back to you!

Good observation. It happens during compilation. I’m a bit surprised by the .linuxbrew loader that shows up in the error message, but I don’t know Linux that well. Also I wonder if the make build output shows the .linuxbrew ld.

One thing we could try is to disable C++ threads altogether, i.e. change the make/local file to set STAN_THREADS=false (or remove the file) and see if the compilation then succeeds.

In the sample phase we would have to update rc = stan_sample(sm, data=observed_data) to rc = stan_sample(sm, use_cpp_chains=false, data=observed_data). That’s not a long term solution, but might help for what you need.

So I did some digging this morning to see if I could fin the issue with the .linuxbrew and I found a few interesting things. first of all the version of gcc and g++ I was using was pretty old (version 5.5).

I updated those and can no pick which versions of the compiler I want to use, using sudo update-alternatives --config gcc. It gives me a nice list like follows:

  Selection    Path            Priority   Status
------------------------------------------------------------
* 0            /usr/bin/gcc-9   9         auto mode
  1            /usr/bin/gcc-7   7         manual mode
  2            /usr/bin/gcc-8   8         manual mode
  3            /usr/bin/gcc-9   9         manual mode

Press <enter> to keep the current choice[*], or type selection number: 0

The interesting issue is that when I select g++ to use , all goes as expected and when i try to check the g++ version I get:

(base) charper@barrel:~/test$ g++ --version
g++ (Ubuntu 9.4.0-1ubuntu1~18.04) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

HOWEVER, regardless of what I select for gcc I get the following:

(base) charper@barrel:~/test$ gcc --version
bash: /home/linuxbrew/.linuxbrew/bin/gcc: No such file or directory

So maybe something is weird with how my system is pointing to gcc? We see this .linuxbrewer show up in the error thrown when I try to run the julia script?

I got it to work! I just manually sett my gcc with update-alternatives --set gcc "/usr/bin/gcc-9" and it worked perfectly! I guess that last issue was with my version and location of gcc? Thanks so much for all the help and I hope you have a good weekend!

Thanks for sticking with it Chris. Have a great weekend.

Hello,

I am having the same issue. The difference in my case is that I am using a Mac M1.

Details:
Mac M1
OS: macOS Monterey 12.0.1
CmdStan 2.29
Julia 1.7
StanSample 6.2.3

Any suggestions to fix it would be appreciated.

Not sure if it matters, but here is the stacktrace I get (just in case anything is different):

/var/folders/r1/t2s9jjkj4k56jy6_29_s3svm0000gn/T/jl_K5WQCR/test_model.stan updated.
ERROR: IOError: cd(""): no such file or directory (ENOENT)
Stacktrace:
 [1] uv_error
   @ ./libuv.jl:97 [inlined]
 [2] cd(dir::String)
   @ Base.Filesystem ./file.jl:89
 [3] cd(f::StanSample.var"#25#26"{IOBuffer, String, String}, dir::String)
   @ Base.Filesystem ./file.jl:109
 [4] SampleModel(name::String, model::String, tmpdir::String)
   @ StanSample ~/.julia/packages/StanSample/gacEa/src/stanmodel/SampleModel.jl:110
 [5] SampleModel(name::String, model::String)
   @ StanSample ~/.julia/packages/StanSample/gacEa/src/stanmodel/SampleModel.jl:101
 [6] top-level scope
   @ REPL[6]:1

The error message tells us that there probably is an issue compiling the Stan program.
You can get a more informative error message by running CmdStan by hand. To do this, navigate to your CmdStan directory (the one where your CMDSTAN environment variable points to, and type make examples/bernoulli/bernoulli (or the path to any other .stan file).

As Daniel suggested, this usually indicates cmdstan-2.29.0 is not properly installed. Running the default example in the CMDSTAN directory might provide pointers. Would you also mind sending me your Julia script and the test_model.stan? A lot has changed with the support for C++ threads in StanSample.jl v9, hence my question.

Do you run Julia in ARM mode or under Rosetta?

Thanks for the answers. I should have clarified that I have tried most of the steps listed above, including providing values for CMDSTAN_HOME. No idea how to update gcc on a Mac (as charperflow did) as my Google searches have not been helpful in this area.

Before posting, I ran the bernoulli example in CmdStan on its own, so CmdStan should be installed correctly.

I am running julia in ARM mode, although I get the same message when I use Julia 1.6.5, which runs under Rosetta.

Apologies for failing to mention some of this before.

Do you have a make/local file in the CMDSTAN folder?

Something like:

# To use this template, make a copy from make/local.example to make/local and uncomment options as needed.
# Be sure to run `make clean-all` before compiling a model to make sure everything gets rebuilt.

# Change the C++ compiler
CXX=clang++

# Enable threading
STAN_THREADS=true

# Enable the MPI backend (requires also setting (replace gcc with clang on Mac)
# STAN_MPI=true
# CXX=mpicxx
# TBB_CXX_TYPE=gcc

Just for completeness, these are the steps I use on my M1 to install cmdstan:

git clone https://github.com/stan-dev/cmdstan.git --recursive cmdstan
cd cmdstan
# create a ./make/local file as listed above in this Discourse post
make -j9 build   # or make clean-all
  # 9 is the number of threads used for compiling cmdstan

# Test the Bernoulli example

make examples/bernoulli/bernoulli

# Run the example

./examples/bernoulli/bernoulli num_threads=6 sample num_chains=4 data
        file=examples/bernoulli/bernoulli.data.json

bin/stansummary output_1.csv

Once Steve gets it up and running I’ll add that to the StanSample.jl and Stan.jl README docs.

Daniel, I did see your post on Conda install and worked through that a while ago (for someone on Windows). I haven’t had time to work through that for C++ threads.

1 Like

So after having some issues, I removed and reinstalled CmdStan using your instructions above. Here is the code I used in the terminal in case it helps updating the README.

git clone https://github.com/stan-dev/cmdstan.git --recursive cmdstan
cd cmdstan
ls -lia ./make/local
#ls: ./make/local: No such file or directory
ls -lia ./make/local.example
cp -R ./make/local.example ./make/local
ls -lia ./make/local
touch ./make/local 
echo "STAN_THREADS=true" > ./make/local         
make -j4 build 


# run model
#cd cmdstan
make examples/bernoulli/bernoulli
./examples/bernoulli/bernoulli num_threads=4 sample num_chains=4 data file=examples/bernoulli/bernoulli.data.json

ls -l output_1.csv
bin/stansummary output_1.csv

And with that, it now works with StanSample. Thank you for the help!

Thanks for the update Steve and glad it works. Will use a (slightly) simplified version of this.

Looking at above steps in the terminal I believe the echo ... step creates a new ./make/local file (thus the previous cp ... is undone) with only a single line in it (STAN_THREADS=true). Thus you have compiled using gcc, not clang++ I guess.

If you ever would like to switch to clang++ you can use:

echo "CXX=clang++\nSTAN_THREADS=true" > ./make/local

but either compiler toolset works.

1 Like