Hi there, I am wondering how I could use langchain from Python as part of a webserver.
I am running into segmenation faults certainly due to multithreading. Does someone know a way whether the new @unlock and @lock are enough for this use case? Or does someone know a good way to spawn threads on thread one (because that is where PythonCall locks the GIL by default)?
Maybe you are using different terms than I am used to but I don’t think you can spawn thread 1. Thread 1 is the thread your program starts and ends with. I assume you mean like queueing a task on thread 1. I know that is common in swift and objective C but I’ve never seen that elsewhere. Usually not necessary.
Lock and unlock does not change which thread you are executing on. It prevents different threads from accessing the same data at the same time.
i would consider avoiding threads if it causes this extra complexity. Python web servers usually have no threading but use async instead. Julia has async too.
At least don’t lose track of the main thread which the program started in.
the thing is that the webserver will use all available threads to distribute the load, so that if I have a PythonCall call inside the web response, it may be run from another thread than 1
It actually seems to work with unlock and lock for now…