I am comparing a sensitivity analysis using both the Morris and Sobol method for the most part the methods agree.
However 2 parameters appear to have low influence according to the Sobol method. But, when using the Morris method it appears these parameters have a high mean and a high variance.
How could this be possible? 
Any help appreciated, cheers
Because Morris’ method is quite inaccurate.
Hi @xztt,
[This is a kind of “off-topic” to your discussion] [Disclaimer, this is a new topic for me, so pls be gentle].
May I ask, is there a Julia package that you are using to prepare this global sensitivity analysis or is this your own raw code for Morris and Sobol methods?
What is my use case [Again, I’d like to underline that this is a new topic for me and I might not be quite precise here.]:
I have a deterministic projection consisting of several time frames in “a form of” moving dynamical systems expressed in u and v vectors that represent speed and direction;
I have several corresponding ensemble projections, between 30 to 100, prepared mostly with singular vectors technique also consisting of several time frames that also include moving dynamical systems expressed in u and v vectors representing speed and direction;
I have a moving particle from point A to point B that moves on deterministic data that I mentioned earlier, with performance expressed in a form of a polar diagram and depending on speed and direction of above mentioned moving systems;
What I would like to do is to somehow visualize sensitivity of this particle that is moving on a deterministic system wrt scenarios expressed in a form of above mentioned ensembles and potentially to quantify uncertainty, I mean quantify in a sense, I do not quite know, I guess in a sense of time and position of my particle and of those dynamical systems and their properties.
Would you have any comments pls? Any technique / package recommendation?
EDIT: I should probably add that those u an v vectors as I understand it currently are expressed on a Cartesian grid.
I’ve been comparing Morris and eFAST lately (eFAST computes sobol indices in a more efficient way) and I’ve found that like you say most parameters are ranked roughly the same but a few parameters are completely the opposite. If you have the computing capacity to calculate the Sobol indices until they stabilize then I would suggest going with those and ignoring the Morris results. I also found that Morris is fairly inconsistent from one run to the next, even with a lot of samples. eFAST gives the same thing everytime if N is high enough.
Hi there,
Firstly I think the main packages you will need here are GlobalSensitivity.jl: Global Sensitivity Analysis (GSA) · GlobalSensitivity.jl this package is for global sensitivity analysis. So from what you have said above I don’t know if this will concern you as GSA is concerned with seeing how the inputs affect the outputs. i.e the input parameters of your dynamical system.
However another package to see the uncertainty in the outputs is Uncertainty Quantification · DifferentialEquations.jl
Where here this allows you to see the uncertainty of the outputs across multiple different ensembles.
Hope this helps
It’s very complete and supports a lot of things like batching for GPU parallelism. Tutorial and paper coming out by JuliaCon too 
I need to complete the redo of that package, I have a deadline of like 2 weeks so it has to come soon though 
Will be very interested in that paper I assume there will be some comparisons to the standard SAlib and the countless toolboxes that exist in Matlab I can only postulate what the differences will be .
I am currently writing up a paper now so would be good to include a paper such as this
Maybe … I have misunderstood there something … not sure about it, I mean I dont think so … however, a complete rewrite in … just two weeks … I don’t know. We shell see …
Satielli gave a really nice talk at the ISoP QSP SIG that describes exactly why this occurs. It’s just a fact of the Morris method. It’s quick but unreliable. There are some diagnostics you could use to figure out when it might be reliable, but in general there’s a probability that it can be junk. What it computes is just not rigorously defined like Sobol indices are 
.
Thank you @xztt.
I’m recalling I’ve been reading a bit about Sobol and Morris methods some time ago, however, I’m usually missing the drawings [seriously in most of those books I am reading I usually always missing the notation :- )]. It was some time ago thus I’m just wondering, are Sobol and Morris methods in the same brackets as singular vectors, Kalman filter (derivatives), bred-vectors or i.e. empirical orthogonal function or are these terms closer to the Koopman operator?
So from what you have said above I don’t know if this will concern you as GSA is concerned with seeing how the inputs affect the outputs. i.e the input parameters of your dynamical system.
Yeah, I’m wondering about similar thing.
However another package to see the uncertainty in the outputs is Uncertainty Quantification · DifferentialEquations.jl
But how can I do it with this package?
What I managed to do so far is:
a) to visualize this uncertainty in a form of some strings on a lattice (which allows me to (usually) notice some clusters by my bare eye [however, I still don’t quite fully know how to computationally cluster those string in a spatial context (for this I was hoping to ping @juliohm in the near future with a hope of some advice) and also I have to admit that I still hope to mark some readings on those strings that would indicate the properties of the dynamical systems between which my particle is moving so I could better understand it but still with so many ensemble members it’s not that easy],
b) to visualize decomposition of those ensembles, particularly to visualize in a spatial and time context relationships between the deterministic prediction and each perturbed member of the ensemble but not only this,
d) to visualize in a spatial and time context relationship between updated (new, the most up to date, near real time data) with each perturbed ensemble member so I could maybe get some indication about the next deterministic prediction run.
Would you maybe have any additional suggestions?
I am currently writing up a paper now so would be good to include a paper such as this
Wow, it sounds super interesting, is there maybe any chance to take a look at a draft of your piece?
Not close.
This is not related to this thread at all, you may want to start a new one.
You’ll surely be disappointed. We’re just doing a JOSS submission because there’s nothing particularly special about the package, no new methods or anything, it’s just a well-tested, fast, and easy to parallelize version of the methods.
Comparisons don’t make sense for the GSA package because all of the time is spent in the f evaluations, so of course we’d beat SALib and MATLAB, but that benchmark would just be a benchmark of the ODE solvers and parallelism overhead in any realistic example. So yes, it will be like 100x-1000x faster for cases people use it for, but the reason wouldn’t be GlobalSensitivity.jl per say, rather the f is faster because it’s using some other SciML package (or it’s using the batching interface to implement parallelism, something that is quite unique but is just a “this is the right way to implement it” kind of thing)
I second what Chris has said I think it would be best for you to start a new thread dedicated to your question above.
But I think I would first look at what the problem is you are trying to address whether it is sensitivity analysis or uncertainty quantification. If you search the web for these there will be loads of resources which should make the difference clear from Kalman filter etc.
Just wanted to add that I had nothing personal on my mind, nor absolutely any remarks to your very interesting package. Just pure curiosity. Hard to understand situation @ChrisRackauckas.
Yes no worries and no ill-will. I think it’s just a misunderstanding that sensitivity analysis and uncertainty quantification are closely related.
Well, I’m not an expert in this field (of course), however, even judging only by the book covers one can get such an understanding, so, I’d say that such a statement wouldn’t be totally unjustified.
Do folks in this thread have good sources for non-experts to learn more about GSA and the different methods and interpreting results?
This book here is probably the best place to start if you are brand new to the subject. The from that you can find countless review papers with applications to different fields online.