LightHouse.jl Implementation

Specific Domains Visualization
1

Good Day:

II am attempting to create an ROC_CURVE
with Lighthouse.jl. I am attempting to generate
one with the MLJ.@load_boston dataset.

My general questions are:

  1. How do I designate class? names(df)?
    Should I:
classes = ["class $i" for i in 1:5]
  1. What should I use as my curve value?
    I was able to create one using:
curve_clf = MLJ.learning_curve(clf, feat, targ,range=r_clf,
					   resampling=MLJ.Holdout(fraction_train=0.8),
					   measure=MLJ.cross_entropy)

or should I:

curves = [(LinRange(0, 1, 10), range(0, stop=i/2, length=10) .+ (stable_randn(10) .* 0.1)) for i in -1:3]

(Extra: In this case, how would I prefix stable_rand?)

In all I would like to apply Lighthouse.plot_roc_curves). But
am having trouble choosing a starting point.

Does anyone have experience with Lighthouse or is there
some other variety (not from SciKitLearn.jl) that is practical
to implement?

2

There are some gotchas in these analyses. This and this may be helpful in thinking through issues. Iā€™m not sure if Lighthouse incorporated such considerations into their package.

1 Like
3

@George9000 Thank you for sharing!

Might you know of any 2018 or more
recent sources? I have noticed that
these items have about am 18 month
cycle (or less) before some of the
modules are deprecated.