Hello All,
I am starting to play with TensorFlow.jl. I am thinking about how to add l2 regularization into all weights of all layers. Adding that during the definition stage seems to me to be a nuisance. I would like something automated, though I cannot come with a right filter to autimatically select all weight variables in the graph.
So far, my solution is
‘’’
l2reg=mapreduce(w->nn.l2_loss(get_tensor_by_name(w.name)),+,filter(op->contains(op.name,“weight”),map(v->v.var_node,g.collections[:TrainableVariables])))
‘’’
where I use the assumption that each weight to be regularized have weight in the name.
Does anyone have better solution?
Thanks for any suggestions.
Tomas