Hypernetworks are an increasingly popular tool for several applications, including continual learning, generative modeling etc. Naively, I’d think it would be relatively easy to create a simple interface to create an arbitrary primary network using Julia. Flux.destructure already provides the ability to reconstruct a network based off of a vector of parameters, but this doesn’t work for hypernetworks off-the-shelf since it doesn’t know to “expand” the weights by the batch size when reconstructing the network. i.e., if you’re creating an instance of your network for each batch of the weight matrix generated by your hypernetwork.
Anyone want to help me write such an interface?