DataLoaders.jl Workers systematically end up outside bounds

Hi Clark, author here. The issue seems to be a discrepancy in how MLDataPattern.BatchView and DataLoaders.BatchViewCollated interpret nobs. For the latter, every batch is one observation, but the former treats counts every single observation.

For your use case, passing collate = true to the DataLoader call should fix it; this is probably what you want anyway if you’re training on batches. Let me know if you run into any other problems. I’ll raise the inconsistency as an issue.

1 Like