I agree that focusing on infrastructure first seems like a very practical direction. Beyond training a larger base model, I think it would be interesting to explore what a Julia-native stack for model development and inference could look like in practice. For example, having Julia-native tooling for efficient inference, context management, recursive workflows, and benchmarking could make it much easier to experiment with these newer architectures. It might also be useful to define a few concrete benchmarks so we can compare different approaches and see where Julia can offer real advantages rather than simply reproducing existing workflows.