# TileInfer.jl: A tiny vLLM built to play with cuTile.jl

**URL:** <https://discourse.julialang.org/t/tileinfer-jl-a-tiny-vllm-built-to-play-with-cutile-jl/142856>\
**Category:** GPU\
**Tags:** performance, cuda, project\
**Created:** [November 10, 2026, 12:35pm UTC](https://discourse.julialang.org/t/tileinfer-jl-a-tiny-vllm-built-to-play-with-cutile-jl/142856 "2026-11-10T12:35:49Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![AntoineBut](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/antoinebut/32/213352_2.png) [@AntoineBut](https://discourse.julialang.org/u/AntoineBut)\
**Post date:** [November 10, 2026, 12:35pm UTC](https://discourse.julialang.org/t/tileinfer-jl-a-tiny-vllm-built-to-play-with-cutile-jl/142856/1 "2026-11-10T12:35:49Z")

</div>

Hey everyone,

The last few weeks, I’ve been building my own LLM inference engine because I wanted to try working with Tile-based GPU programming. And since we now have [cuTile.jl](https://github.com/JuliaGPU/cuTile.jl) in the JuliaGPU ecosystem, I did it in Julia.

I’ve been learning a lot of stuff, and I thought it would be a waste to keep it to myself so I wrote a _technical-report-style_ kind of [blog post](https://antoinebut.github.io/blog/tileinfer/) to explain the project, feel free to check it out if you ever wondered how this things works under the hood!

And if you’re extra curious, and already know a fair bit on GPUs and Transformers, I’ve explained the 3 _most interesting parts_\* in more detail in the documentation ([KV-Cache](https://antoinebut.github.io/TileInfer.jl/dev/kv_cache/), [Attention Kernels](https://antoinebut.github.io/TileInfer.jl/dev/attention/), [Speculative Decoding](https://antoinebut.github.io/TileInfer.jl/dev/speculative/)).

As mentioned at the beginning of the blog post, I’ve mostly made this for myself because I wanted to. I don’t plan on registering it, especially since cuTile.jl is still in beta, but if you somehow have an actual use case for this, don’t hesitate to get in touch!

Blog post: [Building an LLM inference engine in Julia to try out cuTile.jl | Antoine Buttier](https://antoinebut.github.io/blog/tileinfer/)  
Repo: [GitHub - AntoineBut/TileInfer.jl: A single-GPU LLM inference engine in Julia, built for learning purposes and for fun :D · GitHub](https://github.com/AntoineBut/TileInfer.jl)  
\*_The metric of interestingness is purely subjective_