Introduction & Audience
Hello everyone! I’d like to open this thread to gather real-world feedback on the RAGTools experimental submodule currently included in PromptingTools.jl. Specifically, this is for people who have actually used or experimented with RAGTools in their workflows. If you have experience integrating, customizing, or maintaining RAGTools in your projects, we’d love to hear from you. (Of course, if you’re simply curious, you’re more than welcome to chime in, too!)
What Are We Looking For?
- Your Use Cases: How are you using RAGTools now? Which features do you rely on the most?
- Pain Points: Anything that feels cumbersome or overly complex?
- Feature Requests: Anything you desperately need that’s missing? Or changes that would smooth out your workflow?
- Stability Concerns: Let us know if there’s something we shouldn’t break to avoid major disruption in your projects.
Ultimately, the plan is to separate RAGTools into a separate package under the JuliaGenAI organization for better governance and continuity. Before we make that transition, we want to hear what matters most to you. Also, if you have any specific views on how this transition should happen, please share!
Background
RAGTools has been part of PromptingTools.jl for some time. While it has remained stable, it’s grown into an area that many people rely on for retrieval-augmented generation (RAG) workflows. We’d like to:
- Carve it out into a dedicated package so it can evolve more rapidly and independently.
- Simplify dependencies (/extensions) in PromptingTools.jl by reducing the extra overhead RAGTools introduces, but also removing the need to import tonne of packages to trigger the extensions explicitly
- Simplify how to configure and customize RAGTools for different RAG pipelines without making the codebase unwieldy. Also, ideally make it easier for everyone to build on top of it.
Since this library is already used in generative AI workflows, we want to ensure the transition and further development reflect the community’s needs.
Proposed Changes & My Wish List
1) Separate Repository
- Why?
- Reduce the size and complexity of PromptingTools.jl.
- Allow RAGTools to move at its own pace without being tied to PromptingTools releases.
- Make dependency management clearer: one place to import for RAG features rather than juggling multiple extensions.
- What We Need from You
- Feedback on how you currently import and use RAGTools. Would changing your import path be a big hassle?
2) Easier Configuration
- The Challenge
- Right now, there’s a lot of dispatching around types and a tangle of keyword arguments that are function-specific (align to each stage/step of the RAG pipeline).
- Complex RAG pipelines may have multiple nested steps (often 2+ layers), which leads to confusion about where or how to specify parameters (e.g., embedding model vs. chat model) and nested named tuples are really inconvenient
- Possible Approaches
- Move Kwargs to Dispatch Types
- Each dispatch type in RAGTools would define its own specialized fields, making it clearer what arguments exist.
- Potentially less “magic,” but more explicit and possibly easier to read. It creates a lot of duplication though (Eg, verbosity kwarg…)
- Nested Context Libraries (like
EasyContext)- Provide a neat interface for hierarchical configuration.
- Possibly simpler for end users to set everything up in one place.
- Move Kwargs to Dispatch Types
- Trade-offs
- Neither of the above solve the challenge of changing a “model” (depending on the pipeline, you might want to update 1 step or 4 steps, a
modelcan be for embedding or planning or generation or modality translation/audio…, so a lot of ambiguity but also a lot of potential for overengineering…) - Too much “magic” can hide what’s happening and make customizations harder.
- Too little abstraction means more lines of code for common tasks.
- We’d love to hear your thoughts on the balance between ease-of-use and flexibility.
- Neither of the above solve the challenge of changing a “model” (depending on the pipeline, you might want to update 1 step or 4 steps, a
3) Removing FlashRank dep
- I have used FlashRank.jl many times for quick local ranking (when you fuse embedding- and keyword-based retrieval), but I appreciate most people might be calling APIs, so it might be a completely unnecessary dependency (it depends on OnnxRuntime.jl and a few things for tokenization).
Other Areas to Discuss
- Code Organization & Cleanup: Are there parts of RAGTools that feel scattered or disorganized? I know Pablo V. has some thoughts there!
- Context Handling: Does the current approach for feeding context into generative models work for you? @Sixzero will have some thoughts here
- Feature Stability: Are there features or APIs you depend on that you need to remain backward compatible? Anything mention above that concerns you?
Moving Forward (Most likely)
The first step is to move RAGTools into its own repository under the JuliaGenAI organization once we have a clearer idea of the community’s needs. We envision a short transition period, after which PromptingTools.jl will no longer bundle RAGTools. Existing code will need a slight adjustment (likely just a new import path), but we aim to keep the core functionality stable.
We Want Your Feedback!
- Have you tried the RAG approach in your own projects?
Tell us what you liked or disliked. - Any must-have features or improvements?
Don’t hold back—let’s make it the best tool possible. But please note that we are NOT looking to match all features of Python libraries
- Worried about something changing?
Let us know so we can minimize disruption or plan a smooth deprecation path.
Thanks in advance for sharing your thoughts and experiences!
EDIT: If you’re not familiar with it, you can find a quick intro here
