Probabilistic Guardrails for Agentic AI

General Usage
1

Hello

I am writing this as an invitation to bring the toolkit of Actuarial Science like probabilistic bounds and confidence intervals as guardrails for Agentic AI and Modern AI more broadly.

Background

Advanced Expositions

Naive Example

in an investment authority, a senior portfolio manager may set Value at Risk (VaR) then a junior executes within the given scope. Similarly, an agent may be sandboxed to ensure it takes autonomous actions only if total VaR is below a given threshold. In Credit Scoring for example, take n to be the number of loans or transactions, and p to be the probability of default by a historical average. We can use the Percentile of a Binomial Distribution to upper bound the number of defaults with a probability lower bound. If n = 1000 and p = 0.03 then at most 40 defaults may happen with confidence 0.97.

Thesis

AI guardrail engineering is commonly tackled by schema checking and classifiers. Nonetheless, autonomy scalability is limited as Modern AI like Agentic AI is intrinically probabilistic. On the other hand, Actuarial Science is a well-established domain for risk management under uncertainty. I am initiating the community here to bring its toolkit to design safety guardrails for modern AI. I believe Jev is an excellent step and I can imagine beautiful use-cases if it got combined with Probabilistic Reasoning like Probabilistic Logic.

Why Julia?

The community here knows AI as science. We have a mature native ecosystem, notably Julia Actuary, Turing, and POMDP.

Discussion

  • Do you see potential in bringing Actuarial Science toolkit to modern AI safety? Are there critical challenges?
  • Are you aware of any community following that direction?
  • If you were to initiate a new direction, what would be your first steps?
2

I was reading the first part of this post thinking :thinking: “is he talking about jev?” because Im down the rabbit hole with this one in spectacular fashion for whatever reason and then you mentioned jev and I agree with all of this. I had to look up actuarial though and needed that link.

3

Thank you for the kind words, @obsidianjulua.

Feel free to share any question or concern you have. I am looking for feedback at this stage.

4

My feedback is jev is like a JIT agent and julia is a JIT compiler, do I think using julias compute power with a agent thats fast enough to abuse it is a good idea… definitely. Any system 1 type agent that can take data from julia and be able to act fast enough to distribute, then its just architecture. The agent can basically be a dynamic kernel for a codebase and be small enough and fast enough to be worth running sidecar with a JIT. thats my feedback.

My question is are you proposing to build this in julia based off feedback from this post?

5

Jev is also cloud access only, and latency becomes a per system number, Have you considered Laya or any other type 1 models other than a cloud type 1 only?

6

Thank you @obsidianjulua.

Let me clarify the post is not about Jev but in particular about utilizing Actuarial Toolkit for designing Agentic Sandbox. If we have solid probabilistic guardrails, then Jev may be notably useful but it is not the goal.

What is attractive in Julia is its native ecosystem for probabilistic modeling.

I don’t see the problem of agents in a faster language, but clearly in token cost and hallucination.

Let me know if you have any other thoughts.