Probabilistic Guardrails for Agentic AI

General Usage
1

Hello

I am writing this as an invitation to bring the toolkit of Actuarial Science like probabilistic bounds and confidence intervals as guardrails for Agentic AI and Modern AI more broadly.

Background

Advanced Expositions

Naive Example

in an investment authority, a senior portfolio manager may set Value at Risk (VaR) then a junior executes within the given scope. Similarly, an agent may be sandboxed to ensure it takes autonomous actions only if total VaR is below a given threshold. In Credit Scoring for example, take n to be the number of loans or transactions, and p to be the probability of default by a historical average. We can use the Percentile of a Binomial Distribution to upper bound the number of defaults with a probability lower bound. If n = 1000 and p = 0.03 then at most 40 defaults may happen with confidence 0.97.

Thesis

AI guardrail engineering is commonly tackled by schema checking and classifiers. Nonetheless, autonomy scalability is limited as Modern AI like Agentic AI is intrinically probabilistic. On the other hand, Actuarial Science is a well-established domain for risk management under uncertainty. I am initiating the community here to bring its toolkit to design safety guardrails for modern AI. I believe Jev is an excellent step and I can imagine beautiful use-cases if it got combined with Probabilistic Reasoning like Probabilistic Logic.

Why Julia?

The community here knows AI as science. We have a mature native ecosystem, notably Julia Actuary, Turing, and POMDP.

Discussion

  • Do you see potential in bringing Actuarial Science toolkit to modern AI safety? Are there critical challenges?
  • Are you aware of any community following that direction?
  • If you were to initiate a new direction, what would be your first steps?
2

I was reading the first part of this post thinking :thinking: “is he talking about jev?” because Im down the rabbit hole with this one in spectacular fashion for whatever reason and then you mentioned jev and I agree with all of this. I had to look up actuarial though and needed that link.