Not that I’ve seem. Multi-objective MDPs and other sequential decision problems seem under-studied (speaking as someone who wrote Bi-objective multistage stochastic linear programming | Mathematical Programming).
Most people scalarize the objective a-priori and then solve the single objective problem.