Every density function is defined wrt a base measure, so it’s not enough to say whether the density function exists or not. We must define the base measure. The reason this fails in Distributions is that every distribution has an implicit base measure (absolutely continuous wrt the Lebesgue measure for Normal and absolutely continuous wrt the counting measure for Dirac), and MixtureModel requires these be the same measure.
But we can also define a mixture measure that is a mixture of different measures and use that as a base measure for a mixture model, which allows for mixing continuous and discrete components. This is the implicit base measure of Censored. See also https://twitter.com/sethaxen/status/1488314028803997698?t=YdU1ZRQXaiCROopv8l53eg&s=19.
So the best you can probably do now is use a narrow Normal but this will have problems. There are some Distributions issues tracking allowing MixtureModel to mix measures and in general supporting Distributions with atoms, and I can share some of those later when I’m at a computer.