Up to where you combine the balances it seems to be reasonable. The problem was that I exclusively looked at the left-hand side. When also considering the right-hand side, i.e. the enthalpy addition by the mass flow into the control volume, one again ends up with a difference of internal energies. This was the missing “link”; thanks a lot!
Let’s finish this. You define the direction of the flow to be positive when it leaves the control volume. The equations get nicer when we define it the other way around. The mass in the control volume is
m(T) := m_{cv}(T) = V \cdot \rho(T)
and thus its time derivative is
\dot m(T) = V \cdot \dot \rho(T) = V \cdot \rho'(T) \cdot \dot T.
So, naturally, this is also the mass flow going into the control volume, from the tank (since this is the only flow that can make up this change of mass in the control volume).
We also have to keep in mind that if the flow is from the tank into the vessel, the temperature of this flow is not the same as in the vessel. I indicate this by an index \Delta. Also, I denote specific quantites by lower-case letters, so h_\Delta is the specific enthalpy of the flow between tank and vessel. The one equation resulting from the combination of mass and energy balances thus is:
\dot U(T) = \frac{d}{dt}(m(T) \cdot u(T)) = \dot m(T) \cdot u(T) + m(T) \cdot \dot u(T) = \dot m(T) \cdot h(T_\Delta, p) + \dot Q(t)
Collecting terms:
\dot m(T) \cdot (u(T) - h(T_\Delta, p)) + m(T) \cdot c(T) \cdot \dot T = \dot Q(t)
Expanding m(T) and \dot m(T):
V \cdot \rho'(T) \cdot \dot T \cdot (u(T) - h(T_\Delta, p)) + V \cdot \rho(T) \cdot c(T) \cdot \dot T = \dot Q(t)
And collecting terms again:
V \cdot \dot T \cdot \left( \rho'(T) \cdot (u(T) - h(T_\Delta, p)) + \rho(T) \cdot c(T) \right) = \dot Q(t)
With h(T_\Delta, p) = u(T_\Delta) + p \cdot \rho(T_\Delta)^{-1} and rearranging for \dot T, the ODE for the temperature is:
\dot T = \frac{\dot Q(t)}{V \cdot \left( \rho'(T) \cdot (u(T) - u(T_\Delta) - p \cdot \rho(T_\Delta)^{-1}) + \rho(T) \cdot c(T) \right)}
Where we only have a difference in the internal energies,
u(T) - u(T_\Delta) = \int_{T_\Delta}^T c(\tau) d\tau,
which can be explicitly found by the differenc of the antiderivative of c(T). The temperature of the flow between tank and vessel depends on the flow direction:
T_\Delta = \left\{ \begin{array}{l} T, \text{if } \dot m(T) < 0 \\ T_\text{tank}, \text{otherwise} \end{array} \right.
Does this make any sense? I have to digest it first, then do some plausibility checks… but dont’ have time for it right now.