The solution closest to requirements 1, 2, and 4 (works for multiple languages, lossless and efficient for simple bits types) is HDF5.
To maintain more metadata across languages, you would need some (implicit) protocol, eg that hdf5 nodes at some level are columns of a data frame, etc.
Feather is becoming rather standard for column data, if you are OK with data types that eg R can handle natively.
At the moment, I would recommend the following for moderately sized datasets:
Maintain your parsed and cleaned “master” data, possibly ingested from CSV or similar, in HDF5. This format will most likely endure for decades, and can be shared with people who use different languages/frameworks.
If you find that some other format works better for day to day work and can be generated easily from the above dataset, use it in the short run, with the implicit assumption that in the worst case scenario you just regenerate everything from the master data above. Eg JLD2 works fine for saving interim results within Julia.
For “final” results, again archive in HDF5.