What fileformat to use to load data for high performance computing

Like you said it’s designed for OLAP. I think for OLTP in the big data space things like NoSQL stores (e.g. Cassandra) or in-memory solutions like Ignite (has good integration with Spark) or SnappyData (even tighter integration with Spark: in fact it fuses Gemfire with Spark and changes the Spark code making Spark a “real” OLTP SQL DB) are used.

If you’re interested in knowing more about carbondata, here the presentation where I first learned about it: Apache Carbondata: An Indexed Columnar File Format for Interactive Query by Jacky Li/Jihong Ma - YouTube