# \#spark

**URL:** https://discourse.julialang.org/tag/spark/256.md

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

---

## [Expected Performance of Julia within a Spark Environment?](https://discourse.julialang.org/t/expected-performance-of-julia-within-a-spark-environment/91377)

<div class="topic-metadata">

**Author:** [@TheCedarPrince](https://discourse.julialang.org/u/TheCedarPrince)\
**Replies:** 7\
**Last updated:** [December 7, 2022, 10:26pm UTC](https://discourse.julialang.org/t/expected-performance-of-julia-within-a-spark-environment/91377 "2022-12-07T22:26:28Z")

</div>

Hey folks, I need some help on answering a question I was posed: Assuming an algorithm is written in R or Python with some attempt to leverage the optomizations available in a Spark environment for parallelization etc…

---

## [\[ANN\] Spark.jl, reborn](https://discourse.julialang.org/t/ann-spark-jl-reborn/82275)

<div class="topic-metadata">

**Author:** [@dfdx](https://discourse.julialang.org/u/dfdx)\
**Replies:** 0\
**Last updated:** [June 5, 2022, 3:51pm UTC](https://discourse.julialang.org/t/ann-spark-jl-reborn/82275 "2022-06-05T15:51:14Z")

</div>

I’m pleased to announce a new release of Spark.jl - Julia interface to Apache Spark. Apache Spark is a ubiquitous distributed data processing framework used by thousands of organizations for large scale data engineering…

---

## [Error while parallelizing](https://discourse.julialang.org/t/error-while-parallelizing/74049)

<div class="topic-metadata">

**Author:** [@Sumit\_Malbari](https://discourse.julialang.org/u/Sumit_Malbari)\
**Replies:** 0\
**Last updated:** [January 4, 2022, 5:43pm UTC](https://discourse.julialang.org/t/error-while-parallelizing/74049 "2022-01-04T17:43:30Z")

</div>

Hi All, I ran JULIA\_COPY\_STACKS=yes julia -e ‘using Pkg;Pkg.add(Pkg.PackageSpec(;name=“Spark”, version=“0.5.1”));Pkg.build(“Spark”);using Spark;Spark.init();sc = SparkContext(master=“yarn”);’ all this worked fine. C…

---

## [Setting up Julia on Spark on AWS EMR](https://discourse.julialang.org/t/setting-up-julia-on-spark-on-aws-emr/63781)

<div class="topic-metadata">

**Author:** [@Sumit\_Malbari](https://discourse.julialang.org/u/Sumit_Malbari)\
**Replies:** 24\
**Last updated:** [January 4, 2022, 3:13pm UTC](https://discourse.julialang.org/t/setting-up-julia-on-spark-on-aws-emr/63781 "2022-01-04T15:13:50Z")

</div>

Hi folks, I want to use Julia on Spark on AWS EMR as this is one of the requirement at my workplace. I tried to follow the steps from Julia & Spark but I was not able to succeed. If anyone has implemented it on AWS EM…

---

## [When will Julia compete with Spark?](https://discourse.julialang.org/t/when-will-julia-compete-with-spark/25218)

<div class="topic-metadata">

**Author:** [@merlin](https://discourse.julialang.org/u/merlin)\
**Replies:** 16\
**Last updated:** [June 5, 2021, 5:09pm UTC](https://discourse.julialang.org/t/when-will-julia-compete-with-spark/25218 "2021-06-05T17:09:45Z")

</div>

My org uses Spark + EMR to query and transform static files on S3 and output to files that either get loaded to a DB or used as input for analysis. It seems like Julia could also do this and do it well, of course Spark …