# Creating a custom Julia AWS Layer Runtime

**URL:** <https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166>\
**Category:** General Usage\
**Created:** [November 30, 2018, 12:03am UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166 "2018-11-30T00:03:24Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Seanny123](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/seanny123/32/5271_2.png) [@Seanny123](https://discourse.julialang.org/u/Seanny123)\
**Post date:** [November 30, 2018, 12:03am UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/1 "2018-11-30T00:03:25Z")

</div>

To deploy a Julia program on AWS Lambda currently involves making a custom Docker Image. Recently, Amazon announced the ability to create a [Custom Runtime](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html), which lowers the barrier to entry. However, their tutorial is quite lacking and they mostly seem to be [partnering with companies](https://aws.amazon.com/blogs/aws/new-for-aws-lambda-use-any-programming-language-and-share-common-components/)?

Does anyone know how difficult it is to make a Custom Runtime? Alternatively, are there any plans to partner with AWS Lambda?

---

<div class="post-metadata">

**Author:** ![quinnj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/quinnj/32/11_2.png) [@quinnj](https://discourse.julialang.org/u/quinnj)\
**Post date:** [December 3, 2018, 9:35pm UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/2 "2018-12-03T21:35:29Z")

</div>

I’ve looked into this a bit; I don’t think it will be too bad to create a julia-based custom runtime, along with “layers” for popular packages. I’ll probably be playing around with creating the julia runtime this week a bit.

---

<div class="post-metadata">

**Author:** ![Seanny123](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/seanny123/32/5271_2.png) [@Seanny123](https://discourse.julialang.org/u/Seanny123)\
**Post date:** [December 3, 2018, 10:28pm UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/3 "2018-12-03T22:28:43Z")

</div>

Please let me know if there’s any way I can help with this. I looked at the documentation and was absolutely baffled.

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [December 5, 2018, 7:54pm UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/4 "2018-12-05T19:54:07Z")

</div>

I am currently at the AWS London User Group - a report back on re:Invent  
The speaker is discussign these custom runtime - GO, Rust, cobol(!) C++  
But no mention of Julia. Its a high level overview, so no mention of how hard it is to create a specific one for Julia.  
the talk is by Danilo Poccia, Principal Evangelist Serverless, AWS  
I will ask if the slides are available.

Count me in for testing too @Seanny123 @quinnj

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [December 6, 2018, 8:54am UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/5 "2018-12-06T08:54:36Z")

</div>

I talked with Danilo after the meeting, he thinks it should be easy to create a custom runtime for Julia. Not much time to talk, but mentioned a bootstrap process.  
I dont know how this would fit in with Julias package structure - maybe we woudl need a different runtime for each mix of packages we need - to be seen when the rubber hits the road of course.

---

<div class="post-metadata">

**Author:** ![yakir12](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yakir12/32/297_2.png) [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Post date:** [December 6, 2018, 9:02am UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/6 "2018-12-06T09:02:00Z")

</div>

> [@johnh](#):
>
> Danilo Poccia, Principal Evangelist Serverless, AWS

This line reads as if it was taken from some dystopian SciFi book…

---

<div class="post-metadata">

**Author:** ![bibourokushi](https://avatars.discourse-cdn.com/v4/letter/b/9de053/32.png) [@bibourokushi](https://discourse.julialang.org/u/bibourokushi)\
**Post date:** [December 10, 2018, 5:20pm UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/7 "2018-12-10T17:20:38Z")

</div>

Here is start or some seed?

> [Files · master · bibourokushi / try aws custom runtime for julia · GitLab](https://gitlab.com/bibourokushi/try-aws-custom-runtime-for-julia/tree/master)

---

<div class="post-metadata">

**Author:** ![Seanny123](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/seanny123/32/5271_2.png) [@Seanny123](https://discourse.julialang.org/u/Seanny123)\
**Post date:** [December 10, 2018, 8:54pm UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/8 "2018-12-10T20:54:13Z")

</div>

This is a good seed! However, I’m still confused by the set of invocations and how to handle them.

---

<div class="post-metadata">

**Author:** ![PrabodhaSrimal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/prabodhasrimal/32/6399_2.png) [@PrabodhaSrimal](https://discourse.julialang.org/u/PrabodhaSrimal)\
**Post date:** [December 11, 2018, 9:08am UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/9 "2018-12-11T09:08:26Z")

</div>

An experimental AWS lambda runtime for Julia can be found in [https://github.com/FugroRoames/aws-lambda-julia-runtime](https://github.com/FugroRoames/aws-lambda-julia-runtime)

---

<div class="post-metadata">

**Author:** ![Seanny123](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/seanny123/32/5271_2.png) [@Seanny123](https://discourse.julialang.org/u/Seanny123)\
**Post date:** [December 20, 2018, 2:51pm UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/10 "2018-12-20T14:51:50Z")

</div>

Thanks to my co-worker’s heroic efforts, [we managed to update your run-time to Julia 1.0](https://gitlab.com/emagin-open-source/awslambdajulia/tree/master). We’ll update the instructions and provide background on how the steps required in the future.

---

<div class="post-metadata">

**Author:** ![ValdarT](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/valdart/32/24146_2.png) [@ValdarT](https://discourse.julialang.org/u/ValdarT)\
**Post date:** [December 20, 2018, 6:46pm UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/11 "2018-12-20T18:46:52Z")

</div>

The link doesn’t work for me.

---

<div class="post-metadata">

**Author:** ![Seanny123](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/seanny123/32/5271_2.png) [@Seanny123](https://discourse.julialang.org/u/Seanny123)\
**Post date:** [December 20, 2018, 7:35pm UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/12 "2018-12-20T19:35:25Z")

</div>

Sorry, I didn’t realize the project was private. I’ve made it public now.

---

<div class="post-metadata">

**Author:** ![Seanny123](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/seanny123/32/5271_2.png) [@Seanny123](https://discourse.julialang.org/u/Seanny123)\
**Post date:** [December 20, 2018, 7:36pm UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/13 "2018-12-20T19:36:01Z")

</div>

Current issues still being worked on:

1. Telling Julia where to find packages
2. Resolving bugs we haven’t noticed yet…

---

<div class="post-metadata">

**Author:** ![ValdarT](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/valdart/32/24146_2.png) [@ValdarT](https://discourse.julialang.org/u/ValdarT)\
**Post date:** [January 5, 2019, 10:44am UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/14 "2019-01-05T10:44:32Z")

</div>

I would be relly interested in hearing more about your (and others’) experiences. I understand you weren’t quite successful in the end @Seanny123? What were the remaining problems? How did it work apart from these?

---

<div class="post-metadata">

**Author:** ![Seanny123](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/seanny123/32/5271_2.png) [@Seanny123](https://discourse.julialang.org/u/Seanny123)\
**Post date:** [January 5, 2019, 3:25pm UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/15 "2019-01-05T15:25:46Z")

</div>

We got Hello World going, but communication was impossible due to being unable to import `Http.jl`. It existed. It seemed to be in the expected place, however Julia could not find it.

---

<div class="post-metadata">

**Author:** ![\_rob](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/_rob/32/4276_2.png) [@\_rob](https://discourse.julialang.org/u/_rob)\
**Post date:** [March 14, 2019, 8:06am UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/16 "2019-03-14T08:06:44Z")

</div>

Has anyone had any more luck with this yet? I have been using `samoconnor/AWSLambda.jl` but would like to use my Julia1. code.

---

<div class="post-metadata">

**Author:** ![ianfiske](https://avatars.discourse-cdn.com/v4/letter/i/58f4c7/32.png) [@ianfiske](https://discourse.julialang.org/u/ianfiske)\
**Post date:** [April 26, 2019, 7:21pm UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/17 "2019-04-26T19:21:04Z")

</div>

I’ve been working on this a bit using your code as a start @Seanny123. The main problem I’ve run into is the AWS limit on the size of the zip. The runtime zipped is ~70MB, but AWS requires a max of 50MB. Were you able to shrink the size down any further?

---

<div class="post-metadata">

**Author:** ![Seanny123](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/seanny123/32/5271_2.png) [@Seanny123](https://discourse.julialang.org/u/Seanny123)\
**Post date:** [April 29, 2019, 1:54am UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/18 "2019-04-29T01:54:09Z")

</div>

I thought AWS Lambda “layers” had different limits than functions? I think that’s why we were able to run our layer despite the larger size.

---

<div class="post-metadata">

**Author:** ![ianfiske](https://avatars.discourse-cdn.com/v4/letter/i/58f4c7/32.png) [@ianfiske](https://discourse.julialang.org/u/ianfiske)\
**Post date:** [April 29, 2019, 1:45pm UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/19 "2019-04-29T13:45:30Z")

</div>

Hmmm, yeah, that’s what I thought too. But

```julia
aws lambda publish-layer-version --layer-name julia-runtime --zip-file fileb://packaging/julia-v1.zip 

An error occurred (RequestEntityTooLargeException) when calling the PublishLayerVersion operation: Request must be smaller than 69905067 bytes for the PublishLayerVersion operation

```

---

<div class="post-metadata">

**Author:** ![ianfiske](https://avatars.discourse-cdn.com/v4/letter/i/58f4c7/32.png) [@ianfiske](https://discourse.julialang.org/u/ianfiske)\
**Post date:** [April 29, 2019, 3:05pm UTC](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166/20 "2019-04-29T15:05:34Z")

</div>

Actually, this limit seems to be for direct zip-upload only. The limit is removed if we put the zip file in s3 and point to the zip file in s3.

[Next page](https://discourse.julialang.org/t/creating-a-custom-julia-aws-layer-runtime/18166.md?page=2)