Development Process for Working with Genie and Javascript REST API

whenning · June 13, 2020, 6:20pm

Describe the Goal
I am attempting to call my local Genie web server using a REST API in javascript. Is this an appropriate development process?

The server works great when I call something like http://127.0.0.1:8000/ directly in the browser. However, when I try to access the running server using my javascript REST API, I get an error:

404 Not Found Not found because of proxy error: Error: connect ECONNREFUSED 127.0.0.1:8000

To reproduce
For this simple example, I am using CORS anywhere
https://robwu.nl/cors-anywhere.html http://127.0.0.1:8000

mthelm85 · June 13, 2020, 6:57pm

I think it would be helpful if you could describe the goal more explicitly. What does your Genie web server do? Do you have a front-end GUI for your app? Is your REST API written in Node.js?

I can see calling a Genie API with JavaScript from your front-end app but I’m having trouble thinking of a use case for a Node.js API to call a Genie.jl API.

I would post the code from your Genie server, or an equivalent minimal working example, here and I think that will give us a better idea of what you’d like to do.

whenning · June 13, 2020, 7:26pm

Thank you for your quick reply.

I create my front end using React and I run a Julia web server in a Docker container on Google Compute Engine. I currently use Joseki.jl to run the web server and it works great, but Genie seems like a bigger project so I thought I would check it out.

The biggest pain of my development stack is that I every time I make a change to my Julia code, I have to push it all the way out to my Kubernetes cluster before I can test it. Awful. So I am trying to figure out a more efficient approach.

This is what I am trying to understand. What is the most simple way for me to make a fetch from React while developing? So that I can go painlessly back and forth between the computational and interface development environments.

I do not use any Node.js. I host my app on Firebase. If this is not the right way of thinking about this development process, I appreciate advice

Create a simple function to test a POST API call
routes.jl

using Genie, Genie.Router, Genie.Renderer.Json, Genie.Requests
using HTTP

Genie.config.run_as_server = true
Genie.config.cors_allowed_origins = ["*"]

route("/random", method=POST) do
  data = jsonpayload()
  dim = parse(Int, data["dim"])
  num = parse(Int, data["num"])
  (:randomdata => rand(dim,num)) |> json
end


Genie.startup()

*Then I try and call the web server from my javascript app. Maybe this is not possible, and I am not experienced here so likely mistakes. The CORS stuff is not clear to me. *

App.js


  const proxyurl = "https://cors-anywhere.herokuapp.com/";
  const url = 'http://127.0.0.1:8000/random'
  let data = {
    dim: '1',
    num: '3'
  }
// The parameters we are gonna pass to the fetch function
let fetchData = { 
    method: 'POST', 
    body: data,   
    headers: new Headers()
}
fetch(proxyurl + url)
.then(function() {
    // Handle response you get from the server
    console.log('hi data')
})
.catch(()=>console.log('Wade- you cant access the url response'));

whenning · June 17, 2020, 2:15pm

@mthelm85
What does your development process look like? How do you call Julia from your GUI’s? Where do you host your applications?

mthelm85 · June 17, 2020, 2:27pm

Here’s a sample (this is on an older version of Genie, not sure if this still works but I can confirm it worked great at the time):

Genie.jl API

JuliaAPI/src/ ApiExample.jl:

using Genie
using Genie.Router
import Genie.Router: route
import Genie.Renderer: json
include("Sum.jl")

function launch_server(port)
    Genie.config.run_as_server = true
    Genie.config.server_host = "0.0.0.0"
    Genie.config.server_port = port
    Genie.config.cors_allowed_origins = ["*"]

    route("/addxy") do
        x = parse(Int, get(@params, :x, "0"))
        y = parse(Int, get(@params, :y, "0"))
        (:answer => addxy(x,y)) |> json
    end

    Genie.startup()
end

launch_server(parse(Int, ARGS[1]))

JuliaAPI/src/ Sum.jl:

function addxy(x::Int64, y::Int64)
    return sum([x,y])
end

Vue.js frontend

The relevant code is just this:

import axios from 'axios'

export default {
  name: 'HelloWorld',
  data: () => ({
    x: null,
    y: null,
    answer: null
  }),

  methods: {
    async getAnswer() {
      try {
        const res = await axios.get('https://cors-anywhere.herokuapp.com/https://julia-api-example.herokuapp.com/addxy', {
          params: {
            x: this.x,
            y: this.y
          }
        })
        this.answer = res.data.answer
      } catch (err) {
        alert(err)
      }
    }
  }
}

All of the code is in these two GitHub repos:

https://github.com/mthelm85/JuliaAPI

https://github.com/mthelm85/JuliaAPIClient

Also, I pushed both the API and the GUI to free Heroku dynos so you can test out the front-end here:

https://julia-api-client.herokuapp.com/

Since they are free dynos, they both go into “sleep mode” until someone hits the URL. It may take a few minutes for the GUI to “wake up” and then once you try to compute a sum it will reach out to the Julia API and it will also have to “wake up.” I also just tried it out and for the first few minutes it was giving me a CORS error which, oddly enough, stopped happening after a few minutes (it’s working fine at the time I’m writing this but will probably not work right for the first few minutes that you try). This was just a proof of concept so I never tried to work out the bugs (of which there appear to be many ).

fonsp · June 17, 2020, 2:46pm

You can’t use CORS anywhere to proxy your localhost (127.0.0.1) because CORS anywhere can only connect you to addresses on the web. The address 127.0.0.1 is only accessible inside your machine.

Why are you using a CORS proxy? Is the CORS header not working? You can use the Network tab in the chrome/ffox dev tools to check which headers you received.

fonsp · June 17, 2020, 3:02pm

The workflow that I would suggest in your case is to run both servers locally on different ports. Make sure that the Julia API server sets the header

Access-Control-Allow-Origin: *

and you can then access it from the browser using:

fetch("http://localhost:1234/...", {mode: "cors"}).then(...)

where 1234 is the port of the Julia API.

If your front-end needs to be compiled using Node, then install Node locally and use a dev server. Running locally makes your life a lot easier!

Lots of apps can be written without a compile step - have a look at Preact and HTM to replace React and JSX - they work in the browser right away.

whenning · June 17, 2020, 3:22pm

Thanks @mthelm85 . This is similar to what I am currently doing, except that I have been thinking about trying heroku for basic computing and I appreciate your example.

whenning · June 17, 2020, 3:24pm

Thanks @fonsp. This is what I have been trying to achieve without success, but also with not enough understanding. I will retry with your advice and report back.

whenning · June 29, 2020, 3:42pm

@fonsp I am able to successfully communicate between my development front-end and development Julia servers (which is great), but I am getting a CORS error when I try and fetch:

Access to fetch at 'http://127.0.0.1:8000/random' from origin 
'http://localhost:3000' has been blocked by CORS policy: 
The 'Access-Control-Allow-Origin' header contains the invalid value ''. 
Have the server send the header with a valid value, or, if an opaque 
response serves your needs, set the request's mode to 'no-cors' 
to fetch the resource with CORS disabled.

routes.jl

using Genie, Genie.Router, Genie.Renderer.Json, Genie.Requests
using HTTP

Genie.config.run_as_server = true
Genie.config.cors_allowed_origins = ["*"]

route("/random", method="POST") do
  dim = parse(Int, get(@params, :dim, "2"))
  num = parse(Int, get(@params, :num, "3"))
  (:random => rand(dim,num)) |> json
end

Genie.startup()

relevant App.js

let params = {
  dim: '1',
  num: '3'
}

const url = 'http://127.0.0.1:8000/random'
fetch(url, {
            method: 'POST',
            mode: "cors", 
            body: params               
          })
.then(response => response.json())
.then(data => console.log(data))
.catch(()=>console.log('Wade- you cant access the url response'));

whenning · June 30, 2020, 8:16pm

@mthelm85 Do you have a good source for the Heroku process of pushing the Julia server to Heroku?

whenning · June 30, 2020, 8:20pm

@mthelm85 @fonsp UPDATE: I posted the CORS problem to the Genie.jl issues on Github and they found an error on their side. Thanks for all your help. I just assumed that I was doing something wrong, and I appreciate your help in guiding me.