# \[ANN\] SeaPearl.jl - A Constraint Programming solver that can use Reinforcement Learning to improve its heuristics

**URL:** <https://discourse.julialang.org/t/ann-seapearl-jl-a-constraint-programming-solver-that-can-use-reinforcement-learning-to-improve-its-heuristics/60607>\
**Category:** Package Announcements\
**Tags:** package, announcement, jump, flux\
**Created:** [May 5, 2021, 9:51pm UTC](https://discourse.julialang.org/t/ann-seapearl-jl-a-constraint-programming-solver-that-can-use-reinforcement-learning-to-improve-its-heuristics/60607 "2021-05-05T21:51:35Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![ilancoulon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ilancoulon/32/24791_2.png) [@ilancoulon](https://discourse.julialang.org/u/ilancoulon)\
**Post date:** [May 5, 2021, 9:51pm UTC](https://discourse.julialang.org/t/ann-seapearl-jl-a-constraint-programming-solver-that-can-use-reinforcement-learning-to-improve-its-heuristics/60607/1 "2021-05-05T21:51:35Z")

</div>

I am pretty proud to announce that [SeaPearl](https://github.com/corail-research/SeaPearl.jl), the package [Félix Chalumeau](https://github.com/felixchalumeau) and I have been working for the past year, is finally out.

> **[GitHub - corail-research/SeaPearl.jl: Julia hybrid constraint programming...](https://github.com/corail-research/SeaPearl.jl)**
>
> Julia hybrid constraint programming solver enhanced by a reinforcement learning driven search. - GitHub - corail-research/SeaPearl.jl: Julia hybrid constraint programming solver enhanced by a reinf...

It intends to be a whole Constraint Programming solver, but that can use a Reinforcement Learning agent as a value-selection heuristic to choose in a smarter way which value to branch on.  
It comes with a few generators that allow the agent to learn on randomly generated instances of the graph-coloring problem, the knapsack and the TSPTW; and with a set of examples that can be found on a [separate repository](https://github.com/corail-research/SeaPearlZoo).

Whereas the [paper](https://arxiv.org/abs/2102.09193) accompanying the launch has shown a first proof of concept on previously done experiments, it should be seen as a great sandbox tool for other researchers to explore this (in our opinion) promising path of RL&CP combination.

> **[SeaPearl: A Constraint Programming Solver guided by Reinforcement Learning](https://arxiv.org/abs/2102.09193)**
>
> The design of efficient and generic algorithms for solving combinatorial optimization problems has been an active field of research for many years. Standard exact solving approaches are based on a clever and complete enumeration of the solution set....

The project, endorsed by Polytechnique Montréal through [Quentin Cappart](https://www.polymtl.ca/expertises/en/cappart-quentin-0) and [Louis-Martin Rousseau](https://www.polymtl.ca/expertises/en/rousseau-louis-martin), has welcomed new team members who are Tom Marty, Kim Rioux-Paradis, Tom Sander and Pierre Tessier. They are already improving the existing tool and will try out new exciting experiments with it.

Feel free to try it or come back to us if you have any question! One of our next step is to make it easier to use, for instance by adding a JuMP support.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [May 5, 2021, 10:05pm UTC](https://discourse.julialang.org/t/ann-seapearl-jl-a-constraint-programming-solver-that-can-use-reinforcement-learning-to-improve-its-heuristics/60607/2 "2021-05-05T22:05:08Z")

</div>

Cool stuff! Swing by the [developer chatroom](https://gitter.im/JuliaOpt/JuMP-dev) if you have queries interfacing with JuMP. @dourouc05 has been working on this as well [https://github.com/dourouc05/ConstraintProgrammingExtensions.jl](https://github.com/dourouc05/ConstraintProgrammingExtensions.jl). I’ll also plug [JuMP-dev 2021 | JuMP](https://jump.dev/meetings/juliacon2021/) as a place where people would be keen to hear more about it.

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [May 6, 2021, 12:12am UTC](https://discourse.julialang.org/t/ann-seapearl-jl-a-constraint-programming-solver-that-can-use-reinforcement-learning-to-improve-its-heuristics/60607/3 "2021-05-06T00:12:12Z")

</div>

@ilancoulon @felixchalumeau very nicely done

---

<div class="post-metadata">

**Author:** ![ilancoulon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ilancoulon/32/24791_2.png) [@ilancoulon](https://discourse.julialang.org/u/ilancoulon)\
**Post date:** [May 9, 2021, 9:20am UTC](https://discourse.julialang.org/t/ann-seapearl-jl-a-constraint-programming-solver-that-can-use-reinforcement-learning-to-improve-its-heuristics/60607/4 "2021-05-09T09:20:15Z")

</div>

Thanks a lot, we’ll look into it when the time comes!