# Optimization on unit sphere?
**URL:** https://discourse.julialang.org/t/optimization-on-unit-sphere/130720
**Category:** Optimization (Mathematical)
**Created:** [July 14, 2025, 4:29pm UTC](https://discourse.julialang.org/t/optimization-on-unit-sphere/130720 "2025-07-14T16:29:34Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@ForceBru](https://discourse.julialang.org/u/ForceBru)
#### Post date: [July 14, 2025, 4:29pm UTC](https://discourse.julialang.org/t/optimization-on-unit-sphere/130720/1 "2025-07-14T16:29:34Z")
Random question about Differential Evolution: do you happen to know if there’s a variant of DE that keeps all vectors of the population _on the unit sphere_ or on the unit simplex?
My optimization problem is of the form:
\min\_x f(x) \quad \text{s.t.} \sum\_k x\_k^2 = 1
so x should be on the sphere. Thus, I can’t use basic DE because the mutation operation will leave the sphere.
Should I just run the usual DE step (mutation + crossover) and then project the result of the sphere, like in projected gradient descent? I tried it and it kinda works (seems to minimize the objective), but I don’t know if it’s the right way.
I also tried projecting DE’s “donor” points onto the plane tangent to the current x\_i, running mutation & crossover on these projections and then projecting/“retracting” the result back onto the sphere. This seems to minimize the objective too, but again I’m not sure if it’s a fluke.
Same with x inside the unit simplex:
\min\_x f(x) \quad \text{s.t.} \sum\_k x\_k = 1, x\_k \> 0\;\forall k
Are there variants of DE that respect such constraints? I’ve been searching on and off for a couple months, but found literally nothing on this topic.
Sorry, I know it’s off-topic for this thread, but since DE was mentioned, I have to snatch the opportunity
---
### Author:  [@PeterSimon](https://discourse.julialang.org/u/PeterSimon)
#### Post date: [July 14, 2025, 4:50pm UTC](https://discourse.julialang.org/t/optimization-on-unit-sphere/130720/2 "2025-07-14T16:50:37Z")
> [@ForceBru](#):
>
> do you happen to know if there’s a variant of DE that keeps all vectors of the population _on the unit sphere_ or on the unit simplex?
The [`Manopt.jl`](https://github.com/JuliaManifolds/Manopt.jl) package supports optimization on manifolds (such as the unit sphere) and contains several population-based metaheuristic optimizers such as PSO and CMA-ES.
---
### Author:  [@stevengj](https://discourse.julialang.org/u/stevengj)
#### Post date: [July 14, 2025, 8:10pm UTC](https://discourse.julialang.org/t/optimization-on-unit-sphere/130720/3 "2025-07-14T20:10:20Z")
> [@ForceBru](#):
>
> Random question about Differential Evolution: do you happen to know if there’s a variant of DE that keeps all vectors of the population _on the unit sphere_ or on the unit simplex?
A simple trick is to just change variables, to
\min\_{y \in \mathbb{R}^n} f(y / \Vert y \Vert\_2)
---
### Author:  [@ForceBru](https://discourse.julialang.org/u/ForceBru)
#### Post date: [July 14, 2025, 8:56pm UTC](https://discourse.julialang.org/t/optimization-on-unit-sphere/130720/4 "2025-07-14T20:56:59Z")
Yeah, this is very simple, it’s what I was using initially. But I know that f(x) is convex and thus “easy” to optimize + the optimum is unique, so I don’t want to lose convexity.
Doesn’t the y/\lVert y\rVert\_2 transformation make the function _non-convex_ in y and thus harder to optimize? Perhaps it introduces local minima?
Similarly, I can use the f(\operatorname{softmax}(y)) transformation for the unit simplex case. But then again, I think this makes the resulting function non-convex and potentially arbitrarily complex, since Softmax is non-convex?
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [July 14, 2025, 10:23pm UTC](https://discourse.julialang.org/t/optimization-on-unit-sphere/130720/5 "2025-07-14T22:23:37Z")
A different simple trick is also just to define:
x\_n = 1 - \sum\_k^{n-1} x\_k^2
and then only optimize x\_1, \ldots, x\_{n-1}, since by definition any choice of those values is on the manifold.
---
### Author:  [@kbarros](https://discourse.julialang.org/u/kbarros)
#### Post date: [July 14, 2025, 10:38pm UTC](https://discourse.julialang.org/t/optimization-on-unit-sphere/130720/6 "2025-07-14T22:38:13Z")
Yet another way is to optimize within the space of [stereographic projection](https://en.wikipedia.org/wiki/Stereographic_projection).
These [helper functions](https://github.com/SunnySuite/Sunny.jl/blob/6bd796003900f7179ce58f90354044f674d98ce0/src/Optimization.jl#L2-L63) may be of interest.
In practice, we use nonlinear conjugate gradient optimization for some fixed number of “sub” iterations with a fixed stereographic projection axis. The [outer loop](https://github.com/SunnySuite/Sunny.jl/blob/6bd796003900f7179ce58f90354044f674d98ce0/src/Optimization.jl#L127-L153) updates the projection axis as the current manifold point.
---
### Author:  [@kellertuer](https://discourse.julialang.org/u/kellertuer)
#### Post date: [July 15, 2025, 10:14am UTC](https://discourse.julialang.org/t/optimization-on-unit-sphere/130720/7 "2025-07-15T10:14:34Z")
Concerning convexity, if you minimise a function on the sphere, convexity is tricky.
_Globally_ any function that is convex (on _all_ of the sphere) is constant – which is a bit boring optimisation wise, since every point is a minimiser.
I am a bit biased (developing Manopt.jl and Manifolds.jl) and my knowledge on DE algorithms is a bit vague, but if you (a) determine the direction Y you want to “step into” as a vector tangent at the point on the sphere you are at (that is \langle x^{(k)}, Y \rangle = 0 you could use any retraction, or even the exponential map, which informally means to follow the great arc that emanates from the point in direct Y – and you would follow that for the length \lVert Y\rVert.
Then you do not have to think about constraints even 🙂
The solution from Chris might also work but you have to be careful that the values of x\_1,...x\_{n-1} still have to be constrained.
---
### Author:  [@stevengj](https://discourse.julialang.org/u/stevengj)
#### Post date: [July 15, 2025, 3:04pm UTC](https://discourse.julialang.org/t/optimization-on-unit-sphere/130720/8 "2025-07-15T15:04:21Z")
> [@ChrisRackauckas](#):
>
> A different simple trick is also just to define: x\_n = 1 - \sum\_k^{n-1} x\_k^2
You forgot the square root on the right-hand side. But with this approach you have the problem of picking a sign, and of constraining \sum\_k^{n-1} x\_k^2 \le 1; see also [Interior-Point Newton tries points outside of the constraint set](https://discourse.julialang.org/t/interior-point-newton-tries-points-outside-of-the-constraint-set/127215)
> [@kellertuer](#):
>
> determine the direction Y you want to “step into” as a vector tangent at the point on the sphere you are at
The change of variables x = y / \Vert y \Vert\_2 does this automatically, since \nabla\_y f(y / \Vert y \Vert\_2) \perp y.
> [@ForceBru](#):
>
> But I know that f(x) is convex and thus “easy” to optimize + the optimum is unique, so I don’t want to lose convexity.
You have already lost convexity (unless f is constant) because the feasible set is non-convex.
However, if your f(x) has a unique local (= global) optimum on the unit sphere, then f(y / \Vert y \Vert\_2) has a unique ray of optima y = \alpha x (\alpha \> 0), i.e. a unique value of x = y / \Vert y \Vert\_2. Local optimization will find an arbitrary y on that line, but will give the same x.
PS. As an aside, I would try hard to get a gradient of your function, if possible, so that you can use a better algorithm than DE.
---
### Author:  [@ForceBru](https://discourse.julialang.org/u/ForceBru)
#### Post date: [July 15, 2025, 7:46pm UTC](https://discourse.julialang.org/t/optimization-on-unit-sphere/130720/9 "2025-07-15T19:46:39Z")
Indeed, according to (Boyd & Vandenberghe, section 3.1.1), convexity of a function requires convexity of its domain. The probability simplex \{x : x\_k \ge 0, \sum\_k x\_k = 1\} is convex (looks like a triangle in 3D), but a sphere (as opposed to a ball, for example) is actually not, so did lose convexity by going to the sphere, huh. I totally missed that, even though it’s literally mentioned in the paper [1] I was inspired by. However, their theorem says that if f(x) is convex, then any 2nd-order KKT point z^\* of f(z \odot z) with z on the sphere will give the global minimizer x^\* = z^\* \odot z^\*, so I guess it’s not that bad…
I’m using DE because my actual objective function has other unconstrained variables and appears to have many local minima (but it is convex wrt the vector x on the simplex, with other variables fixed; perhaps this convexity doesn’t actually matter because the function isn’t convex in _all_ of its parameters). I’m trying to leverage DE’s ability to explore the loss landscape and maybe find multiple optima simultaneously. Of course, another option is to use gradient-based optimization with multistart, but I feel like it’s slow.
1. Li, Qiuwei, Daniel McKenzie, and Wotao Yin. “From the Simplex to the Sphere: Faster Constrained Optimization Using the Hadamard Parametrization.” arXiv, August 30, 2022. [[2112.05273] From the simplex to the sphere: Faster constrained optimization using the Hadamard parametrization](http://arxiv.org/abs/2112.05273).
---
### Author:  [@stevengj](https://discourse.julialang.org/u/stevengj)
#### Post date: [July 15, 2025, 8:46pm UTC](https://discourse.julialang.org/t/optimization-on-unit-sphere/130720/10 "2025-07-15T20:46:41Z")
> [@ForceBru](#):
>
> Of course, another option is to use gradient-based optimization with multistart, but I feel like it’s slow.
In a moderate number of dimensions, I’ve had good success with the MLSL multistart algorithm (which can be combined with a gradient-based local optimizer). One thing to be careful of with multistart algorithms is to tune the convergence tolerance of the local optimizer — you can often get away with a coarse convergence tolerance, and just go back at the end and “polish” your best optimum by running a local optimizer with a smaller tolerance.
---
### Author:  [@ForceBru](https://discourse.julialang.org/u/ForceBru)
#### Post date: [July 17, 2025, 7:20am UTC](https://discourse.julialang.org/t/optimization-on-unit-sphere/130720/11 "2025-07-17T07:20:05Z")
I’ll have a look, thanks!
---
### Author:  [@RoyiAvital](https://discourse.julialang.org/u/RoyiAvital)
#### Post date: [July 19, 2025, 7:40am UTC](https://discourse.julialang.org/t/optimization-on-unit-sphere/130720/12 "2025-07-19T07:40:49Z")
A simple approach (Similar to others here) is to use the Projected Gradient Descent:
- Gradient Step: \hat{\boldsymbol{x}}^{k + 1} = \boldsymbol{x}^{k} - \mu {\nabla}\_{\boldsymbol{x}} f \left( \boldsymbol{x}^{k} \right).
- Projection Step: \boldsymbol{x}^{k + 1} = {P}\_{\mathcal{C}} \left( \hat{\boldsymbol{x}}^{k + 1} \right).
In you case the projection is given by: {P}\_{\mathcal{C}} \left( \boldsymbol{x} \right) = \frac{\boldsymbol{x}}{ {\left\| \boldsymbol{x} \right\|}\_{2} }.
You can easily add Acceleration into the operation (FISTA).
It will converge, though no guarantees about the optimality.
Yet, in the case of \mathcal{C} being the Unit Simplex (A convex set), it will converge to the optimal solution.
---
### Author:  [@langestefan](https://discourse.julialang.org/u/langestefan)
#### Post date: [September 3, 2025, 6:01pm UTC](https://discourse.julialang.org/t/optimization-on-unit-sphere/130720/13 "2025-09-03T18:01:45Z")
Boyd also suggested a simple method to solve these kind of problems, namely Convex Concave Programming (CCP). It is treated in the course EE364A. See also:
> **[GitHub - cvxgrp/cvx\_ccv\_slides: Convex-Convex Lecture Slides and Code](https://github.com/cvxgrp/cvx_ccv_slides/tree/main)**
>
> main
and
> **[GitHub - cvxgrp/dccp: A CVXPY extension for convex-concave programming](https://github.com/cvxgrp/dccp)**
>
> A CVXPY extension for convex-concave programming
(unfortunately not available in Julia yet, but I recently rewrote this package)
We want to solve:
\begin{align} \min\_{x} \quad & f(x) \\ \text{s.t.} \quad & \|x\|\_2^2 = 1. \end{align}
We rewrite this as:
\begin{align} \|x\|\_2^2 \leq 1, \\ \|x\|\_2^2 \geq 1. \end{align}
The first constraint is convex. The second is nonconvex and can be expressed in DC form:
\begin{align} 1 - \|x\|\_2^2 \;\leq\; 0, \end{align}
with g(x)=1 and h(x)=\|x\|\_2^2.
At iteration k of CCP, linearize h(x) around the current iterate x^{(k)}:
\begin{align} \|x\|\_2^2 \approx \|x^{(k)}\|\_2^2 + 2(x^{(k)})^\top (x - x^{(k)}). \end{align}
Convex subproblem at iteration k becomes:
\begin{align} \min\_{x} \quad & f(x) \\ \text{s.t.} \quad &\|x\|\_2^2 - 1 \leq 0, \\ 1 - \|x^{(k)}\|\_2^2 + 2(x^{(k)})^\top &(x - x^{(k)}) \leq 0 \end{align}
If needed, you can add slack variables to the right sides and increasingly penalize violations.
Typically you can get to a decent solution within only a few iterations, using any off the shelf convex solver that can handle your f(x). Not as fancy as the other methods here ofcourse, but it does work. DCCP handles the reformulation for you automatically.