GLPK can solve (mixed integer) linear programs. Your constraints, and the objective function is non-linear because you defined x, y, z, and i as variables and multiply them with each other. This is essentially what the error message says, i.e., GLPK does not support quadratic functions.
There is also some redundancy in the constraints and the objective function. The dot product, e.g., x' * y, is already a sum of the products of the coordinates of vectors x and y. Thus, you could skip the sum() functions.
Variables i are defined as binary (i.e., integer), and MadNLP does not support that. Thus, you need a solver which supports solving mixed integer non-linear programs ((MI)NLPs) such as Alpine. See the list of solvers supported by JuMP, and pick an appropriate one. I have never used MINLP solvers and therefore cannot recommend one.
Btw: you are more likely to receive help if you move your post into the category ‘Optimization (Mathematical)’.