RFC/WIP: add complex norm-1 and norm-infinity cones

[chriscoey]

Hypatia supports real and complex norm-1 and norm-infinity cones (and real/complex versions of various other cones, such as spectral and PSD/rootdet/logdet). This is the least-complicated way to support these two cones, I think. If we agree on this soon then I can generalize the spectral/nuclear cones in a similar way after #976 is merged, and then we can discuss how to do it for Hermitian PSD/rootdet/logdet.

[codecov-io]

Codecov Report

Merging #996 into master will decrease coverage by 0.06%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master     #996      +/-   ##
==========================================
- Coverage   95.25%   95.19%   -0.07%     
==========================================
  Files          97       98       +1     
  Lines       10821    11050     +229     
==========================================
+ Hits        10308    10519     +211     
- Misses        513      531      +18

Impacted Files	Coverage Δ
src/sets.jl	94.38% <0%> (-1.96%)	⬇️
src/FileFormats/MOF/write.jl	66.3% <0%> (-1.48%)	⬇️
src/Test/contconic.jl	99.22% <0%> (-0.28%)	⬇️
src/FileFormats/MOF/MOF.jl	100% <0%> (ø)	⬆️
src/Test/UnitTests/basic_constraint_tests.jl	98.21% <0%> (ø)	⬆️
src/Bridges/Constraint/norm_spec_nuc_to_psd.jl	91.66% <0%> (ø)
src/FileFormats/MOF/read.jl	93.1% <0%> (+0.12%)	⬆️
src/Bridges/Constraint/Constraint.jl	96.42% <0%> (+0.27%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 80098c0...4fc232a. Read the comment docs.

[chriscoey]

I've expanded the descriptions of the sets as stacking real and imaginary parts of complex elements. This is the same idea Hypatia uses for these cones and for complex spectral/nuclear and hermitian PSD/logdet/rootdet cones (only complex elements ie not epigraphs or hermitian diagonals get two stacked elements).

[blegat]

I am still undecided between this and using complex coefficients in the functions.
The argument against complex coefficients is that you need to convert the real diagonal elements of hermitian matrices to a complex value but it's similar to the SingleVariable functions that are converted to affine functions in Complements and Indicator to fit in a VAF.
The argument for complex coefficients is that you need to duplicate each set definition and bridges, even if the bridge work in both the complex and real case.

[chriscoey]

It's not clear to me that the bridges won't need significant changes to work for real and complex cases anyway.
Duplicating cone definitions isn't such a big deal if we structure the descriptions of the complex cases to reference the real cases and just describe the differences.

[odow]

So my hesitations with adding these cones, and the other ones in this series of PRs are:

1. we don't know we're going to support complex numbers in JuMP, so it seems JuMPing (if you will) the gun to add complex cones.
2. We're adding cones that aren't obvious how widely they will be supported or used
3. Adding lots of cones increases the long-term maintenance burden of MOI
4. We probably have to add these cones to MathOptFormat

I get that these cones are useful for modeling, and Hypatia supports them. But I'm strongly in favor of adding them in Hypatia, or in a ConicModelingExtensions.jl package first, letting the definitions and issues settle down in that package, and then adding them to MOI in the long-term once it's obvious we want to support them.

Edit: a good bar to pass is: if it's in CVX or Convex.jl (e.g., spectral norm is in CVX), we should add it. If not, we should thing long and hard about the utility of adding it.

cc @mlubin thoughts?

[mlubin]

@blegat will have a better estimate than I do of the extra maintenance burden from additional sets.

In general, if a set is supported by only one solver and we have no short-term plans to provide special modeling support for it in JuMP or another upstream modeling package, then I don't see a strong motivation for including it in MOI.

@chriscoey, is there a technical reason why it helps you to have these cones defined in MOI instead of in Hypatia or a hypothetical ConicModelingExtensions package?

[chriscoey]

@mlubin No there's no reason I need them in MOI, I was doing it because I believe they are useful modeling constructs and bridges for the community to have. Some are more useful than others of course, and there's a chicken and egg problem in that no one will use them if they don't exist.

One goal of the complex cones discussion here is deciding how to handle the Hermitian PSD cone, for which there exists a less efficient real PSD reformulation.

[odow]

I believe they are useful modeling constructs and bridges for the community to have.

I think we all agree with this.

no one will use them if they don't exist.

We're debating between

using JuMP, Hypatia
model = Model()
@variable(model, x in MOI.MyCoolCone())

and

using JuMP, Hypatia, ConicModelingExtensions
model = Model()
@variable(model, x in ConicModelingExtensions.MyCoolCone())
# or even
@variable(model, x in MyCoolCone())

One goal of MOI was to make it extensible so that people could add new cones without having to modify JuMP/MOI.

[chriscoey]

So would you propose moving sets like logdet and rootdet and geometric mean to that package? What would be the criteria for which cones belong where?

[odow]

If it's in CVX(py), it's a solid candidate: e.g., https://www.cvxpy.org/api_reference/cvxpy.atoms.other_atoms.html#geo-mean

I don't think we should necessarily move cones out of MOI. I'm just advocating adding them more carefully in future.

[ericphanson]

@mlubin No there's no reason I need them in MOI, I was doing it because I believe they are useful modeling constructs and bridges for the community to have. Some are more useful than others of course, and there's a chicken and egg problem in that no one will use them if they don't exist.

FWIW, I think Convex.jl could definitely depend on a hypothetical ConicModelingExtensions package and using those cones for formulating its problems, which might negate some of the visibility problem. Convex.jl also has complex-number support (although we just reformulate everything as real before passing to MOI, currently).

[chriscoey]

this can be revisited later if there is a desire. depends on what the plan is for complex modeling in MOI.