How to optimally trap points in high-dimensional spaces inside ellipsoids (opens in new tab)

(adrianriv.com)

84 pointstartakovsky2y ago17 comments

17 comments

If you find this interesting, you might enjoy my write up of the Matoušek-Sharir-Welzl (MSW) algorithm used by D3’s circle-packing layout.

https://observablehq.com/@d3/d3-packenclose

I still haven’t quite figured out how to make D3’s implementation robust, though. Volodymyr Agafonkin’s robust-predicated would probably help… https://github.com/mourner/robust-predicates

fiforpg2y ago

There is a very nice short proof of John's ellipsoid theorem by Gruber and Schuster:

https://www.dmg.tuwien.ac.at/gruber/gruber_arbeiten/johnelli...

— one elegant trick I remember from there was that the value of a quadratic form with matrix A on vectors u and v (^T for transpose):

u^T A v

is interpreted as the dot product between the matrix A and the tensor product u v^T,

A • (u v^T)

— and dot product • on matrices is just from them being n×n vectors.

With that a lot of things are really nice now, e.g. interiors of ellipsoids correspond to intersections of halfspaces of matrices with the positive semidefinite cone. And halfspaces are simple to reason about and intersect!

This trick is also implicitly in the parent post, of course.

roger_2y ago

Well written piece, like that they walk through the problem formulation starting with the basics.

BTW is there a hackernews-type site or other aggregator that’s nothing but content like this? Maybe a subreddit? I’d love to read a few articles like this every day.

jakeogh2y ago

not quite an aggregator, but you might like https://rjlipton.wpcomstaging.com/

makerdiety2y ago

> So, our ellipse E is the set of all points y such that y = Ax where x is in the unit ball [and A is a transformation matrix].

So an ellipse is only something that exists following a transformation of a unit ball? So, these "unit balls" are the elemental atoms of this ellipsis physics? Technically speaking at least.

dellamonica2y ago

Every ellipse can be encoded by the matrix A (and the geometric concept is generalized to arbitrary dimensions).

Not sure I follow the physics analogy though. A unit ball is a specific case of an ellipse where A is the identity matrix. Perhaps the entries of A would be the atoms in this case as they uniquely shape it?

makerdiety2y ago

How about a software development analogy then?:

A unit ball along with the possibility of doing linear transformations on the unit ball are dependencies for constructing an ellipse.

The physics metaphor is meant to communicate that you can have different materials (like aluminum or air), depending on the combinations or parameters used for a particular ellipsis case. And the physics or dynamics of ellipses would be the study of how the combinations on the dependencies (that is, the atomic elements) create different ellipsis shapes and properties.

1 more reply

johnsutor2y ago

This is the premise behind some more recent self-supervised learning papers (see https://arxiv.org/abs/2303.03307). Turns out, it's a solid analog for classification tasks.

jjgreen2y ago

Nice short piece, reduces the geometric problem to a semidefinite program which can be solved by generic optimisation codes without drowning the reader in detail.

mycologos2y ago

How useful is reducing to a semidefinite program in reality? A fair amount of stuff seems to conclude with "now that we've reduced to an SDP, it's all polytime from here baby, so we're done modulo boring implementation details that nobody cares about". But I've tried and failed to understand how meaningful that polytime is in a practical sense. Anybody know?

cevi2y ago

In practice, even storing the matrices that show up in the problem formulation can be prohibitive, let alone running a solver. Often you need to take advantage of sparsity in a nontrivial way (see e.g. COSMO [1]), or take advantage of cases where the solution can be approximated by a low rank matrix, and recently there has been a trend in using first-order methods like ADMM rather than full interior-point algorithms. The review article [2] from 2019 gives a feel for how much work has gone into making large-scale semidefinite programming possible, but has this sentence in their conclusion section:

"Semidefinite programming is still far from being a mature technology like linear or quadratic programming."

[1] https://link.springer.com/article/10.1007/s10957-021-01896-x [2] https://www.annualreviews.org/doi/pdf/10.1146/annurev-contro...

nihzm2y ago

From my understanding nowadays SDP solvers are becoming well developed, even the open source ones. Also, from my limited experience it is usually the case that the problem being cast into an SDP has no other good ways of solving it, so polytime is better than no solution / NP solutions.

a_gnostic2y ago

Sweet. Now to implement this into orbital planning for KSP…

j / k navigate · click thread line to collapse

17 comments

mbostock2y ago

If you find this interesting, you might enjoy my write up of the Matoušek-Sharir-Welzl (MSW) algorithm used by D3’s circle-packing layout.

https://observablehq.com/@d3/d3-packenclose

I still haven’t quite figured out how to make D3’s implementation robust, though. Volodymyr Agafonkin’s robust-predicated would probably help… https://github.com/mourner/robust-predicates

fiforpg2y ago

There is a very nice short proof of John's ellipsoid theorem by Gruber and Schuster:

https://www.dmg.tuwien.ac.at/gruber/gruber_arbeiten/johnelli...

— one elegant trick I remember from there was that the value of a quadratic form with matrix A on vectors u and v (^T for transpose):

u^T A v

is interpreted as the dot product between the matrix A and the tensor product u v^T,

A • (u v^T)

— and dot product • on matrices is just from them being n×n vectors.

This trick is also implicitly in the parent post, of course.

roger_2y ago

Well written piece, like that they walk through the problem formulation starting with the basics.

BTW is there a hackernews-type site or other aggregator that’s nothing but content like this? Maybe a subreddit? I’d love to read a few articles like this every day.

jakeogh2y ago

not quite an aggregator, but you might like https://rjlipton.wpcomstaging.com/

makerdiety2y ago

> So, our ellipse E is the set of all points y such that y = Ax where x is in the unit ball [and A is a transformation matrix].

So an ellipse is only something that exists following a transformation of a unit ball? So, these "unit balls" are the elemental atoms of this ellipsis physics? Technically speaking at least.

dellamonica2y ago

Every ellipse can be encoded by the matrix A (and the geometric concept is generalized to arbitrary dimensions).

makerdiety2y ago

How about a software development analogy then?:

A unit ball along with the possibility of doing linear transformations on the unit ball are dependencies for constructing an ellipse.

1 more reply

johnsutor2y ago

This is the premise behind some more recent self-supervised learning papers (see https://arxiv.org/abs/2303.03307). Turns out, it's a solid analog for classification tasks.

jjgreen2y ago

Nice short piece, reduces the geometric problem to a semidefinite program which can be solved by generic optimisation codes without drowning the reader in detail.

mycologos2y ago

cevi2y ago

"Semidefinite programming is still far from being a mature technology like linear or quadratic programming."

[1] https://link.springer.com/article/10.1007/s10957-021-01896-x [2] https://www.annualreviews.org/doi/pdf/10.1146/annurev-contro...

nihzm2y ago

a_gnostic2y ago

Sweet. Now to implement this into orbital planning for KSP…

j / k navigate · click thread line to collapse