[QUOTE=ccorn;221024]You almost got it. It's not exactly what you stated, but related. Think even simpler. And some ingredient of standard deviation is in there.[/QUOTE]Method of least squares?

[QUOTE=lavalamp;221022]As an aside, just naturally thinking about an "equivalent rectangle" for a shape, I would suggest that the rectangle should have the same area as the original shape, and that it's four corners lie in such a way as to minimise the standard deviation of their distances to the corners of the original shape.

Though this is obviously a different direction to the one you have taken.[/QUOTE]
The area is not exactly preserved, though its relative change is quadratic in a parameter epsilon which you may define as (difference of diagonal lengths) / (sum of diagonal lengths) of the given quadrilateral. Edit: The quadrilateral depicted in post #3 has epsilon = 1/3, so the fitting rectangle [strike]should have (1-epsilon^2) = 8/9[/strike] has 1/(1-epsilon^2) = 9/8 the area of the quadrilateral. In this (quite extreme) example, the area grows by 1/8. Smaller epsilons result in (roughly) quadratically smaller area growth.

There is a way to preserving the area, but it requires "weighing" the distances, and that approach is slightly more complicated. Do the given problem first, then you can take off and perhaps rise to findings of area-preserving and barycenter-centered fitting rectangles.

[QUOTE=lavalamp;221025]Method of least squares?[/QUOTE]
YES. State it in its applied form here, and you have the key to questions (1) and (2) of this puzzle.

[QUOTE=lavalamp;221025]Method of least squares?[/QUOTE]
As said above, yes. Regrettably, no concretization has followed. After four days, I find that I have to spell it out myself.

Given the vertices [tex]\mathbb{x}_1\ldots\mathbb{x}_4[/tex] of a quadrilateral, find a rectangle with vertices [tex]\mathbb{x}_1'\ldots\mathbb{x}_4'[/tex] such that [tex]\mathbb{x}_1,\mathbb{x}_3[/tex] are diagonally opposite and [tex]S =^{\text{def}} \sum_{i=1}^4 \left|\mathbb{x}_i-\mathbb{x}_i'\right|^2 = \text{min.!}[/tex]

Note that this is a constrained optimization problem. The constraint is that the closed polygon [tex](\mathbb{x}_1'\ldots\mathbb{x}_4')[/tex] shall form a rectangle. This may be expressed most easily by the circle-of-Thales approach from post #17: Substitute
[tex]\begin{array}{rl}
\mathbb{x}_1' &= \mathbb{x}_{\text{M}} - \mathbb{v}_1\\
\mathbb{x}_2' &= \mathbb{x}_{\text{M}} - \mathbb{v}_2\\
\mathbb{x}_3' &= \mathbb{x}_{\text{M}} + \mathbb{v}_1\\
\mathbb{x}_4' &= \mathbb{x}_{\text{M}} + \mathbb{v}_2
\end{array}[/tex]
and take into account the constraint [tex]\left|\mathbb{v}_1\right|^2-\left|\mathbb{v}_2\right|^2 = 0[/tex].

The minimized error square sum [I]S[/I] is a measure for un-rectangleness. That is the answer to question (1). (You still have to find a suitable reference entity to compare with. That is question 3).

Answering question (2) is straightforward if you know how to do optimizations of the above form. Briefly, you have to look for stationary points of [tex]S_{\text{c}}(\mathbb{x}_{\text{M}},\mathbb{v}_1, \mathbb{v}_2,\lambda) = \sum_{i=1}^4 \left|\mathbb{x}_i-\mathbb{x}_i'\right|^2 + \lambda\,\left(\left|\mathbb{v}_1\right|^2-\left|\mathbb{v}_2\right|^2\right)[/tex] (substitute [tex]\mathbb{x}_1'\ldots\mathbb{x}_4'[/tex] on the right-hand side using [tex]\mathbb{x}_{\text{M}},\mathbb{v}_1,\mathbb{v}_2[/tex]).

Doing this exercise, you will quickly find [tex]\mathbb{x}_{\text{M}} = \frac{1}{4}\left(\mathbb{x}_1+\cdots+\mathbb{x}_4\right)[/tex], as recognized by lavalamp in post #15. Go on and find [tex]\mathbb{v}_1, \mathbb{v}_2[/tex], then interpret the result geometrically. (You will also have to find the optimal value for the Lagrange multiplier [tex]\lambda[/tex] for that).

Here is the figure of post #3, this time with the diagonals of the quadrilateral indicated.

Solution
 

If the problem was not interesting enough, how about its solution?

(1) Solved above, indicated by lavalamp. The minimized distance-squared sum is chosen as a measure for un-rectangleness.

(2) Midpoint found by lavalamp. The diagonals of the fitting rectangle are parallel to the diagonals of the given quadrilateral (i. e. diagonal directions are preserved), and the length of each fitting rectangle's diagonal is the arithmetic mean of the lengths of the given quadrilateral's diagonals (i. e. the sum of the diagonal lengths is preserved). These results can be found by analytically optimizing [tex]S_\text{c}[/tex].

(3) Now you have the fitting rectangle with side lengths [I]a[/I], [I]b[/I] and the minimized distance-squared sum [I]S[/I]. Without loss of generality, let [tex]a \ge b[/tex], thus [tex]b = \min\{a,b\}[/tex]. Now choose some dimensionless positive number [tex]\mu[/tex] and compare [I]S[/I] with [tex]\mu^2 b^2[/tex]. If and only if [I]S[/I] is greater, call the quadrilateral [I]not rectangular enough[/I]. Reducing [tex]\mu[/tex] reduces the tolerance of un-rectangleness. This test is clearly scale-invariant, and choosing [tex]b^2[/tex] as reference for [I]S[/I] ensures an upper bound of [tex]O(\mu)[/tex] for errors in angles and relative errors in opposite side lengths. Furthermore, choosing [tex]\mu < 1[/tex] excludes X-like shapes.