NM-method

{{Short description|Statistical NM-method}} thumb|right|alt=NM-method|NM-method The '''NM-method''' or '''Naszodi–Mendonca method''' is the operation that can be applied in statistics, econometrics, economics, sociology, and demography to construct counterfactual contingency tables. The method finds the matrix <math>X</math> (<math> X \in \mathbb{R}^{n \times m } </math>) which is "closest" to matrix <math>Z</math> (<math> Z \in \mathbb{N}^{n \times m } </math> called the seed table) in the sense of being ranked the same but with the row and column totals of a target matrix <math>Y</math> <math>( Y \in \mathbb{N}^{n \times m }) </math>. While the row totals and column totals of <math>Y</math> are known, matrix <math>Y</math> itself may not be known.

Since the solution for matrix <math>X</math> is unique, the NM-method is a function: <math> X=\text{NM}(Z, Y e^T_m, e_nY): \mathbb{N}^{n \times m} \times \mathbb{N}^{n} \times \mathbb{N}^{m} \mapsto \mathbb{R}^{n \times m}</math>, where <math>e_n</math> is a row vector of ones of size <math>1\times n</math>, while <math>e^T_m</math> is a column vector of ones of size <math>m\times 1</math>.

The NM-method was developed by Naszodi and Mendonca (2023)<ref name=NM2021>{{Cite journal |last1=Naszodi |first1=A. |last2=Mendonca |first2=F. |year=2023 |title=A new method for identifying the role of marital preferences at shaping marriage patterns |journal=Journal of Demographic Economics |volume=1 |issue=1 |pages=1–27 |doi=10.1017/dem.2021.1 |doi-access=free}}</ref> (and first applied by Naszodi and Mendonca (2019)<ref name=NM2019>{{Cite journal |last1=Naszodi |first1=A. |last2=Mendonca |first2=F. |year=2019 |title=Like marries like |journal=Fairness Policy Brief Series| url=https://knowledge4policy.ec.europa.eu/sites/default/files/fairness_pb2019_assortative_mating_jrc115102.pdf | archive-url=https://web.archive.org/web/20230330004917/https://joint-research-centre.ec.europa.eu/scientific-activities-z/fairness-0/fairness-policy-briefs-series_en |archive-date=2023-03-30 }}</ref> to solve for matrix <math>X</math> in problems, where matrix <math>\boldsymbol{Z}</math> is not a sample from the population characterized by the row totals and column totals of matrix <math>Y</math>, but represents another population.

Their application aimed at quantifying intergenerational changes in the strength of educational homophily and thus measuring the historical change in social inequality between different educational groups in the US between 1980 and 2010. The trend in inequality was found to be U-shaped, supporting the view that with appropriate social and economic policies inequality can be reduced.

==Definition of matrix ranking== The closeness between two matrices of the same size can be defined in several ways. The Euclidean distance, and the Kullback–Leibler divergence are two well-known examples.

The NM-method is consistent with a definition relying on the ordinal Liu–Lu index<ref name=LL2006>{{cite journal |last1=Liu |first1=H. |last2=Lu |first2=J. |year=2006 |title=Measuring the degree of assortative mating |journal=Economics Letters |volume=92 |issue=3 |pages=317–322|doi=10.1016/j.econlet.2006.03.010 |doi-access=free}}</ref> which is the slightly modified version of the Coleman-index defined by Eq. (15) in James Coleman (1958).<ref name=Coleman1958>{{cite journal |last=Coleman |first=J. |year=1958 |title=Relational Analysis: The Study of Social Organizations with Survey Methods |journal=Human Organization |volume=17 |issue=4 |pages=28–36 |doi=10.17730/humo.17.4.q5604m676260q8n7 }}</ref> According to this definition, matrix <math>X</math> is "closest" to matrix <math>Z</math>, if their Liu–Lu values are the same. In other words, if they are ranked the same by the ordinal Liu–Lu index.

If <math>Z</math> is a 2×2 matrix, its {{Anchor|svLL}} scalar-valued Liu–Lu index is defined as

<math> \text{LL}(Z)=\frac{Z_{1,1}-Q^-(Z_{1,1})}{ \text{min}(Z_{1,.} , Z_{.,1}) -Q^-(Z_{1,1})} </math> , where <math> Z_{1,.}= Z_{1,1}+ Z_{1,2} </math>; <math>Z_{.,1}= Z_{1,1}+ Z_{2,1} </math>; <math>Z_{.,.}=Z_{.,1}+Z_{1,.}</math>; <math>Q(Z_{1,1})={Z_{1,.} Z_{.,1}}/{Z_{.,.}} </math>; <math>Q^-(Z_{1,1})=int[Q(Z_{1,1})] </math>.

Following James Coleman (1958),<ref name=Coleman1958/> this index is interpreted as the “actual minus expected over maximum minus minimum”, where <math>Z_{1,1}</math> is the actual value of the <math>1,1 </math> entry of the seed matrix <math> Z</math>; <math> Q^-</math> is its expected (integer) value under the counterfactual assumptions that the corresponding row total and column total of <math> Z</math> are predetermined, while its interior is random. Also, <math> Q^-</math> is its minimum value if the association between the row variable and the column variable of <math> Z</math> is non-negative. Finally, <math> \text{min}(Z_{1,.} , Z_{.,1}) </math> is the maximum value of <math>Z_{1,1}</math> (<math> Z \in \mathbb{N}^{n \times m } </math>) for given row total <math>Z_{1,.}</math> and column total <math>Z_{.,1}</math>.

For matrix <math>Z</math> of size n×m (<math> n \geq 2</math>, <math> m \geq 2</math>), the Liu–Lu index was generalized by Naszodi and Mendonca (2023)<ref name=NM2021/> to a matrix-valued index. One of the {{Anchor|prc}} preconditions for the generalization is that the row variable and the column variable of matrix <math>Z</math> have to be ordered. Equating the generalized, matrix-valued Liu–Lu index of <math>Z</math> with that of matrix <math>X</math> is equivalent to dichotomizing their ordered row variable and ordered column variable {{Anchor|dichotpossible}} in <math> (n-1) \times (m-1)</math> ways by explointing the ordered nature of the row and column variables. Than, equating the original, scalar-valued Liu–Lu indices of the 2×2 matrices obtained with the dichotomizations. I.e., for any pair of <math>i,j</math> (<math> i \in \{1, \ldots, n-1 \} </math>, and <math>j \in \{1, \ldots, m-1 \}</math>) the restriction <math>\text{LL}(V_i X W^T_j) = \text{LL}(V_i Z W^T_j) </math> is imposed, where <math>V_i </math> is the <math>2 \times n </math> matrix <math display="block"> V_i=\begin{bmatrix} \color{red}1 & \color{red}\cdots & \color{red}1 & \color{blue}0 & \color{blue}\cdots & \color{blue}0 \\ \color{red}0 & \color{red}\cdots & \color{red}0 & \color{blue}1 & \color{blue}\cdots & \color{blue}1 \end{bmatrix} </math> with its {{font color|red|first block}} being of size <math>2 \times i </math>, and its {{font color|blue|second block}} being of size <math>2 \times (n-i) </math>. Similarly, <math> W^T_j </math> is the <math>m \times 2 </math> matrix given by the transpose of <math display="block"> W_j=\begin{bmatrix} \color{red}1 & \color{red}\cdots & \color{red}1 & \color{blue}0 & \color{blue}\cdots & \color{blue}0 \\ \color{red}0 & \color{red}\cdots & \color{red}0 & \color{blue}1 & \color{blue}\cdots & \color{blue}1 \end{bmatrix} </math> with its {{font color|red|first block}} being of size <math>2 \times j </math>, and its {{font color|blue|second block}} being of size <math>2 \times (m-j) </math>.

==Constraints on the row totals and column totals== Matrix <math>X</math> should satisfy not only <math> \text{LL}(V_i X W^T_j)= \text{LL}(V_i Z W^T_j) </math> but also the pair of constraints on its row totals and column totals: <math>Xe^T_m=Ye^T_m </math> and <math>e_n X=e_n Y </math>.

==Solution== Assuming that <math> \text{LL}(V_i Z W^T_j) \geq 0 </math> for all pairs of <math>i,j</math> (where <math> i \in \{1, \ldots, n-1 \} </math>, and <math>j \in \{1, \ldots, m-1 \}</math>), the solution for <math>X</math> is unique, deterministic, and given by a closed-form formula.<ref name=NM2021/>

For matrices <math>Y</math> and <math>Z</math> of size <math>\boldsymbol{2\times 2}</math>, the {{Anchor|x11}} solution is

<math>X_{1,1} = \frac{\left[ Z_{1,1} - \text{int}\left({Z_{1,\cdot} Z_{\cdot,1}}/{Z_{\cdot,\cdot}}\right)\right] \left[{\text{min}\left(Y_{1,\cdot} , Y_{\cdot,1} \right)- \text{int}\left({Y_{1,\cdot}Y_{\cdot,1}}/ {Y_{\cdot,\cdot}} \right) }\right] }{\text{min}\left(Z_{1,\cdot}, Z_{\cdot,1} \right)- \text{int}\left({Z_{1,\cdot}Z_{\cdot,1}}/{Z_{\cdot,\cdot}} \right) } +\text{int}\left({Y_{1,\cdot} Y_{\cdot,1}}/{Y_{\cdot,\cdot}}\right) </math>.

The other 3 cells of <math> X </math> are uniquely determined by the row totals and column totals. So, this is how the NM-method works for 2×2 seed tables.

For <math>Y </math>, and <math> Z </math> matrices of size <math>\boldsymbol{n\times m}</math> (<math> n \geq 2</math>, <math> m \geq 2 </math>), the solution is obtained by dichotomizing their ordered row variable and ordered column variable in all possible meaningful ways before solving <math>(n-1)(m-1) </math> number of problems of 2×2 form. Each problem is defined for an <math> i,j </math> pair (<math> i \in \{1,..., n-1\} </math> and <math> j \in \{1,..., m-1\} </math>) with <math> \text{LL}(V_i X W^T_j)= \text{LL}(V_i Z W^T_j) </math>, and the target row totals and column totals: <math> V_i X e^T_{m}= V_i Y e^T_{m} </math>, and <math> e_{n} X W^T_j = e_{n} Y W^T_j </math>, respectively. Each problem is to be solved separately by the formula for <math>X_{1,1} </math>. The set of solutions determine <math>(n-1) (m-1) </math> number of entries of matrix <math>X </math>. Its remaining <math> m+n-1</math> elements are uniquely determined by the target row totals and column totals.

Next, let us see how the NM-method works if matrix <math>Z </math> is such that the second precondition of <math>\boldsymbol{\text{LL}(V_i Z W^T_j) \geq 0}</math> is not met for <math> \boldsymbol{\forall i,j}</math>.

If <math>\boldsymbol{\text{LL}(V_i Z W^T_j) \leq 0 } </math> for all pairs of <math> \boldsymbol{i,j}</math>, the solution for <math>X</math> is also unique, deterministic, and given by a closed-form formula. However, the corresponding concept of matrix ranking is slightly different from the one discussed above. Liu and Lu (2006)<ref name=LL2006/> define it as <math>\text{LL}^-(Z)=\frac{Z_{1,1}-Q^+(Z_{1,1})}{ Q^+(Z_{1,1})- max(0; Z_{1,.}-Z_{.,2}) } </math> , where <math>Z_{.,2}= Z_{1,2}+ Z_{2,2} </math>; <math>Q^+(Z_{1,1})</math> is the smallest integer being larger than or equal to <math>Q</math>.

{{Anchor|preNM}} Finally, neither the NM-method, nor <math> \boldsymbol{\text{LL}(Z)}</math> is defined if <math> \exist (i,j)</math> pair such that <math>\boldsymbol{\text{LL}(V_i Z W^T_j) > 0 } </math>, while for another pair of <math>k,l (\neq i,j) </math> <math>\boldsymbol{\text{ LL}(V_k Z W^T_l) < 0} </math>.

==A numerical example== Consider the following {{font color|green|matrix}} <math>\color{green}Z </math> complemented with its row totals and column totals and the targets, i.e., the {{font color|orange|row totals}} and {{font color|orange|column totals of matrix}} <math> \color{orange}Y </math>: {| class="wikitable" style="margin:1em auto;" !Z !1 !2 !3 !4 !TOTAL !TARGET |- |1 |{{font color|green|120}} |{{font color|green|70}} |{{font color|green|30}} |{{font color|green|20}} |240 |{{font color|orange|400}} |- |2 |{{font color|green|50}} |{{font color|green|100}} |{{font color|green|50}} |{{font color|green|35}} |235 |{{font color|orange|300}} |- |3 |{{font color|green|30}} |{{font color|green|40}} |{{font color|green|75}} |{{font color|green|40}} |185 |{{font color|orange|150}} |- |4 |{{font color|green|10}} |{{font color|green|20}} |{{font color|green|30}} |{{font color|green|80}} |140 |{{font color|orange|150}} |- |TOTAL |210 |230 |185 |175 |800 | |- |TARGET |{{font color|orange|400}} |{{font color|orange|300}} |{{font color|orange|200}} |{{font color|orange|100}} | |1,000 |- |}

As a first step of the NM-method, {{font color|green|matrix}} <math>\color{green} Z </math> is multiplied by the <math>\boldsymbol{V_i}</math>, and <math>\boldsymbol{W^T_j} </math> matrices for each pair of <math>i,j</math> (<math> i \in \{1, 2, 3 \} </math>, and <math>j \in \{1, 2, 3 \} </math>). It yields the following 9 matrices of size 2×2 with their target row totals and column totals:

{| class="wikitable" style="margin:1em auto;" |+ |- | {| class="wikitable" |+ |- !<math>i=1, j=1</math> !!1 !!2 !!TOTAL !!TARGET |- |1 |120 |120 |240 |400 |- |2 |90 |470 |560 |600 |- |TOTAL |210 |590 |800 | |- |TARGET |400 |600 | |1,000 |- |} || {| class="wikitable" |+ |- ! <math>i=1, j=2</math> !!1 !!2 !!TOTAL !!TARGET |- | 1 || 190 ||50 || 240 || 400 |- | 2 || 250 || 30 || 560 || 600 |- | TOTAL || 440 || 360 || 800 || |- | TARGET|| 700 || 300 || || 1,000 |- |} || {| class="wikitable" ! <math>i=1, j=3</math> ||1 || 2 || TOTAL || TARGET |- | 1 || 220 ||20 || 240 || 400 |- | 2 ||405 || 155 || 560 || 600 |- | TOTAL || 625 || 175 || 800 || |- | TARGET|| 900 || 100 || || 1,000 |- |} |- | {| class="wikitable" ! <math>i=2, j=1</math> ||1 || 2 || TOTAL || TARGET |- | 1 || 170 || 305 || 475 || 700 |- | 2 ||40 || 285 || 325 || 300 |- | TOTAL || 210 ||590 || 800 || |- | TARGET|| 400 || 600 || || 1,000 |- |} || {| class="wikitable" ! <math>i=2, j=2</math> ||1 || 2 || TOTAL || TARGET |- | 1 || 340 || 135 || 475 || 700 |- | 2 ||100 || 225 || 325 || 300 |- | TOTAL || 440 ||360 || 800 || |- | TARGET|| 700 || 300 || || 1,000 |- |} || {| class="wikitable" !<math>i=2, j=3</math> !1 !2 !TOTAL !TARGET |- |1 |420 |55 |475 |700 |- |2 |205 |120 |325 |300 |- |TOTAL |625 |175 |800 | |- |TARGET |900 |100 | |1,000 |- |} |- | {| class="wikitable" !<math>i=3, j=1</math> !1 !2 !TOTAL !TARGET |- |1 |200 |460 |660 |850 |- |2 |10 |130 |140 |150 |- |TOTAL |210 |590 |800 | |- |TARGET |400 |600 | |1,000 |- |} || {| class="wikitable" ! <math>i=3, j=2</math> ||1 || 2 || TOTAL || TARGET |- | 1 || 410 || 250 || 660 || 850 |- | 2 ||30 || 110 || 140 || 150 |- | TOTAL || 440 ||360 || 800 || |- | TARGET|| 700 || 300 || || 1,000 |- |} || {| class="wikitable" ! <math>i=3, j=3</math> ||1 || 2 || TOTAL || TARGET |- | 1 || 565 || 95 || 660 || 850 |- | 2 ||60 || 80 || 140 || 150 |- | TOTAL || 625 ||175 || 800 || |- | TARGET|| 900 || 100 || || 1,000 |- |} |}

The next step is to calculate the generalized matrix-valued Liu–Lu index <math>\text{LL}({Z})</math>, (where <math>\text{LL}({Z})_{i,j}=\text{LL}(V_i Z W^T_j)</math>) by applying the formula of the original scalar-valued Liu–Lu index to each of the 9 matrices: {| class="wikitable" style="margin:1em auto;" ! <math>\text{LL(Z)}</math> ||<math>j=1</math> || <math>j=2</math> || <math>j=3</math> |- | <math>i=1</math> || 0.39 || 0.54 || 0.62 |- | <math>i=2</math> || 0.53 || 0.44 || 0.47 |- | <math>i=3</math> || 0.73 || 0.61 || 0.45 |- |}

Apparently, matrix <math>\text{LL}(Z)</math> is positive. Therefore, the NM-method is defined. Solving each of the 9 problems of the 2×2 form yields 9 entries of the <math>X</math> matrix. Its other 7 entries are uniquely determined by the target row totals and column totals. The solution for <math>\boldsymbol{X}</math> is: {| class="wikitable" style="margin:1em auto;" !<math>{X}</math> !1 !2 !3 !4 !TOTAL |- |1 |253.1 |91.4 |40.5 |15.1 |400 |- |2 |91.1 |147.1 |39.8 |21.9 |300 |- |3 |39.6 |36.8 |64.2 |9.3 |150 |- |4 |16.2 |24.7 |55.5 |53.6 |150 |- |TOTAL |400 |300 |200 |100 |1,000 |- |}

==Another numerical example taken from Abbott et al.(2019)== Consider the following {{font color|green|matrix}} <math>\color{green}Z </math> complemented with its row totals and column totals and the targets,{{Anchor|Abbott_Z}} i.e., the {{font color|orange|row totals}} and {{font color|orange|column totals of matrix}} <math> \color{orange}Y </math>: {| class="wikitable" style="margin:1em auto;" !Z !1 !2 !3 !TOTAL !TARGET |- |1 |{{font color|green|1,070}} |{{font color|green|270}} |{{font color|green|20}} |1,360 |{{font color|orange|1,600}} |- |2 |{{font color|green|300}} |{{font color|green|4,980}} |{{font color|green|560}} |5,840 |{{font color|orange|5,900}} |- |3 |{{font color|green|20}} |{{font color|green|420}} |{{font color|green|2,360}} |2,800 |{{font color|orange|2,500}} |- |TOTAL |1,390 |5,670 |2,940 |10,000 | |- |TARGET |{{font color|orange|1,390}} |{{font color|orange|5,670}} |{{font color|orange|2,940}} | |10,000 |- |}

As a first step of the NM-method, {{font color|green|matrix}} <math>\color{green} Z </math> is multiplied by the <math>\boldsymbol{V_i}</math>, and <math>\boldsymbol{W^T_j} </math> matrices for each pair of <math>i,j</math> (<math> i \in \{1, 2 \} </math>, and <math>j \in \{1, 2 \} </math>). It yields the following 4 matrices of size 2×2 with their target row totals and column totals:

{| class="wikitable" style="margin:1em auto;" |+ |- | {| class="wikitable" |+ |- !<math>i=1, j=1</math> !!1 !!2 !!TOTAL !!TARGET |- |1 |1,070 |290 |1,360 |1,600 |- |2 |320 |8,320 |8,640 |8,400 |- |TOTAL |1,390 |8,610 |10,000 | |- |TARGET |1,390 |8,610 | |10,000 |- |} || {| class="wikitable" |+ |- ! <math>i=1, j=2</math> !!1 !!2 !!TOTAL !!TARGET |- | 1 || 1,340 || 20 || 1,360 || 1,600 |- | 2 || 5,720 || 2,920 || 8,640 || 8,400 |- | TOTAL || 7,060 || 2,940 || 10,000 || |- | TARGET|| 7,060 || 2,940 || || 10,000 |- |} |- | {| class="wikitable" ! <math>i=2, j=1</math> ||1 || 2 || TOTAL || TARGET |- | 1 || 1,370 || 5,830 || 7,200 || 7,500 |- | 2 || 20 || 2,780 || 2,800 || 2,500 |- | TOTAL || 1,390 ||8,610 || 10,000 || |- | TARGET|| 1,390 ||8,610|| || 10,000 |- |} || {| class="wikitable" ! <math>i=2, j=2</math> ||1 || 2 || TOTAL || TARGET |- | 1 || 6,620 || 580 || 7,200 || 7,500 |- | 2 ||440 || 2,360 || 2,800 || 2,500 |- | TOTAL || 7,060 ||2,940 || 10,000 || |- | TARGET|| 7,060 || 2,940 || || 10,000 |- |} |}

The next step is to calculate the generalized matrix-valued Liu–Lu index <math>\text{LL}({Z})</math>, (where <math>\text{LL}({Z})_{i,j}=\text{LL}(V_i Z W^T_j)</math>) by applying the formula of the original scalar-valued Liu–Lu index to each of the 4 matrices: {| class="wikitable" style="margin:1em auto;" ! <math>\text{LL(Z)}</math> ||<math>j=1</math> || <math>j=2</math> |- | <math>i=1</math> || 0.75 || 0.95 |- | <math>i=2</math> || 0.95 || 0.78 |- |}

Apparently, matrix <math>\text{LL}(Z)</math> is positive. Therefore, the NM-method is defined. Solving each of the 4 problems of the 2×2 form yields 4 entries of the <math>X</math> matrix. Its other 5 entries are uniquely determined by the target row totals and column totals. {{Anchor|Abbott_X}} The solution for <math>\boldsymbol{X}</math> is: {| class="wikitable" style="margin:1em auto;" !<math>{X}</math> !1 !2 !3 !TOTAL |- |1 |1,101 |476 |24 |1,600 |- |2 |271 |4,819 |809 |5,900 |- |3 |18 |375 |2,107 |2,500 |- |TOTAL |1,390 |5,670 |2,940 |10,000 |- |}

==Implementation== The NM-method is implemented in Excel,<ref name=code>{{Cite journal|title=Code for A New Method|year=2021 |doi=10.17632/x2ry7bcm95.2 |url=http://dx.doi.org/10.17632/x2ry7bcm95.2|last1=Naszodi |first1=Anna |last2=Mendonca |first2=Francisco |volume=2 |publisher=Mendeley }}</ref> Visual Basic,<ref name=code/> R,<ref name=code/> and also in Stata.<ref name=code_GNM>{{Cite book|title=Code for "A New Method for Identifying What Cupid's Invisible Hand Is Doing. Is It Spreading Color Blindness While Turning Us More "Picky" About Spousal Education?"|year=2023 |doi=10.17632/95k6mmrxvg |chapter-url=http://dx.doi.org/10.17632/95k6mmrxvg|last1=Naszodi |first1=Anna |last2=Mendonca |first2=Francisco |chapter=Assortative Mating |volume=2 |publisher=Mendeley }}</ref>

==Applications== The NM-method can be applied to study various phenomena including assortative mating, intergenerational mobility as a type of social mobility,<ref name="NC2024">{{Cite journal|last1=Naszodi|first1=A. |last2=Cuccu|first2=L.|year=2024|title=Are high school degrees and university diplomas equally heritable in the US? A new measure of relative intergenerational mobility|journal=Journal of Applied Economics|volume=28|issue=3|article-number=2432803 |doi=10.1080/15140326.2024.2432803}}</ref> residential segregation, recruitment and talent management.

In all of these applications, matrices <math> X </math>, <math> Y </math>, and <math> Z </math> represent joint distributions of one-to-one matched entities (e.g. husbands and wives, or first born children and mothers, or dwellings and main tenants, or CEOs and companies, or chess instructors and their most talended students) characterized either by a dichotomous categorical variable (e.g. taking values vegetarian/non-vegetarian, Grandmaster/or not), or an ordered multinomial categorical variable (e.g. level of final educational attainment, skiers' ability level, income bracket, category of rental fee, credit rating, FIDE titles). Although the NM-method has a wide range of applicability, all the examples to be presented next are about assortative mating along the education level. In these applications, the two preconditions (of ordered trait variable, and positive assortative mating in all educational groups) are not debated to be met.

Assume that {{Anchor|1a}} matrix <math> Z </math> characterizes the joint educational distribution of husbands and wives in Zimbabwe, while matrix <math> Y </math> characterizes the same in Yemen. Matrix <math> X </math> to be constructed with the NM-method tells us what would be the joint educational distribution of couples in Zimbabwe, if the educational distributions of husbands and wives were the same as in Yemen, while the overall desire for homogamy (also called as aggregate marital preferences in economics, or marital matching social norms/social barriers in sociology) were unchanged.

In a {{Anchor|2a}} second application, matrices <math> Z </math> and <math> Y </math> characterize the same country in two different years. Matrix <math> Z </math> is the joint educational distribution of American newlyweds in 2040, where the husbands are from Generation Z and being young adults when observed. Matrix <math> Y </math> is the same but for Generation Y observed in year 2024. By constructing matrix <math> X </math>, one can study in the future what would be the educational distribution among the just married American young couples if they sorted into marriages the same way as the males in Generation Z and their partners do, while the education level were the same as among the males in Generation Y and their partners.

In a {{Anchor|3a}} third application, matrices <math> Z </math> and <math> Y </math> characterize again the same country in two different years. In this application, matrix <math> Z </math> is the joint educational distribution of Portuguese young couples (where the male partners' age is between 30 and 34 years) in 2011. And <math> Y </math> is the same but it is observed in year 1981. One may aim to construct matrix <math> X </math> in order to study what would have been the educational distribution of Portuguese young couples if they had sorted into marriages like their peers did in 2011, while their gender-specific educational distributions were the same as in 1981.

In each of the first two applications, matrix <math> X </math> represents a counterfactual joint distribution. It can be used to quantify certain ceteris paribus effects. More precisely, to quantify on a cardinal scale the difference between the directly unobservable degree of marital sorting in Zimbabwe and Yemen, or in Generation Z and Generation Y with a counterfactual decomposition. For the decomposition, the counterfactual table <math> X </math> is used to calculate the contribution of each of the driving forces (i.e., the observed structural availability of potential partners with various education levels determining the opportunities at the population level; and the unobservable non-structural drivers, e.g., aggregate matching preferences, desires, norms, barriers) and that of their interaction (i.e., the effect of changes in aggregate preferences/desires/norms/barriers due to changes in structural availability) to an observable cardinal scaled statistics (e.g. the share of educationally homogamous couples).

The third application was used by Naszodi and Mendonca (2023)<ref name=NM2021/> as an example for a non-sense counterfactual: the education level has changed so drastically in Portugal over the three decades studied that this counterfactual is impossible to be obtained. Surprisingly, a method, which was commonly used in the assortative mating literature until recently, hallucinates a solution for the impossible counterfactual in the third example, while the NM-method rejects to construct it.

==Some features of the NM-method== First, the NM-method does not yield a meaningful solution if it reaches the limit of its applicability.<ref name=NM2021/> For instance, in the third application, the NM-method signals with a negative entry in matrix <math> X </math> that the counterfactual is impossible (see: AlternativeMethod_US_1980s_2010s_age3035_main.xls Sheet PT_A1981_P2011_Not_meaningful).<ref name=code/> In this respect, the NM-method is similar to the linear probability model that signals the same with a predicted probabiity outside the unit interval <math> [0,1] </math>.

Second, the NM-method commutes with merging neighboring categories of the row variable and that of the column variable:<ref name=NM2021/> <math> \text{NM}(M_r Z, M_r Y e^T_m, M_r e_nY)=M_r \text{NM}(Z, Y e^T_m, e_nY)</math>, where <math>M_r</math> is the row merging matrix of size <math>(n-1) \times n</math>; and <math> \text{NM}(Z M_c, Y e^T_m M_c, e_n Y M_c)=\text{NM}(Z, Y e^T_m, e_nY) M_c</math>, where <math>M_c</math> is the column merging matrix of size <math>m \times (m-1)</math>.

Third, the NM-method works even if there are zero entries in matrix <math>Z</math>.<ref name=NM2021/>

==Comparison with the IPF== The iterative proportional fitting procedure (IPF) is also a function:<ref>Sinkhorn, Richard (1964). “A Relationship Between Arbitrary Positive Matrices and Doubly Stochastic Matrices”. In: Annals of Mathematical Statistics 35.2, pp. 876–879.</ref><ref>Bacharach, Michael (1965). “Estimating Nonnegative Matrices from Marginal Data”. In: International Economic Review 6.3, pp. 294–310.</ref><ref name=Bishop1967>{{Cite journal |last=Bishop |first=Y. M. M. | year=1967 |title=Multidimensional contingency tables: cell estimates |journal=PhD Thesis. Harvard University}}</ref><ref name=Fienberg1970>{{Cite journal |last=Fienberg |first=S. E.|author-link=Stephen Fienberg |year=1970 |title=An Iterative Procedure for Estimation in Contingency Tables |journal=Annals of Mathematical Statistics |volume=41 |issue=3 |pages=907–917 |mr=266394 | zbl = 0198.23401 | jstor = 2239244 |doi=10.1214/aoms/1177696968 |doi-access=free}}</ref><math> \text{IPF}(Z, Y e^T_m, e_nY): \mathbb{R}^{n \times m} \times \mathbb{R}^{n} \times \mathbb{R}^{m} \mapsto \mathbb{R}^{n \times m}</math>. It is the operation of finding the fitted matrix <math>\boldsymbol{F}</math> (<math> F \in \mathbb{R}^{n \times m}</math>) which fulfills a set of conditions similar to those met by matrix <math>X</math> constructed with the NM-method. E.g., matrix <math>F</math> is the closest to matrix <math>\boldsymbol{Z}</math> but with the row and column totals of the target matrix <math>\boldsymbol{Y}</math>.

However, there are differences between the IPF and the NM-method. The IPF defines closeness of matrices of the same size by the cross-entropy, or the Kullback–Leibler divergence.<ref>Kullback S. and Leibler R.A. (1951) On information and sufficiency, Annals of Mathematics and Statistics, 22 (1951) 79-86.</ref> Accordingly, the IPF compatible concept of distance between the 2×2 matrices <math>F</math> and <math>Z</math> is zero, if their crossproduct ratios<ref name=Fienberg1970/> (also known as the odds ratio) are the same: <math> { F_{1,1} F_{2,2} }/ { F_{1,2} F_{2,1} } ={ Z_{1,1} Z_{2,2} }/ { Z_{1,2} Z_{2,1} } </math>.<ref name=Naszodi2023>{{Cite arXiv |last=Naszodi |first=A. |year=2023 |title=The iterative proportional fitting algorithm and the NM-method: solutions for two different sets of problems |class=econ.GN |eprint=2303.05515}}</ref> To recall, the NM-method's condition for equal ranking of matrices <math>X</math> and <math>Z</math> is <math> \text{LL}(X)=\frac{X_{1,1}-int[{X_{1,.} X_{.,1}}/{X_{.,.}}]}{ \text{min}(X_{1,.} , X_{.,1}) -int[{X_{1,.} X_{.,1}}/{X_{.,.}}]} =\frac{Z_{1,1}-int[{Z_{1,.} Z_{.,1}}/{Z_{.,.}}]}{ \text{min}(Z_{1,.} , Z_{.,1}) -int[{Z_{1,.} Z_{.,1}}/{Z_{.,.}}]} =\text{LL}(Z)</math>.

The following numerical example highlights that the IPF and the NM-method are not identical: <math> \text{IPF}(Z, Y e^T_m, e_nY) \neq \text{NM}(Z, Y e^T_m, e_nY) </math>. Consider the {{font color|Green|matrix}} <math>\color{Green}Z</math> with its {{font color|Orange|targets}}: {| class="wikitable" style="margin:1em auto;" ! !1 !2 !TOTAL !TARGET |- |1 |{{font color|Green|450}} |{{font color|Green|150}} |600 |{{font color|Orange|1,050}} |- |2 |{{font color|Green|50}} |{{font color|Green|350}} |400 |{{font color|Orange|450}} |- |TOTAL |500 |500 | | |- |TARGET |{{font color|Orange|1,000}} |{{font color|Orange|500}} | |1,500 |- |}

The NM-method yields the following matrix <math>X</math>: {| class="wikitable" style="margin:1em auto;" ! <math>X </math> !1 !2 !TOTAL |- |1 |925 |125 |1,050 |- |2 |75 |375 |450 |- |TOTAL |1,000 |500 |1,500 |- |} Whereas the solution for matrix <math>F</math> obtained with the IPF is: {| class="wikitable" style="margin:1em auto;" ! <math>F</math> !1 !2 !TOTAL |- |1 |900 |150 |1,050 |- |2 |100 |350 |450 |- |TOTAL |1,000 |500 |1,500 |- |}

The IPF is equivalent to the maximum likelihood estimator<ref name=Bishop1967/> of a joint population distribution, where matrix <math>F</math> (the estimate for the joint population distribution) is calculated from matrix <math>Z</math>, the observed joint distribution in a random sample taken from the population characterized by the row totals and column totals of matrix <math>Y</math>. In contrast to the problem solved by the IPF, matrix <math>Z</math> is not sampled from this population in the problem that the NM-method was developed to solve. In fact, in the NM-problem, matrices <math>Z</math> and <math>Y</math> characterize two different populations (either observed simultaneously like in the application for Zimbabwe and Yemen, or observed in two different points in time like in its application for the populations of Generation Z and Generation Y). This difference facilitates the choice between the NM-method and the IPF in empirical applications.<ref name=Naszodi2023/>

Deming and Stephan(1940),<ref name=DS1940>{{Cite journal |last1=Deming |first1=W. E.|author-link=W. Edwards Deming |last2=Stephan |first2=F. F. |year=1940 |title=On a Least Squares Adjustment of a Sampled Frequency Table When the Expected Marginal Totals are Known |journal=Annals of Mathematical Statistics |volume=11 |issue=4 |pages=427–444 |mr=3527 |doi=10.1214/aoms/1177731829 |doi-access=free}}</ref> the inventors of the IPF, illustrated the application of their method on a classic maximum likelihood estimation problem, where matrix <math>Z</math> was sampled from the population characterized by the row totals and column totals of matrix <math>Y</math>. They were aware of the fact that in general, the IPF is not suitable for counterfactual predictions: they explicitly warned that their algorithm is “not by itself useful for prediction” (see Stephan and Deming 1940 p.&nbsp;444).<ref name=DS1940/><ref name=Naszodi2023/>

In addition, the domains are different for which the IPF and the NM-method yield solutions. First, unlike the NM-method, the IPF does not provide a solution for all seed tables <math>{Z}</math> with zero entries (Csiszár (1975)<ref name=csiszar1975>{{Cite journal |last=Csiszár |first=I.|author-link=Imre Csiszár |year=1975 |title=''I''-Divergence of Probability Distributions and Minimization Problems |journal=Annals of Probability |volume=3 |issue=1 |pages=146–158 |mr=365798 | zbl = 0318.60013 | jstor = 2959270 |doi=10.1214/aop/1176996454 |doi-access=free}}</ref> found necessary and sufficient conditions for applying the IPF with general tables having zero entries). Second, the precondition of the NM-method (of either <math>\boldsymbol{\text{LL}(Z)\geq 0}</math> or <math>\boldsymbol{\text{LL}(Z)\leq 0}</math>) is not a precondition for the applicability of the IPF. {{Anchor|ImpPort}} Third, unlike the NM, the IPF provides a seeminly meaningful solution for pairs of matrices <math>{Z}</math> and <math>{Y}</math> defining impossible counterfactuals, such as the pair of matrices in our third numerical example with Portugal (Naszodi 2025<ref name="naszodi2025">{{Cite book|author=Naszodi|first=A.|year=2025|title=New Methods for Measuring Inequality by Analyzing Assortative Mating|url=https://link.springer.com/book/9783031982767|series=The Springer Series on Demographic Methods and Population Analysis|publisher=Springer Cham|isbn=978-3-031-98276-7}}</ref>). Finally, unlike the NM, the IPF does not commute with the operation of merging neighboring categories of the row variable and that of the column variable as it is illustrated with a numerical example in Naszodi(2023) (see page 10).<ref name=Naszodi2023WP>{{Cite arXiv |last=Naszodi |first=A. |year=2023 |title=What do surveys say about the historical trend of inequality and the applicability of two table-transformation methods? |class=econ.GN | eprint=2303.05895}}</ref> For this reason, the transformed table obtained with the IPF can be sensitive to the choice of the number of trait categories.

Kenneth Macdonald (2023)<ref name=KM2023>{{cite journal |last1=Macdonald |first1=K. |year=2023 |title=The marginal adjustment of mobility tables, revisited |journal=OSF |pages=1–19 |url=https://osf.io/z4u2d/download}}</ref> is at ease with the conclusion by Naszodi (2023)<ref name=N_IPF_NM_2023 >{{cite arXiv |last=Naszodi |first=A. |year=2023 |title=The iterative proportional fitting algorithm and the NM-method: solutions for two different sets of problems |class=econ.GN |eprint=2303.05515}}</ref> that the IPF is suitable for sampling correction tasks, but not for generation of counterfactuals. Similarly to Naszodi, Macdonald also questions whether the row and column proportional transformations of the IPF preserve the structure of association within a contingency table that allows us to study social mobility.

==Comparison with the Minimum Euclidean Distance Approach == The Minimum Euclidean Distance Approach (MEDA) (defined by Abbott et al., 2019 following Fernández and Rogerson, 2001) is also a function:<ref name=Abbott2019>{{Cite journal |last1=Abbott |first1=B. |last2=Gallipoli |first2=G.| last3=Meghir |first3=C.|last4=Violante |first4=G.L. |year=2019 |title=Education policy and intergenerational transfers in equilibrium |journal=Journal of Political Economy |volume=127 |issue=6 |pages=2569–2624 |doi=10.1086/702241|s2cid=14693929 |url=https://discovery.ucl.ac.uk/id/eprint/10089197/ |hdl=10419/173937 |hdl-access=free }}</ref> <ref name=FR2001>{{Cite journal |last1=Fernández |first1=R. |last2=Rogerson |first2=R.|year=2001 |title=Sorting and long-run inequality |journal=The Quarterly Journal of Economics |volume=116 |issue=4 |pages=1305–1341 |doi=10.1162/003355301753265589 |url=http://papers.nber.org/papers/w7508.pdf }}</ref> <math> \text{MEDA}(Z, Y e^T_m, e_nY): \mathbb{R}^{n \times m} \times \mathbb{R}^{n} \times \mathbb{R}^{m} \mapsto \mathbb{R}^{n \times m}</math>.

First, MEDA assigns a scalar to matrix <math> Z </math>: it is the weight used for constructing the convex combination of two extreme cases (random and perfectly assortative matching with the pair of marginals <math>(Z e^T_m, e_nZ)</math>) by minimizing the Eucledean distance with <math>Z</math>. E.g. this scalar is <math>v=0.265</math> in the numerical example taken from Abbott et al.(2019).<ref name=Abbott2019/> Second, for any pair of counterfactual marginal distributions (<math>Y e^T_m, e_nY</math>) the MEDA constructs the convex combination of the two extreme cases (random and perfectly assortative matches with the pair of marginals (<math>Y e^T_m, e_nY</math>)).

Differences between the NM and the MEDA: while the NM holds the assortativeness unchanged by keeping the generalized matrix-valued Liu–Lu index <math>\text{LL}({Z})</math> fixed, the MEDA does the same by keeping the scalar <math>v</math> fixed. For <math>Y </math>, and <math> Z </math> matrices of size <math>2\times 2</math> the two methods produces the same transformed table provided <math>v</math> ranks the contingency tables the same as the scalar-valued Liu–Lu index does.<ref name=Ch2021>{{Cite journal |last1=Chiappori |first1=P-A. |last2=Costa-Dias |first2=M.| last3=Meghir |first3=C. |year=2021 |title=The measuring of assortativeness in marriage: A comment |journal=Cowles Foundation Discussion Paper NO. 2316 }}</ref> However, for <math>{Z}</math> matrices larger than 2×2, the generalized Liu–Lu index is matrix-valued, so it is different from the scalar-valued <math>v({Z})</math>. Therefore, the NM-transformed table is also different from the MEDA-transformed table.

For instance, in the numerical example taken from Abbott et al.(2019), the counterfactual table constructed by MEDA {{Anchor|Abbott_F}} is the matrix <math>F</math>: {| class="wikitable" style="margin:1em auto;" ! <math>F</math> !1 !2 !3 !TOTAL |- |1 |1,081 |240 |279 |1,600 |- |2 |217 |5,054 |629 |5,900 |- |3 |92 |376 |2,032 |2,500 |- |TOTAL |1,390 |5,670 |2,940 |10,000 |- |}

The difference between matrix <math>F</math> and matrix <math>X</math> is not negligible. E.g. the share of homogamous couples is 2 percentage points smaller in the MEDA-constructed counterfactual matrix <math>F</math> than in the observed matrix <math>Z</math>, whereas it is 3.4 percentage points smaller in the NM-constructed counterfactual matrix <math>X</math> relative to <math>Z</math>.

Because Abbott's example is not a fictional one, but is based on the empirical educational distribution of American couples, therefore the difference between 2 percentage points and 3.4 percentage points can be interpreted as the MEDA quantifies changes in inequality from one generation to another generation to be significantly smaller compared to the NM.

==See also== * Iterative proportional fitting procedure * Generalized Naszodi-Mendonca method

==External links== * Generalized Naszodi–Mendonca method (GNM-method) {{cite arXiv |last1=Naszodi|first1=A. |last2=Mendonca|first2=F. |year=2021 |title=A new method for identifying what Cupid's invisible hand is doing. Is it spreading color blindness while turning us more "picky" about spousal education? |class=econ.GN |eprint=2103.06991}}

==References== {{reflist}}

Category:Contingency table