ModernMCA: Multiple Correspondence Analysis for scikit-learn

Multiple Correspondence Analysis (MCA) is a multivariate statistical technique originally proposed by Herman Otto Hartley and later popularized by the French statistician Jean-Paul Benzécri.

Standard PCA applied to categorical data is mathematically flawed. MCA solves this by acting as the categorical counterpart to PCA. Instead of maximizing variance based on Euclidean distances of continuous variables, MCA uncovers latent structures by analyzing the Chi-Square distances between categories within an indicator matrix.

Objectives

The primary objective of applying MCA is to find, quantify, and visualize underlying structures, associations, and patterns within complex categorical datasets. It achieves this by projecting both the individuals (rows) and the categories (columns) into a shared, lower-dimensional geometric space (typically 2D or 3D).

By operating on the Chi-Square metric, the distance between two individuals in the projected space reflects the similarity of their profiles (the categories they chose). Similarly, the distance between two categories reflects their tendency to be chosen by the same individuals. The result is a set of orthogonal vectors (factors) that capture the maximum possible categorical inertia (variance).

Notable Works & Epistemological Foundation

MCA is not just an algorithm; it is the mathematical backbone of Geometric Data Analysis (GDA), deeply intertwined with sociological theory. To understand the depth of the insights MCA can provide, it is highly recommended to study the following applications:

• Pierre Bourdieu: Bourdieu transformed MCA from a purely statistical tool into a profound theoretical instrument. In masterworks like "La Distinction" (1979) and "Homo Academicus" (1984), he utilized MCA to empirically construct the concept of the "Social Space" and "Fields." For Bourdieu, the spatial distances generated by MCA are direct representations of social distances, mapping how economic, cultural, and social capitals are distributed and how they shape habitus and lifestyle.

• Frédéric Lebaron: A prominent figure in contemporary Geometric Data Analysis, Lebaron expanded on Benzécri and Bourdieu's traditions. His works rigorously apply MCA to fields like economic sociology, demonstrating how macroeconomic beliefs and institutional power structures (e.g., central bankers, economic elites) can be mapped geometrically. He bridges the gap between robust algebraic structures and deep sociological interpretation. (Lebaron, F. (2001). ECONOMISTS AND THE ECONOMIC ORDER The field of economists and the field of power in France. European Societies, 3(1), 91–110. https://doi:10.1080/14616690120046969)

• Elisa Klüger: Showcasing the power of MCA in state and institutional analysis, Klüger reconstructs the Brazilian National Economic and Social Development Bank (BNDES) as a social space. Her work maps the massive policy inflections between neoliberalism (privatization) and state-led developmentalism (governmental investment) across the 1990s and 2000s by analyzing the trajectories of the bank's presidents and directors. By geometrically projecting the social properties of these agents (via MCA) and intersecting them with network analysis, Klüger demonstrates how macro-level shifts in economic policy are structurally rooted in the changing profiles and cleavages of the bureaucratic elite. It is a masterclass in relational sociology applied to economic institutions.(see: Klüger, E. (2018). Mapping the Inflections in the Policies of the Brazilian National Economic and Social Development Bank during the 1990s and 2000s within Social Spaces and Networks. Historical Social Research, 43(3), 274-302. https://doi.org/10.12759/hsr.43.2018.3.274-302)

• Rodrigo Cantu: Investigates the political and ideological cleavages within the Brazilian corporate elite during the rise of the extreme right in the late 2010s. By applying MCA to an original dataset of corporate actors, the study maps the structural opposition between government supporters and critics. The geometric projection reveals that political alignment is heavily mediated by capital volume and structure: supporters are primarily linked to smaller enterprises with lower cultural capital, whereas the more heterogeneous group of critics is structurally embedded in large corporations with higher cultural resources. This work is a prime example of using MCA to uncover the latent social determinants underlying contemporary political polarization. (Cantu, R. (2024). Clivagens emergentes no mundo empresarial brasileiro: Apoiadores e críticos da extrema direita. Tempo Social, 36(3), 47-75. https://doi.org/10.11606/0103-2070.ts.2024.226295)

Technical Overview

Unlike naive implementations, ModernMCA is mathematically rigorous and accounts for the structural artifacts of indicator matrices.

1. The Indicator Matrix and Chi-Square Metric: The algorithm begins by transforming the categorical dataset $X$ of dimensions $N \times Q$ (where $N$ is the number of individuals and $Q$ is the number of variables) into a complete disjunctive table, or Indicator Matrix $Z$, of dimensions $N \times J$ (where $J$ is the total number of categories). The probability matrix $P = Z / N$ is computed, alongside the row masses ($r$) and column masses ($c$). The standardized residual matrix $S$ is then constructed using the Chi-Square metric: $$S = \frac{P - r c^T}{\sqrt{r c^T}}$$

2. Singular Value Decomposition (SVD): We compute the SVD of the standardized matrix: $S = U \Sigma V^T$. The raw eigenvalues $\lambda_{raw}$ (principal inertias) are derived from the singular values ($\Sigma^2$). These raw values are strictly used to compute the valid coordinates for rows and columns in the spatial projection.

3. The Dimensionality Artifact (Benzécri & Greenacre Corrections): A known geometric artifact of MCA is that the raw eigenvalues severely underestimate the true explained variance. This happens because the implicit Burt Matrix ($Z^T Z$) contains diagonal blocks of identity matrices, artificially inflating the total inertia. ModernMCA solves this natively through two correction strategies:

• Benzécri Correction (correction='benzecri'): Recalculates the eigenvalues to only account for components whose inertia is greater than the average ($1/Q$). It provides a highly optimistic, yet sociologically relevant, view of the explained variance.

• Greenacre Correction (correction='greenacre' - Default): Provides a more rigorous mathematical bound by calculating the exact off-diagonal inertia of the Burt matrix, avoiding the inflation seen in Benzécri's approach while accurately reflecting the true explanatory power of the axes.

How to Use

ModernMCA is designed to be a fully compliant scikit-learn transformer. It easily integrates into complex data pipelines, allowing for seamless transitions between data engineering, statistical modeling, and interactive visualization.

1. Basic Fitting and Transformation

import pandas as pd
from sklearn.pipeline import Pipeline
from mcorrespondence import ModernMCA

# Assuming df is a pandas DataFrame with strictly categorical/object columns
df = pd.read_csv('consultancies_data.csv')

# Initialize the MCA model with Greenacre's rigorous correction
mca = ModernMCA(n_components=5, correction='greenacre')

# Fit and transform the data 
# Output: A DataFrame with the coordinates of the individuals in the new space
individuals_coords = mca.fit_transform(df)

print(f"Total Inertia Explained: {mca.explained_variance_ratio_[:2].sum() * 100:.2f}%")

2. Extracting Deep Statistics

To conduct a true sociological or structural analysis, you need the coordinates and the absolute contributions of the categories to each axis. The contribution tells you which variables are mathematically pulling the dimensions.

# Get row coordinates (Individuals/Organizations)
row_stats = mca.get_row_stats()

# Get column coordinates (Categories) and their Contributions
col_coords, col_contrib = mca.get_column_stats()

# Check the categories that contribute the most to Axis 1
top_contributors_dim1 = col_contrib.sort_values(by='Contr 1', ascending=False).head(10)
print(top_contributors_dim1)

3. Visualization and Interactive Visualization

COMING SOON!