Discrete Numeric X#
Discrete Numeric X → Continuous Y#
from sklearn.datasets import fetch_california_housing
from scipy.stats import ttest_ind, f_oneway, kruskal, spearmanr, pointbiserialr
from sklearn.feature_selection import mutual_info_regression
import pandas as pd
import numpy as np
data = fetch_california_housing()
df = pd.DataFrame(data.data, columns=data.feature_names)
df['target'] = data.target
# Discrete X: convert 'AveBedrms' to integer bins (discrete numeric)
df['discrete_X'] = pd.qcut(df['AveBedrms'], q=3, labels=[0,1,2]).astype(int)
X = df['discrete_X']
y = df['target']
# ANOVA (k groups)
groups = [y[X == v] for v in X.unique()]
anova_f, _ = f_oneway(*groups)
# Kruskal–Wallis
kw, _ = kruskal(*groups)
# Spearman
spearman, _ = spearmanr(X, y)
# Mutual Information
mi = mutual_info_regression(X.values.reshape(-1,1), y)[0]
print("ANOVA:", anova_f)
print("Kruskal–Wallis:", kw)
print("Spearman:", spearman)
print("Mutual Information:", mi)
ANOVA: 91.13854962407781
Kruskal–Wallis: 326.28931935656874
Spearman: -0.11478463010727119
Mutual Information: 0.021553397835710975
Metric |
Value |
What It Measures |
Strength / Meaning |
Interpretation |
|---|---|---|---|---|
ANOVA (F-statistic) |
91.1385 |
Difference in mean values across groups (parametric test) |
Very large F → statistically significant difference |
Group means differ strongly; the continuous variable varies significantly across categories. |
Kruskal–Wallis (H-statistic) |
326.2893 |
Difference in rank distributions across groups (non-parametric) |
Very large H → highly significant |
The groups have different distributions; effect is strong even without assuming normality. |
Spearman |
–0.1148 |
Monotonic (rank-based) association |
Very weak negative |
No meaningful monotonic trend between the variables; direction is negligible. |
Mutual Information |
0.0216 |
Overall dependency (linear + nonlinear) |
Extremely weak |
Variables share almost no information; practically independent. |
Discrete Numeric X → Binary Y#
X = mean texture converted to discrete bins
Y = target (binary)
from sklearn.datasets import load_breast_cancer
from scipy.stats import pointbiserialr, spearmanr, ttest_ind, mannwhitneyu
from sklearn.feature_selection import mutual_info_classif
import pandas as pd
data = load_breast_cancer()
df = pd.DataFrame(data.data, columns=data.feature_names)
df['target'] = data.target
# Discrete numeric X (binning)
df['discrete_X'] = pd.qcut(df['mean texture'], q=3, labels=[0,1,2]).astype(int)
X = df['discrete_X']
y = df['target']
# t-test (binary Y but X must be binary → skip)
# Mann-Whitney U test across categories
u_stat, _ = mannwhitneyu(X[y==0], X[y==1])
# Spearman
spearman, _ = spearmanr(X, y)
# Mutual Information
mi = mutual_info_classif(X.values.reshape(-1,1), y)[0]
print("Mann–Whitney U:", u_stat)
print("Spearman:", spearman)
print("Mutual Information:", mi)
Mann–Whitney U: 56698.0
Spearman: -0.44259107458997965
Mutual Information: 0.09710451880923898
Metric |
Value |
What It Measures |
Strength / Meaning |
Interpretation |
|---|---|---|---|---|
Mann–Whitney U |
56698.0 |
Difference in distributions between two groups (non-parametric) |
Large U; significance depends on sample sizes |
Suggests the two groups differ, but significance cannot be determined without group sizes. |
Spearman |
–0.4426 |
Monotonic (rank-based) association |
Moderate negative |
As X increases, Y tends to decrease in a moderately consistent ranked pattern. |
Mutual Information |
0.0971 |
Overall dependency (linear + nonlinear) |
Weak |
Variables share a small amount of information; dependency exists but is modest. |
Discrete Numeric X → Ordinal Y#
from sklearn.datasets import fetch_california_housing
from scipy.stats import spearmanr, kendalltau
from sklearn.feature_selection import mutual_info_classif
import pandas as pd
data = fetch_california_housing()
df = pd.DataFrame(data.data, columns=data.feature_names)
# Discrete numeric X
df['discrete_X'] = pd.qcut(df['HouseAge'], q=4, labels=[0,1,2,3]).astype(int)
# Ordinal target
df['ordinal_Y'] = pd.qcut(df['MedInc'], q=3, labels=[1,2,3]).astype(int)
X = df['discrete_X']
y = df['ordinal_Y']
# Spearman
spearman, _ = spearmanr(X, y)
# Kendall
kendall, _ = kendalltau(X, y)
# Mutual Information
mi = mutual_info_classif(X.values.reshape(-1,1), y)[0]
print("Spearman:", spearman)
print("Kendall:", kendall)
print("Mutual Information:", mi)
Spearman: -0.14120295707537847
Kendall: -0.12156306154368951
Mutual Information: 0.007406668333802546
Metric |
Value |
What It Measures |
Strength |
Interpretation |
|---|---|---|---|---|
Spearman |
–0.1412 |
Monotonic (rank-based) association |
Very weak |
No meaningful monotonic trend; slight negative direction but too small to matter. |
Kendall |
–0.1216 |
Pairwise concordance (rank agreement) |
Very weak |
Pairs do not show consistent directional agreement; relationship is almost random. |
Mutual Information |
0.0074 |
Overall dependency (linear + nonlinear) |
Extremely weak |
Variables share almost no information; effectively independent. |
Discrete Numeric X → Categorical Nominal Y#
X = petal length binned
Y = species (categorical nominal)
from sklearn.datasets import load_iris
from scipy.stats import f_oneway, kruskal
from sklearn.feature_selection import mutual_info_classif
import pandas as pd
data = load_iris()
df = pd.DataFrame(data.data, columns=data.feature_names)
df['species'] = data.target
# Discrete numeric X
df['discrete_X'] = pd.qcut(df['petal length (cm)'], q=3, labels=[0,1,2]).astype(int)
X = df['discrete_X']
y = df['species']
# ANOVA
groups = [X[y == v] for v in y.unique()]
anova_f, _ = f_oneway(*groups)
# Kruskal–Wallis
kw, _ = kruskal(*groups)
# Mutual Information
mi = mutual_info_classif(X.values.reshape(-1,1), y)[0]
print("ANOVA:", anova_f)
print("Kruskal–Wallis:", kw)
print("Mutual Information:", mi)
ANOVA: 905.4111111111125
Kruskal–Wallis: 138.25480769230774
Mutual Information: 0.9293271682676154
Metric |
Value |
What It Measures |
Strength / Meaning |
Interpretation |
|---|---|---|---|---|
ANOVA (F-statistic) |
905.41 |
Difference in group means (parametric test) |
Extremely large → highly significant |
Group means differ dramatically; very strong evidence of separation between categories. |
Kruskal–Wallis (H) |
138.25 |
Difference in rank distributions (non-parametric) |
Very large → highly significant |
Even without normality assumptions, groups differ strongly in distribution. |
Mutual Information |
0.9293 |
Shared dependency (linear + nonlinear) |
Very strong |
The categorical feature carries substantial predictive information about the continuous variable. |
Discrete Numeric X → Discrete Numeric Y#
X = AveBedrms binned
Y = Population binned
from sklearn.datasets import fetch_california_housing
from scipy.stats import spearmanr, kendalltau, chi2_contingency
from sklearn.feature_selection import mutual_info_regression
import pandas as pd
import numpy as np
data = fetch_california_housing()
df = pd.DataFrame(data.data, columns=data.feature_names)
# Discrete X and discrete Y
df['X_disc'] = pd.qcut(df['AveBedrms'], q=3, labels=[0,1,2]).astype(int)
df['Y_disc'] = pd.qcut(df['Population'], q=3, labels=[0,1,2]).astype(int)
X = df['X_disc']
y = df['Y_disc']
# Spearman
spearman, _ = spearmanr(X, y)
# Kendall
kendall, _ = kendalltau(X, y)
# Chi-square contingency
table = pd.crosstab(X, y)
chi2, p, _, _ = chi2_contingency(table)
# Mutual Information
mi = mutual_info_regression(X.values.reshape(-1,1), y)[0]
print("Spearman:", spearman)
print("Kendall:", kendall)
print("Chi-square:", chi2)
print("Mutual Information:", mi)
Spearman: 0.03415048686575229
Kendall: 0.031019499572029027
Chi-square: 592.9596824985257
Mutual Information: 0.01031983485907606
Metric |
Value |
What It Measures |
Strength |
Interpretation |
|---|---|---|---|---|
Spearman |
0.0342 |
Monotonic (rank-based) association |
Extremely weak |
No meaningful monotonic trend; ordering of X and Y is essentially random. |
Kendall |
0.0310 |
Pairwise rank concordance |
Extremely weak |
Little to no directional agreement between ranked values. |
Chi-square |
592.96 |
Test of independence between categorical variables |
Depends on df; large χ² → deviation from independence |
Indicates the observed category frequencies differ from expected frequencies. Shows some form of association, but does not tell strength. |
Mutual Information |
0.0103 |
General dependency (linear + nonlinear) |
Extremely weak |
Variables share almost no information; nearly independent. |