# Automatic Predictor Selection

**Automatic Predictor Selection** (AutoPrediction) ranks all candidate variables by how strongly they predict a chosen outcome, using a consensus of five machine-learning methods.

---

## 📌 When to Use
- Many potential predictor variables and unclear which ones matter
- Exploratory analysis before building a formal regression model
- Want a data-driven ranking that captures both linear and non-linear effects

---

## 🔬 How It Works

StatInsight runs five feature importance methods and combines their rankings:

| Method | Captures |
|--------|----------|
| L1 (Lasso) Regression | Linear, sparse |
| ElasticNet | Linear + regularisation |
| Random Forest | Non-linear, interactions |
| Mutual Information | Any association |
| Permutation Importance | Model-agnostic |

Each predictor receives a **consensus importance score** — the average rank across all five methods. This consensus reduces the risk of spurious findings from any single method.

---

## 🔬 What StatInsight Reports
- **Forest plot** showing all predictors ranked by –log₁₀(p-value)
  - Square markers = effect size available
  - Diamond markers = p-value only
- **Per-predictor statistical test** (chosen automatically by variable type):
  - Binary outcome + continuous predictor → Mann-Whitney U
  - Categorical outcome + continuous predictor → Kruskal-Wallis
  - Continuous outcome + continuous predictor → Spearman correlation
  - Categorical predictor + any outcome → Chi-square + Cramér's V
- **Collinearity warning**: if two top predictors are highly correlated, StatInsight identifies the specific pair and recommends keeping only one

---

## 🧩 Collinearity
If two predictors in the top results have VIF > 5 or correlation > 0.8, a warning is shown:

> ⚠️ WARNING: [var1] and [var2] are highly correlated. Keep only one in your final model.

---

## 🧪 Assumptions
- Outcome variable must not be a label-type variable
- Works with continuous, binary, and categorical outcomes
- Missing values are handled by row-wise exclusion

---

## 🧠 Interpretation
- **Higher –log₁₀(p)**: stronger evidence of association with the outcome
- **Effect size (Cohen's d, r, Cramér's V)**: magnitude of the association, not just significance
- **Consensus ranking**: trust predictors that appear in the top across multiple methods
- **Collinearity**: correlated predictors inflate apparent importance — keep only the stronger one

---

## 🐍 Python Implementation

```python
from sklearn.linear_model import Lasso, ElasticNet
from sklearn.ensemble import RandomForestClassifier
from sklearn.feature_selection import mutual_info_classif
from sklearn.inspection import permutation_importance
```

---

## 📎 Related Tests

[Multiple Linear Regression](/statistics/regression/multiple-linear.md)

[Multiple Logistic Regression](/statistics/regression/logistic.md)
