Algo Ad Ctr Agent Skill

CTR Prediction Model

Overview

CTR prediction estimates the probability that a user clicks on an ad given context (user, query, ad, position). Forms the core of ad ranking: AdRank = Bid × pCTR. Typically uses logistic regression or gradient-boosted trees. Training on billions of impressions.

When to Use

Trigger conditions:

• Building or improving an ad ranking system
• Predicting click probability for bid optimization
• Evaluating ad creative effectiveness from feature analysis

When NOT to use:

• When predicting post-click conversions (use conversion rate model)
• When setting bid amounts (use bidding strategy skill)

Algorithm

IRON LAW: A CTR Model Must Be CALIBRATED
Predicting relative ranking is insufficient. The predicted probability
must MATCH actual click frequency (e.g., predicted 5% → 5 clicks per
100 impressions). Without calibration, bid optimization breaks:
  Expected Value = Bid × pCTR × pConversion
  If pCTR is off by 2x, bids are wrong by 2x.

Phase 1: Input Validation

Collect impression logs with: user features, ad features, query features, position, click label (0/1). Handle class imbalance (CTR typically 1-5%).

Gate: Sufficient volume (100K+ impressions), click labels verified, no data leakage from position.

Phase 2: Core Algorithm

• Feature engineering: user demographics, ad category, query-ad match, historical CTR, time/device features
• Train model: logistic regression (interpretable) or GBDT (higher accuracy)
• Calibrate predictions: Platt scaling or isotonic regression on holdout set
• Evaluate: log-loss (calibration) + AUC (ranking quality)

Phase 3: Verification

Check calibration: bucket predictions into deciles, compare predicted vs actual CTR per bucket. Plot reliability diagram.

Gate: Calibration curve close to diagonal, AUC > 0.70.

Phase 4: Output

Return predicted CTR with confidence interval and top contributing features.

Output Format

{
  "prediction": {"ctr": 0.035, "confidence_interval": [0.028, 0.042]},
  "top_features": [{"feature": "query_ad_match", "importance": 0.32}],
  "metadata": {"model": "gbdt", "auc": 0.78, "log_loss": 0.21, "calibration_error": 0.008}
}

Examples

Sample I/O

Input: Trained logistic regression with 3 features and these coefficients:

intercept: -3.0
position_1:  0.8
query_ad_match: 1.5
user_is_mobile: 0.3

Features for current request: position_1=1, query_ad_match=1, user_is_mobile=1

Expected: logit = -3.0 + 0.8 + 1.5 + 0.3 = -0.4

pCTR = sigmoid(-0.4) = 1/(1 + e^0.4) ≈ 0.401 → 40.1%

Verify: for features all 0 (baseline), pCTR = sigmoid(-3.0) ≈ 0.047 (4.7%). Calibration is checked by bucketing predictions and comparing to actual CTR in each bucket.

Edge Cases

| Input | Expected | Why |

|-------|----------|-----|

| New ad, no history | Use ad category average | Cold start for features |

| Position 1 vs position 4 | Different CTR, same relevance | Position bias inflates top-slot CTR |

| Very rare query | Low confidence | Insufficient training data for that query |

Gotchas

• Position bias: Ads in position 1 get more clicks regardless of relevance. Train on position-debiased data or include position as a feature and normalize at inference.
• Data freshness: CTR patterns change rapidly (seasonality, trends). Retrain daily or use online learning.
• Feature leakage: Including click-derived features (e.g., historical CTR of this exact ad-query pair) creates leakage if not handled carefully with time-based splits.
• Class imbalance: 97% no-click, 3% click. Use proper evaluation metrics (log-loss, AUC), not accuracy. Consider downsampling negatives during training.
• Multi-task learning: CTR and conversion rate are related but different. Joint models can improve both by sharing lower layers.

References

• For feature engineering best practices, see references/feature-engineering.md
• For position debiasing techniques, see references/position-debiasing.md