19 Ab Test Setup

A/B Test Setup

Thu thap thong tin

Buoc 0: Kiem tra context

Doc .agents/product-marketing-context.md neu co.

Buoc 1: Hoi user

• Test cai gi? (Headline ads / Landing page / Email subject / Gia / CTA button / Creative video)
• Metric chinh? (CTR / Conversion rate / CPMess / Revenue / Open rate)
• Traffic hang ngay cua trang/ads? (De tinh sample size can thiet)
• Muc tieu test? (Tang X% metric chinh / Tim ra version tot nhat)

7 Nguyen tac A/B test

1. Test 1 bien duy nhat

Quy tac cot loi. Neu test 2 bien cung luc → khong biet bien nao tac dong.

• XAU: Doi headline + doi CTA + doi anh → khong ket luan duoc
• TOT: Chi doi headline (A vs B), moi thu khac giu nguyen

2. Gia thuyet co the kiem dinh

Format: "Neu lam [X], metric [Y] se tang [Z%]"

• TOT: "Neu doi CTA tu 'Dang ky' sang 'Nhan tu van mien phi', conv rate tang 15%"
• XAU: "Copy moi tot hon" (khong cu the, khong do duoc)

3. Sample size du

Khong duoc dung test som.

• Toi thieu: 100 conversion moi variant (khong phai 100 visitor)
• Tot nhat: Tinh truoc (xem cong thuc ben duoi)

4. Thoi gian du

Test tung tuan tron (khong 3 ngay, khong 10 ngay) — khach mua hanh vi khac nhau cac ngay.

• Toi thieu: 7 ngay
• Nen co: 14 ngay

5. Khong peek som

Khong check ket qua moi gio. Dung ket qua cho den khi het thoi han + du sample.

• Peek som + stop som = false positive.

6. Significance > 95%

Chi tin khi p-value < 0.05. Neu khong: ngau nhien, khong phai do bien cua ban.

7. Documented

Ghi lai:

• Gia thuyet
• Ngay bat dau / ket thuc
• Sample size
• Ket qua
• Ket luan

→ De sau nay ban khong test lai cai da test.

Cong thuc tinh Sample Size

Cong thuc don gian

Sample size/variant =
  16 × p × (1-p) / MDE²

Trong do:
  p = Baseline conversion rate hien tai (vi du 0.02 = 2%)
  MDE = Minimum Detectable Effect (muc do thay doi toi thieu ban muon phat hien, vi du 0.2 = 20% tang)

Vi du

Landing page hien co conv rate 3%, muon phat hien thay doi 20%+:

• p = 0.03
• MDE = 0.2 × 0.03 = 0.006 (tuc tang tu 3% len 3.6%)
• Sample size = 16 × 0.03 × 0.97 / 0.006² = 12,933 visit/variant

=> Can 25,866 visit total, neu 500 visit/ngay → 52 ngay.

Landing page hien conv rate 10%, muon phat hien 20%+:

• p = 0.10
• MDE = 0.02 (tu 10% len 12%)
• Sample size = 16 × 0.10 × 0.90 / 0.02² = 3,600/variant

=> 7,200 visit total, 500/ngay → 14 ngay (kha thi).

Quy tac nhanh

| Traffic/ngay | Conv rate | Thoi gian can | Co nen test? |

|-------------|-----------|--------------|-------------|

| < 100 visit | Bat ky | 2+ thang | Khong — traffic thap |

| 100-500 | 2-5% | 3-6 tuan | Co, nhung kien nhan |

| 500-2K | 2-5% | 2-3 tuan | Co — ly tuong |

| 2K+ | Bat ky | 1-2 tuan | Co — nhanh |

Neu traffic thap (< 100/ngay) → khong test, thay vao do tap trung tang traffic truoc.

8 Yeu to nen test (uu tien)

1. Headline

Tac dong cao nhat. 80% nguoi doc headline, 20% doc body → headline thay doi toi uu = bien nho nhat.

Idea:

• Question vs Statement
• Number specific vs generic
• Outcome-focused vs Feature-focused
• Short (5-7 tu) vs Long (12-15 tu)

2. CTA Button

Do thay doi duoc de test. Thuong tang 5-25%.

Test:

• Text: "Dang ky" vs "Nhan tu van mien phi" vs "Bat dau ngay"
• Color: Xanh primary vs Do vs Cam
• Size: Standard vs Lon
• Position: Tren cung vs Cuoi trang

3. Hero Image / Video

Test:

• Anh san pham vs Anh nguoi dung
• Video vs Anh tinh
• Testimonial video vs Demo video

4. Price Display

Test:

• 299K vs 299,000 vs 299.000đ
• Co discount strikeout vs khong
• Monthly vs Yearly highlight

5. Social Proof

Test:

• So con so vs Review chi tiet
• Logo bao chi vs So khach
• Testimonial video vs text

6. Form Fields

Test:

• 3 fields vs 5 fields vs 7 fields
• Label tren vs ben trai
• Placeholder example vs khong co

7. Email Subject Line

Test:

• Question vs Benefit
• Personalization ([Ten]) vs Khong
• Emoji vs Khong (VN: khong emoji tot hon)

8. Ad Creative

Test:

• Hook 3s dau khac nhau
• UGC vs Brand
• Format: Single image vs Carousel vs Video

Setup tracking

Tool khuyen dung

| Tool | Phu hop | Chi phi |

|------|---------|---------|

| Meta Ads A/B test built-in | Test ads (creative/audience/placement) | Free |

| TikTok Ads Split test | Test TikTok ads | Free |

| Google Optimize (deprecated) | Da ngung — dung thay the: VWO, Convert.com | Tu $199/thang |

| PostHog | Test product features, landing | Free tier OK |

| Unbounce / Instapage | Landing page built-in A/B | $90+/thang |

| Manual (split URL) | Don gian, 50/50 URL redirect | Free |

Setup don gian (no-tool)

1. Tao 2 phien ban landing page: /page-a va /page-b
2. Split traffic 50/50:
   - Qua Meta Ads: 2 ad set cung audience, khac link dich
   - Qua Google Ads: 2 ad cung traffic, khac link
   - Qua organic: dung URL rewrite (rare)
3. Tracking conversion rieng tung page:
   - Meta Pixel: event "Lead" voi custom parameter "page_version" = A/B
   - GA4: UTM campaign=test-header, utm_content=a/b
4. Sau 14 ngay + du sample, export data

Phan tich ket qua

Tinh significance (p-value)

Dung calculator online: https://www.evanmiller.org/ab-testing/chi-squared.html

Input:

• Variant A: Visitor count + Conversion count
• Variant B: Visitor count + Conversion count

Output:

• p-value: Neu < 0.05 → significant (95% confidence)
• Uplift: Tang bao nhieu %

Ket luan

| p-value | Uplift | Ket luan |

|---------|--------|---------|

| < 0.05 | > 5% | B thang — implement B |

| < 0.05 | < 5% | Significant nhung nho — can nhac cost |

| 0.05-0.10 | > 10% | Ket qua borderline — keo dai test |

| > 0.10 | Bat ky | Khong significant — giu A hoac test tiep |

Bay thuong gap

• Peek som roi stop. Test ngan → ket qua ngau nhien.
• Sample khong deu. Neu split 30/70 thay vi 50/50 → kiem tra xem truyen thong co van de.
• Seasonality. Test T6 Chu nhat khac T2-T3 → luon test du tuan.
• Novelty effect. Phien ban moi thu hut chu y 2-3 ngay dau → sau do bang cu.

Output template

# A/B Test: [Ten test]
Ngay tao: [YYYY-MM-DD]
Chu so huu: [Ten nguoi]

## 1. Gia thuyet
"Neu [thay doi X], [metric Y] se tang [Z%]"

## 2. Bien test
- Variant A (Control): [Mo ta hien tai]
- Variant B (Challenger): [Mo ta thay doi]
- Chi thay doi: [1 thing]

## 3. Metric
- Primary: [VD: Conversion rate]
- Secondary: [VD: Bounce rate, Time on page]

## 4. Sample size & Duration
- Baseline: [p = %]
- MDE: [% change muon phat hien]
- Sample/variant: [so]
- Traffic/ngay: [so]
- Thoi gian can: [so ngay]

## 5. Setup
- Tool: [Ten]
- URL A: [...]
- URL B: [...]
- Tracking: [Meta Pixel / GA4 / PostHog events]
- Split: 50/50 (hoac khac ghi ro)

## 6. Timeline
- Start: [ngay]
- End: [ngay]
- Review: [ngay]

## 7. Ket qua (dien sau khi ket thuc)
| Variant | Visitors | Conversions | Rate | Uplift vs A |
|---------|----------|-------------|------|-------------|
| A | | | | — |
| B | | | | +X% |

p-value: [x]
Significant (p<0.05): [Yes/No]

## 8. Ket luan
[B thang / A thang / Khong significant]

## 9. Hanh dong
[Implement / Test tiep / Rollback]

## 10. Bai hoc
[Dieu hoc duoc → ap dung cho test sau]

Checklist chat luong

• [ ] Chi test 1 bien
• [ ] Co gia thuyet cu the voi so %
• [ ] Sample size du (toi thieu 100 conv/variant)
• [ ] Thoi gian du (toi thieu 1 tuan tron, ly tuong 2 tuan)
• [ ] Khong peek som
• [ ] p-value < 0.05 truoc khi ket luan
• [ ] Document ket qua du thang hay thua
• [ ] Toi thieu 1 test/thang neu traffic du