19 Ab Test Setup Agent Skill
Khi nguoi dung can thiet lap A/B test cho ads, landing page, email, hoac product. Cung dung khi nguoi dung nhac 'A/B test', 'split test', 'test creative', 'test copy', 'test landing', 'test gia', 'kiem dinh thong ke', 'sample size'. Skill nay giup chon bien can test, tinh sample size, setup tracking, va phan tich ket qua thong ke (significance) — phu hop cho team khong phai data scientist.
Install
one command, takes just this skill from the repositorynpx skills add https://github.com/minhnv0807/ai-business-skills --skill 19-ab-test-setupThe instruction itself
34 sections, as written by the authorA/B Test Setup
Thu thap thong tin
Buoc 0: Kiem tra context
Doc .agents/product-marketing-context.md neu co.
Buoc 1: Hoi user
- Test cai gi? (Headline ads / Landing page / Email subject / Gia / CTA button / Creative video)
- Metric chinh? (CTR / Conversion rate / CPMess / Revenue / Open rate)
- Traffic hang ngay cua trang/ads? (De tinh sample size can thiet)
- Muc tieu test? (Tang X% metric chinh / Tim ra version tot nhat)
7 Nguyen tac A/B test
1. Test 1 bien duy nhat
Quy tac cot loi. Neu test 2 bien cung luc → khong biet bien nao tac dong.
- XAU: Doi headline + doi CTA + doi anh → khong ket luan duoc
- TOT: Chi doi headline (A vs B), moi thu khac giu nguyen
2. Gia thuyet co the kiem dinh
Format: "Neu lam [X], metric [Y] se tang [Z%]"
- TOT: "Neu doi CTA tu 'Dang ky' sang 'Nhan tu van mien phi', conv rate tang 15%"
- XAU: "Copy moi tot hon" (khong cu the, khong do duoc)
3. Sample size du
Khong duoc dung test som.
- Toi thieu: 100 conversion moi variant (khong phai 100 visitor)
- Tot nhat: Tinh truoc (xem cong thuc ben duoi)
4. Thoi gian du
Test tung tuan tron (khong 3 ngay, khong 10 ngay) — khach mua hanh vi khac nhau cac ngay.
- Toi thieu: 7 ngay
- Nen co: 14 ngay
5. Khong peek som
Khong check ket qua moi gio. Dung ket qua cho den khi het thoi han + du sample.
- Peek som + stop som = false positive.
6. Significance > 95%
Chi tin khi p-value < 0.05. Neu khong: ngau nhien, khong phai do bien cua ban.
7. Documented
Ghi lai:
- Gia thuyet
- Ngay bat dau / ket thuc
- Sample size
- Ket qua
- Ket luan
→ De sau nay ban khong test lai cai da test.
Cong thuc tinh Sample Size
Cong thuc don gian
Sample size/variant =
16 × p × (1-p) / MDE²
Trong do:
p = Baseline conversion rate hien tai (vi du 0.02 = 2%)
MDE = Minimum Detectable Effect (muc do thay doi toi thieu ban muon phat hien, vi du 0.2 = 20% tang)
Vi du
Landing page hien co conv rate 3%, muon phat hien thay doi 20%+:
- p = 0.03
- MDE = 0.2 × 0.03 = 0.006 (tuc tang tu 3% len 3.6%)
- Sample size = 16 × 0.03 × 0.97 / 0.006² = 12,933 visit/variant
=> Can 25,866 visit total, neu 500 visit/ngay → 52 ngay.
Landing page hien conv rate 10%, muon phat hien 20%+:
- p = 0.10
- MDE = 0.02 (tu 10% len 12%)
- Sample size = 16 × 0.10 × 0.90 / 0.02² = 3,600/variant
=> 7,200 visit total, 500/ngay → 14 ngay (kha thi).
Quy tac nhanh
| Traffic/ngay | Conv rate | Thoi gian can | Co nen test? |
|-------------|-----------|--------------|-------------|
| < 100 visit | Bat ky | 2+ thang | Khong — traffic thap |
| 100-500 | 2-5% | 3-6 tuan | Co, nhung kien nhan |
| 500-2K | 2-5% | 2-3 tuan | Co — ly tuong |
| 2K+ | Bat ky | 1-2 tuan | Co — nhanh |
Neu traffic thap (< 100/ngay) → khong test, thay vao do tap trung tang traffic truoc.
8 Yeu to nen test (uu tien)
1. Headline
Tac dong cao nhat. 80% nguoi doc headline, 20% doc body → headline thay doi toi uu = bien nho nhat.
Idea:
- Question vs Statement
- Number specific vs generic
- Outcome-focused vs Feature-focused
- Short (5-7 tu) vs Long (12-15 tu)
2. CTA Button
Do thay doi duoc de test. Thuong tang 5-25%.
Test:
- Text: "Dang ky" vs "Nhan tu van mien phi" vs "Bat dau ngay"
- Color: Xanh primary vs Do vs Cam
- Size: Standard vs Lon
- Position: Tren cung vs Cuoi trang
3. Hero Image / Video
Test:
- Anh san pham vs Anh nguoi dung
- Video vs Anh tinh
- Testimonial video vs Demo video
4. Price Display
Test:
- 299K vs 299,000 vs 299.000đ
- Co discount strikeout vs khong
- Monthly vs Yearly highlight
5. Social Proof
Test:
- So con so vs Review chi tiet
- Logo bao chi vs So khach
- Testimonial video vs text
6. Form Fields
Test:
- 3 fields vs 5 fields vs 7 fields
- Label tren vs ben trai
- Placeholder example vs khong co
7. Email Subject Line
Test:
- Question vs Benefit
- Personalization ([Ten]) vs Khong
- Emoji vs Khong (VN: khong emoji tot hon)
8. Ad Creative
Test:
- Hook 3s dau khac nhau
- UGC vs Brand
- Format: Single image vs Carousel vs Video
Setup tracking
Tool khuyen dung
| Tool | Phu hop | Chi phi |
|------|---------|---------|
| Meta Ads A/B test built-in | Test ads (creative/audience/placement) | Free |
| TikTok Ads Split test | Test TikTok ads | Free |
| Google Optimize (deprecated) | Da ngung — dung thay the: VWO, Convert.com | Tu $199/thang |
| PostHog | Test product features, landing | Free tier OK |
| Unbounce / Instapage | Landing page built-in A/B | $90+/thang |
| Manual (split URL) | Don gian, 50/50 URL redirect | Free |
Setup don gian (no-tool)
1. Tao 2 phien ban landing page: /page-a va /page-b
2. Split traffic 50/50:
- Qua Meta Ads: 2 ad set cung audience, khac link dich
- Qua Google Ads: 2 ad cung traffic, khac link
- Qua organic: dung URL rewrite (rare)
3. Tracking conversion rieng tung page:
- Meta Pixel: event "Lead" voi custom parameter "page_version" = A/B
- GA4: UTM campaign=test-header, utm_content=a/b
4. Sau 14 ngay + du sample, export data
Phan tich ket qua
Tinh significance (p-value)
Dung calculator online: https://www.evanmiller.org/ab-testing/chi-squared.html
Input:
- Variant A: Visitor count + Conversion count
- Variant B: Visitor count + Conversion count
Output:
- p-value: Neu < 0.05 → significant (95% confidence)
- Uplift: Tang bao nhieu %
Ket luan
| p-value | Uplift | Ket luan |
|---------|--------|---------|
| < 0.05 | > 5% | B thang — implement B |
| < 0.05 | < 5% | Significant nhung nho — can nhac cost |
| 0.05-0.10 | > 10% | Ket qua borderline — keo dai test |
| > 0.10 | Bat ky | Khong significant — giu A hoac test tiep |
Bay thuong gap
- Peek som roi stop. Test ngan → ket qua ngau nhien.
- Sample khong deu. Neu split 30/70 thay vi 50/50 → kiem tra xem truyen thong co van de.
- Seasonality. Test T6 Chu nhat khac T2-T3 → luon test du tuan.
- Novelty effect. Phien ban moi thu hut chu y 2-3 ngay dau → sau do bang cu.
Output template
# A/B Test: [Ten test]
Ngay tao: [YYYY-MM-DD]
Chu so huu: [Ten nguoi]
## 1. Gia thuyet
"Neu [thay doi X], [metric Y] se tang [Z%]"
## 2. Bien test
- Variant A (Control): [Mo ta hien tai]
- Variant B (Challenger): [Mo ta thay doi]
- Chi thay doi: [1 thing]
## 3. Metric
- Primary: [VD: Conversion rate]
- Secondary: [VD: Bounce rate, Time on page]
## 4. Sample size & Duration
- Baseline: [p = %]
- MDE: [% change muon phat hien]
- Sample/variant: [so]
- Traffic/ngay: [so]
- Thoi gian can: [so ngay]
## 5. Setup
- Tool: [Ten]
- URL A: [...]
- URL B: [...]
- Tracking: [Meta Pixel / GA4 / PostHog events]
- Split: 50/50 (hoac khac ghi ro)
## 6. Timeline
- Start: [ngay]
- End: [ngay]
- Review: [ngay]
## 7. Ket qua (dien sau khi ket thuc)
| Variant | Visitors | Conversions | Rate | Uplift vs A |
|---------|----------|-------------|------|-------------|
| A | | | | — |
| B | | | | +X% |
p-value: [x]
Significant (p<0.05): [Yes/No]
## 8. Ket luan
[B thang / A thang / Khong significant]
## 9. Hanh dong
[Implement / Test tiep / Rollback]
## 10. Bai hoc
[Dieu hoc duoc → ap dung cho test sau]
Checklist chat luong
- [ ] Chi test 1 bien
- [ ] Co gia thuyet cu the voi so %
- [ ] Sample size du (toi thieu 100 conv/variant)
- [ ] Thoi gian du (toi thieu 1 tuan tron, ly tuong 2 tuan)
- [ ] Khong peek som
- [ ] p-value < 0.05 truoc khi ket luan
- [ ] Document ket qua du thang hay thua
- [ ] Toi thieu 1 test/thang neu traffic du
Other skills for the same job
different authors, same section of the catalogue
Plan, write, and diagnose Instagram Reels that earn cold-audience reach. Use whenever someone wants a reels script or reels hook for a specific Reel, is debugging why a Reel flopped, wants to know if a draft is worth testing with Trial Reels before going public, or needs a reels caption tuned for the post-hashtag instagram algorithm. Built around what Mosseri has publicly named as the signal hierarchy (watch time, sends per reach, likes per reach), the Trial Reels test-then-publish loop, the Original Content Guidelines and 30-day recovery window, the Edits app, and Reels Insights metrics (skip rate, share rate, followers from this post). Covers a Reels-specific reels strategy: send-driving CTAs, originality without watermarks, audio licensing by account type, captions as the primary SEO signal, and the anti-patterns that quietly cap distribution. Pattern-based guidance, not a virality promise.
Perform relative value analysis on bonds by combining pricing, yield curve context, credit spreads, and scenario stress testing. Use when analyzing bond richness/cheapness, computing spread decomposition, comparing bonds, assessing bond value vs curves, or running rate shock scenarios.
Build quick IRR/MOIC sensitivity tables for PE deal evaluation. Models returns across entry multiple, leverage, exit multiple, growth, and hold period scenarios. Use when sizing up a deal, stress-testing assumptions, or preparing IC returns exhibits. Triggers on "returns analysis", "IRR sensitivity", "MOIC table", "what's the return at", "model the returns", or "back of the envelope".
Design lean startup experiments (pretotypes) for a new product. Creates XYZ hypotheses and suggests low-effort validation methods like landing pages, explainer videos, and pre-orders. Use when validating a new product idea, creating pretotypes, or testing market demand.
Amazon Alexa for Shopping Q&A automation: submits questions to Amazon's Alexa/Rufus AI shopping assistant and collects response text; supports optional keyword search context (navigate to search results page before asking for category-specific answers). Use when user mentions Amazon Alexa, Rufus, Amazon shopping assistant, Amazon AI chat, ask Amazon, Amazon Q&A, automate Alexa questions, Rufus chatbot, Amazon assistant automation, collect Alexa responses, bulk question submission to Amazon, keyword search context, category research. Also applies to extracting Amazon product recommendations from conversational AI, automating repeated queries to Amazon's AI shopping feature, collecting Alexa shopping responses at scale, or market research within a specific product category.
When the user wants to create UGC ad campaigns, recruit UGC creators, generate AI UGC content, or scale with user-generated content. Also use when the user mentions 'UGC,' 'user-generated content,' 'creator ads,' 'Spark Ads,' 'whitelisting,' 'AI UGC,' 'Arcads,' 'Creatify,' 'creator brief,' or 'UGC testing.' This skill covers the UGC growth framework from creator recruitment through AI-powered scaling. Do NOT use for technical implementation, code review, or software architecture.
Parse, modify, validate, and patch simulator input files. Use when working with reservoir simulation input files, testing scenarios, or validating simulation configurations. This implementation supports reference format (.DATA); other simulators use different extensions (e.g., .afi, .DAT). Supports natural language modifications, keyword patching, and syntax validation.
Triage ASM/recon output for ownership before testing — separate the target's real assets from namespace-collision noise. Automated recon keyword-matches on the brand name, so for any target whose name is a common/dictionary word, the output is dominated by assets belonging to UNRELATED same-named companies (repos, cloud buckets, mobile apps, breach corpora, typosquats). Built from an authorized engagement where an ASM report's "Criticals" were overwhelmingly false positives and the combo/repos/mobile/bucket lists were polluted with unrelated same-named orgs. Use at the START of any engagement, immediately on receiving any ASM/recon/OSINT dataset, BEFORE testing anything.
How to use it
Take minhnv0807/19-ab-test-setup from the repository into ~/.claude/skills for personal
use, or into .claude/skills inside a project.
The agent identifies a skill by the name field in its header. Two skills with the
same name cannot sit side by side — one of them will be ignored.