domains / families / marketing / ab-test-setup

ab-test-setup

combined

When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "should I test this," "which version is better," "test two versions," "statistical significance," or "how long should I run this test." Use this whenever someone is comparing two approaches and wants to measure which performs better. For tracking implementation, see analytics-tracking. For page-level conversion optimization, see page-cro. When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," or "hypothesis." For tracking implementation, see analytics-tracking.

When to use

• Use when a product change has enough eligible traffic for a randomized comparison and a measurable outcome. For low-volume launches or qualitative discovery, consider usability research or descriptive measurement instead of claiming causal…

When not to use

• Clustered users, spillovers and repeated observations can invalidate independent-sample calculations.
• Sequential monitoring needs a planned sequential method; fixed-horizon significance does not authorize repeated peeking.
• A tracking gap or sample-ratio mismatch can invalidate inference despite a favorable primary metric.
• This skill does not activate flags, publish variants or establish regulatory compliance automatically.

Skill page

Conflict

The LeoYeAI copies (openclaw-marketing-skills and openclaw-master-skills) are nearly identical, providing a comprehensive guide with specific sample size tables, traffic allocation strategies, and implementation details (client vs. server-side). The sickn33 copy is a more rigorous, gate-based procedure that adds critical steps missing from the LeoYeAI versions, specifically a detailed 'Tracking Verification' checklist (event firing, variant attribution, de-duplication, randomization checks) and 'Refusal Conditions' for when a test cannot be validly run. Because the sickn33 version contains material safety and verification steps that the LeoYeAI versions lack, and the LeoYeAI versions contain specific reference data (sample size tables) that the sickn33 version omits, crowning one would hide a real alternative approach to the task.