Quantitative research questions I can answer, with a statistical method for each.
Hover over a question to see its details.
Did it work?
Did the redesigned website push sessions past the 30-minute benchmark to create more ad exposure?
Yes. Average session hit 31.95 min, up ~2 min.
- Right-tailed one-sample t-test vs. historical baseline
- t(19) = 3.37, p = 0.0016
- Cohen’s d = 0.75 (medium-to-large)
Extra 1–3 min per session expands ad viewability and potential revenue. Push the beta release of the website to full-scale release.
Industry context: Media & Streaming (Ad-Supported)
Did checkout completion beat the 60% benchmark?
Yes. 76.7% completed checkout (23 of 30).
- One-tailed one-proportion z-test
- χ² = 3.47, p = 0.031
- Cohen’s h = 0.36 (small-to-medium)
Completion rose from 60% to ~77%, capturing more revenue from existing traffic. Keep the checkout redesign and validate on a larger sample (n = 30).
Industry context: E-commerce / D2C Retail
Does checkout success differ between legacy and redesign?
Yes, slightly. 80% on the new site vs 76% on the old.
- Two-tailed two-proportion z-test
- χ² = 3.85, p = 0.0497
- Cohen’s h = 0.097 (very small)
A 4-point lift in completion can matter at scale. Deploy the new site fully, and confirm with more data since p sits at .05.
Industry context: E-commerce / D2C Retail
Did email signup exceed the 25% KPI?
Yes. 68% signed up (17 of 25).
- One-tailed binomial test (small sample, n = 25)
- p < 0.0001
- Cohen’s h = 0.89 (large)
Signup beat the KPI by 43 points, growing the direct marketing audience. Lock in the new email content and validate at scale.
Industry context: Publishing / E-commerce
Which option should we ship?
Does session length differ across homepage versions A, B and C?
Yes. C averaged ~34.5 min vs ~28.2 for A; B ~31.7.
- One-way ANOVA with Tukey HSD post-hoc
- F(2, 87) = 9.00, p = 0.00028
- η² = 0.17 (large)
Version C adds ~6 min per session over legacy A, expanding ad viewability. Roll out Version C and retire A.
Industry context: Media & Publishing (Ad-Supported)
Do median session times differ across versions A, B and C?
Yes. B (14.8 min) and C (16.9) beat legacy A (11.0).
- Kruskal–Wallis (non-parametric ANOVA alternative)
- B vs C not significantly different
- η²H = 0.08 (moderate)
Legacy A trails B and C by 4–6 median minutes. Retire A; choose between B and C on load time and hosting cost.
Industry context: Media & Publishing (Ad-Supported)
Do device and homepage version (A vs B) change session length?
No detectable effect from device, version, or their mix.
- Two-way ANOVA (Type III)
- Device p = .081 · Version p = .606 · Interaction p = .095
- Partial η²: device 0.11, interaction 0.10 (n.s.)
No evidence either version wins on either device. Ship the cheapest to maintain and confirm with an equivalence test.
Industry context: Media & Publishing (Ad-Supported)
Did clicks spread evenly across layouts A, B and C?
No. A drew 5,909 clicks vs 3,996 for C.
- Chi-squared goodness-of-fit vs equal split
- χ²(2) = 369.94, p < 0.0001
- Cramér’s V = 0.11 (small)
Layout A draws the most clicks, but clicks are not conversions. Validate with conversion data before retiring B and C.
Industry context: Media / E-commerce
Did a trust badge change checkout completion?
Yes. Badge viewers had ~6× the checkout odds.
- Two-tailed Fisher’s exact (small cells), June 2025 pilot
- p = 0.0078, OR = 6.04
- Bias-corrected Cramér’s V = 0.33 (moderate-to-large)
The badge cut cart abandonment in the pilot. Implement it across live checkout and confirm in a larger test.
Industry context: E-commerce / D2C Retail
Does satisfaction differ across Voice, Gesture and Controller?
Yes. Gesture is rated lower than both Voice and Controller.
- Friedman test, Holm-adjusted Wilcoxon post-hoc
- χ²(2) = 16.31, p = 0.00029
- Kendall’s W = 0.41 (moderate)
Gesture is rated lowest of the three input modes. Lead with Voice and Controller; send Gesture back for usability refinement.
Industry context: Consumer Tech / Hardware (Voice/Gesture)
Does it last?
Do users stay longer in Week 3 than in Week 1?
Yes. Sessions grew ~1.5 min from Week 1 to Week 3.
- Two-tailed paired-samples t-test (n = 24)
- t(23) = 2.21, p = 0.038
- Cohen’s d = 0.45 (small-to-medium)
Engagement grows with use, widening ad exposure from core users. Keep the redesign in market and track Week 4+ for drift.
Industry context: Social / Media / Gaming
Does session length keep changing across four weeks?
Yes. 19.1 → 25.2 min by Week 3, then flat.
- Repeated-measures ANOVA, Bonferroni post-hoc
- F(3, 87) = 54.19, p < 0.0001
- Generalized η² = 0.36 (large)
Adoption deepens for three weeks, then holds 5–6 min above Week 1. Treat it as sustained adoption and keep monitoring.
Industry context: Social / Media / Gaming
Did satisfaction improve from Week 1 to Week 3?
Not reliably. The shift was small and within chance.
- One-tailed Wilcoxon signed-rank (1–7 Likert)
- V = 97, p = 0.171
- Rank-biserial r = 0.225 (small)
Satisfaction has not clearly improved after three weeks. Run interviews and open-ended surveys before claiming a win.
Industry context: SaaS / Subscription Services
Who is struggling, and where?
Do new and returning users differ in session length?
Not conclusively. Returning 34.9 vs new 31.7 min.
- Two-tailed independent-samples t-test
- t(38) = −1.90, p = 0.064
- Hedges’ g = −0.59 (medium; new − returning)
Returning users may engage ~3 min longer, but the gap is unproven. Hold segment-specific UX work until a larger sample confirms it.
Industry context: SaaS / Media
Do mobile and desktop users differ in session length?
Yes. Desktop sessions run shorter than mobile.
- Two-tailed Mann–Whitney U (non-normal data)
- W = 114, p = 0.021
- r = 0.368 (moderate)
Desktop sessions run shorter, which trims ad exposure. Audit the desktop experience for friction first.
Industry context: Omnichannel Digital Product
Is device type linked to signup outcome?
Yes. Mobile abandons and times out more.
- Chi-squared test of independence
- χ²(4) = 141.25, p < 0.0001
- Cramér’s V = 0.11 (small)
Mobile onboarding loses more users to abandonment and timeouts. Prioritize fixing mobile-specific friction to lift conversion.
Industry context: SaaS / Fintech / Consumer Apps
What drives satisfaction?
Do longer sessions go with higher satisfaction?
Yes. Strong positive link (r = 0.63).
- Pearson correlation (normality met)
- r = 0.63, t(58) = 6.17, p < 0.0001
- r² = 0.40 (about 40% of variance)
Session length explains ~40% of satisfaction variance. Use it as a health check between surveys, not as proof of cause.
Industry context: Media / Content Platforms
Do slower tasks feel less efficient?
Yes. Longer task times go with lower perceived efficiency.
- Spearman rank correlation (ordinal, non-normal)
- ρ = −0.77, p < 0.0001
- ρ² = 0.59 (large)
Slower tasks feel less efficient. Streamline the slowest core workflows first and follow up with usability sessions.
Industry context: B2B SaaS / Productivity
Do easier-rated experiences also rank higher on satisfaction?
Yes. Ease-of-use and satisfaction rankings rise together.
- Kendall’s tau (ordinal ranks, ties)
- z = 3.18, p = 0.0015
- τ = 0.62 (large)
Ease of use and satisfaction rise together. Prioritize friction reduction as a likely lever; this is association, not proof.
Industry context: General Digital Product
Where these methods drove real decisions
Applied in shipped work. Details generalized under NDA.