Frontend School
DashboardProblemsLearnPracticeMock InterviewReportsStatsBuy Credits
DashboardProblemsLearnPracticeMock Interview
Frontend System Design Roadmap
  1. 1.HTTP/1.1 vs HTTP/2 vs HTTP/3 (QUIC)
  2. 2.TCP and TLS Handshakes: Connection Setup Costs
  3. 3.Browser Network Stack and Connection Pooling
  4. 4.GET vs POST and Idempotency in HTTP
  5. 5.The Critical Rendering Path (CRP)
  6. 6.DOM, CSSOM and the Render Tree
  7. 7.Layout, Paint and Compositing
  8. 8.Reflows and Repaints Explained
  9. 9.Event Loop: Call Stack, Task and Microtask Queues
  10. 10.Async JavaScript: Promises and async/await
  11. 11.Web Workers: Multi-threading in the Browser
  12. 12.Browser Storage: Cookies, Web Storage, IndexedDB
  13. 13.Client-Side Storage: Limits and Security Risks
  14. 14.Choosing the Right Client-Side Storage
  15. 15.HTTP Caching: Cache-Control, ETag, Last-Modified
  16. 16.Browser-Level Caching Mechanisms
  17. 17.Cache Invalidation for Frontend Apps
  18. 18.Structuring a Large Single-Page App
  19. 19.Component-Based Architecture Principles
  20. 20.Props, State and Component Lifecycle
  21. 21.Module Bundling Basics
  22. 22.Core Web Vitals: LCP, INP and CLS
  23. 23.FCP and Time to Interactive (TTI) Explained
  24. 24.Measuring Performance: Lighthouse and WebPageTest
  25. 25.Asset Compression and Minification Basics
  26. 26.What Is a CDN and How Does It Work?
  27. 27.CDN Benefits: Latency, Availability, Offload
  28. 28.CDN Cache Behavior and Invalidation
  29. 29.Global State vs Local Component State
  30. 30.State Patterns: Redux, Zustand and Context
  31. 31.Data Fetching: React Query and SWR
  32. 32.Immutable Data in Frontend State
  33. 33.SSR vs CSR: Rendering Trade-offs
  34. 34.Hydration and Time to First Byte (TTFB)
  35. 35.Static Site Generation (SSG) for Speed and SEO
  36. 36.Incremental Static Regeneration (ISR)
  37. 37.Micro-Frontends: Core Principles
  38. 38.Micro-Frontend Integration Strategies
  39. 39.Communication Between Micro-Frontends
  40. 40.Micro-Frontends: Pros and Cons for Large Teams
  41. 41.Real-Time: Polling, SSE and WebSockets
  42. 42.Real-Time Use Cases: Chat, Alerts, Live Data
  43. 43.Designing a Real-Time Data Flow
  44. 44.Scaling WebSocket Connections
  45. 45.Responsive Images: srcset, sizes and picture
  46. 46.Modern Image Formats: WebP, AVIF, JPEG XL
  47. 47.Lazy Loading Images and Videos
  48. 48.Video Streaming: HLS, DASH, Adaptive Bitrate
  49. 49.REST and GraphQL for Frontend Engineers
  50. 50.Batching, Throttling and Debouncing API Calls
  51. 51.Handling API Errors in the Frontend
  52. 52.Authentication Flows: OAuth and JWT
  53. 53.Global Error Boundaries in React
  54. 54.Error Reporting: Sentry and LogRocket
  55. 55.Graceful Degradation and Fallback UIs
  56. 56.Real User Monitoring (RUM)
  57. 57.CORS: Cross-Origin Resource Sharing
  58. 58.Preventing XSS (Cross-Site Scripting)
  59. 59.CSRF Protection for Frontend Apps
  60. 60.Content Security Policy (CSP)
  61. 61.Storing Auth Tokens: Cookies vs localStorage
  62. 62.PWA Manifest and Installability
  63. 63.Service Workers: Lifecycle and Caching Strategies
  64. 64.Offline Support and Background Sync
  65. 65.Push Notifications for Web Apps
  66. 66.Webpack vs Vite vs Parcel
  67. 67.Code Splitting, Tree Shaking, Asset Optimization
  68. 68.Module Resolution and Hot Module Replacement
  69. 69.Monorepo Tooling: Nx and Lerna
  70. 70.Atomic Design for Component Libraries
  71. 71.Theming and Customization in Design Systems
  72. 72.Accessibility in Component Libraries
  73. 73.Component Library Docs, Testing and Versioning
  74. 74.Publishing and Consuming a Component Library
  75. 75.Full-Text Search: Levenshtein and Fuzzy Matching
  76. 76.Indexing Large Datasets in the Browser
  77. 77.Search Performance: Debouncing and Web Workers
  78. 78.Integrating Backend Search Services
  79. 79.Choosing a Chart Library: D3, Chart.js, Recharts
  80. 80.Rendering Large Datasets: Canvas vs SVG
  81. 81.Interactive, Responsive Data Visualizations
  82. 82.Data Fetching and Aggregation for Dashboards
  83. 83.A/B Test Design and Rollout Strategies
  84. 84.Client-Side vs Server-Side A/B Testing
  85. 85.Feature Flags and Remote Configuration
  86. 86.Experiment Data Collection and Analytics
Frontend System Design Roadmap
83

Frontend System Design · Lesson 83 of 86 · 5 min

A/B Test Design and Rollout Strategies

This is how you show you can ship features safely and prove they work, not just build them.

When you're asked to design a new feature in an interview, the implementation is only half the answer. The other half is how you get it to users without breaking things. The senior-level answer involves a controlled process of validation and gradual release. You need to articulate a plan that minimizes risk and measures impact.

This is that plan.

How to Frame an Experiment

Before you write a line of code for a new feature variant, you must define what success looks like. Answering "how would you test this?" in an interview requires three parts: a hypothesis, your metrics, and the target audience.

1. Hypothesis: A formal hypothesis follows a simple structure: "We believe that [the change] will result in [the expected outcome] because [the reason]." For example: "We believe that changing the 'Buy Now' button color to green will increase checkout conversions because it creates a stronger visual call to action."

Stating a clear hypothesis shows you think scientifically, not just shipping features based on feelings.

2. Metrics: This is the most critical part. You need to show you understand the feature's business impact and potential side effects. Break your metrics into three categories.

  • Primary Metric: The one number that proves or disproves your hypothesis. For the button color change, it's the click-through rate on that specific button.
  • Secondary Metrics: Other positive outcomes you hope to see. This could be overall checkout completion rate or time to purchase. They provide a fuller picture of user behavior.
  • Guardrail Metrics: The critical numbers you cannot afford to harm. These are your safety net. Common examples are page load time, JavaScript error rate, or user unsubscribe rates. A 10% lift in clicks isn't worth a 5% drop in overall revenue or a 50ms regression in TTI.

3. Segmentation: Who is this experiment for? All users? Only users in a specific region? Only users on mobile devices? Defining the target audience, or segment, ensures the results are relevant and not diluted by noise from unaffected user groups.

The Mechanics of Variant Assignment

The core of any experiment or rollout is a mechanism to show different UIs to different users. This is usually managed by a feature flagging service, but you need to understand the client-side logic that consumes those flags.

The process is straightforward: take a stable user identifier, combine it with an experiment identifier, and use a deterministic hash to bucket the user into a group. This ensures a user consistently sees the same experience.

// A simple, deterministic hashing function.
// In production, use a mature library like MurmurHash.
function getHashBucket(userId: string, experimentId: string): number {
  let hash = 0;
  const input = `${userId}:${experimentId}`;
  for (let i = 0; i < input.length; i++) {
    const char = input.charCodeAt(i);
    hash = (hash << 5) - hash + char;
    hash |= 0; // Convert to 32bit integer
  }
  return Math.abs(hash) % 100; // Bucket user into 0-99
}

interface ExperimentConfig {
  name: string;
  trafficAllocation: number; // % of users in the experiment (e.g., 50%)
  variants: {
    name: string; // e.g., 'control' or 'treatment'
    weight: number; // % of allocated traffic for this variant
  }[];
}

function getVariant(userId: string, config: ExperimentConfig): string {
  const bucket = getHashBucket(userId, config.name);

  // Is the user even in the experiment?
  if (bucket >= config.trafficAllocation) {
    return 'control'; // Default experience
  }

  // If they are, which variant do they get?
  let cumulativeWeight = 0;
  for (const variant of config.variants) {
    cumulativeWeight += variant.weight;
    if (bucket < (config.trafficAllocation * cumulativeWeight) / 100) {
      return variant.name;
    }
  }

  return 'control'; // Fallback
}

This logic decides which variant a user sees. The frontend code then just needs to render the correct component based on the returned string.

The trap: Describing a purely client-side implementation is fine, but you should acknowledge its main weakness: the "flicker" effect. If flags are fetched asynchronously, the user may see the default UI for a moment before it's replaced by the experiment variant. This is a key reason for server-side rendering of experiments.

Choosing a Rollout Strategy

Not every change is a 50/50 A/B test. The strategy you choose depends on the risk and goal of the change.

Phased Rollout: This is your default for shipping new features. You release the feature to a gradually increasing percentage of users: 1%, 10%, 25%, 50%, 100%. At each stage, you monitor your guardrail metrics. If error rates spike at 1%, you turn the flag off, impacting a minimal number of users. This is about risk mitigation.

A/B Test: Use this when you have a specific hypothesis you want to validate, like whether a green button converts better than a blue one. Traffic is typically split 50/50 (or evenly among multiple variants) for a fixed duration to achieve statistical significance. This is about learning.

Canary Release: This term is often used interchangeably with phased rollout, but it's more specific. A canary release typically routes a small percentage of live traffic to a completely separate, new deployment of your infrastructure. It's best for validating high-risk infrastructure changes (e.g., a new service version, a library upgrade) rather than product features.

Dark Launch: You release code to 100% of users, but the feature is invisible. This is for testing the performance and stability of a new backend system or data pipeline under full production load. For example, you could shadow a new search algorithm, sending it the same queries as the old one and logging its performance and results without ever showing them to the user.

When asked "How would you launch X?", your answer should demonstrate risk awareness. Start with: "It depends on the risk of the change."

  • For a risky checkout flow rewrite: "I'd use a phased rollout, starting with 1% of users and closely monitoring error rates and conversion funnels before ramping up."
  • For a low-risk homepage text change: "A 50/50 A/B test would be appropriate to measure the impact on engagement."

Statistical Traps That Invalidate Your Results

Running an experiment is easy. Running a valid experiment is hard. Mentioning these concepts shows you understand the difference.

The trap of "peeking": Don't stop an experiment as soon as it looks like it's winning. Short-term variance is normal, and results often regress to the mean over time. You must determine the required sample size and run duration before the experiment starts, and then stick to it. Ending early because the numbers look good is a classic way to fool yourself.

The novelty effect: Users often react to change itself, not the quality of the change. A new design might get more clicks simply because it's new. This effect wears off. Run experiments for at least a full week to capture different daily usage patterns and allow the novelty to fade.

A/A tests: How do you know your testing tool is working correctly? By running an A/A test. You deploy the exact same experience to two groups of users. If your primary metric shows any statistically significant difference between them, your randomization or logging is broken. It’s a sanity check for your entire experimentation platform.

Try it

Pick a feature on a website you use daily (e.g., YouTube's comment section, Amazon's product recommendations). Propose a change. Write down a one-sentence hypothesis, then list one primary, one secondary, and two guardrail metrics for your proposed change.

Next: We'll dig into the trade-offs between Client-Side vs Server-Side A/B Testing.

Lock it in — apply this lesson under interview conditions.

Try a free mock interviewPractice system design problems
Previous

Data Fetching and Aggregation for Dashboards

Next

Client-Side vs Server-Side A/B Testing