Frontend School
DashboardProblemsLearnPracticeMock InterviewReportsStatsBuy Credits
DashboardProblemsLearnPracticeMock Interview
Frontend System Design Roadmap
  1. 1.HTTP/1.1 vs HTTP/2 vs HTTP/3 (QUIC)
  2. 2.TCP and TLS Handshakes: Connection Setup Costs
  3. 3.Browser Network Stack and Connection Pooling
  4. 4.GET vs POST and Idempotency in HTTP
  5. 5.The Critical Rendering Path (CRP)
  6. 6.DOM, CSSOM and the Render Tree
  7. 7.Layout, Paint and Compositing
  8. 8.Reflows and Repaints Explained
  9. 9.Event Loop: Call Stack, Task and Microtask Queues
  10. 10.Async JavaScript: Promises and async/await
  11. 11.Web Workers: Multi-threading in the Browser
  12. 12.Browser Storage: Cookies, Web Storage, IndexedDB
  13. 13.Client-Side Storage: Limits and Security Risks
  14. 14.Choosing the Right Client-Side Storage
  15. 15.HTTP Caching: Cache-Control, ETag, Last-Modified
  16. 16.Browser-Level Caching Mechanisms
  17. 17.Cache Invalidation for Frontend Apps
  18. 18.Structuring a Large Single-Page App
  19. 19.Component-Based Architecture Principles
  20. 20.Props, State and Component Lifecycle
  21. 21.Module Bundling Basics
  22. 22.Core Web Vitals: LCP, INP and CLS
  23. 23.FCP and Time to Interactive (TTI) Explained
  24. 24.Measuring Performance: Lighthouse and WebPageTest
  25. 25.Asset Compression and Minification Basics
  26. 26.What Is a CDN and How Does It Work?
  27. 27.CDN Benefits: Latency, Availability, Offload
  28. 28.CDN Cache Behavior and Invalidation
  29. 29.Global State vs Local Component State
  30. 30.State Patterns: Redux, Zustand and Context
  31. 31.Data Fetching: React Query and SWR
  32. 32.Immutable Data in Frontend State
  33. 33.SSR vs CSR: Rendering Trade-offs
  34. 34.Hydration and Time to First Byte (TTFB)
  35. 35.Static Site Generation (SSG) for Speed and SEO
  36. 36.Incremental Static Regeneration (ISR)
  37. 37.Micro-Frontends: Core Principles
  38. 38.Micro-Frontend Integration Strategies
  39. 39.Communication Between Micro-Frontends
  40. 40.Micro-Frontends: Pros and Cons for Large Teams
  41. 41.Real-Time: Polling, SSE and WebSockets
  42. 42.Real-Time Use Cases: Chat, Alerts, Live Data
  43. 43.Designing a Real-Time Data Flow
  44. 44.Scaling WebSocket Connections
  45. 45.Responsive Images: srcset, sizes and picture
  46. 46.Modern Image Formats: WebP, AVIF, JPEG XL
  47. 47.Lazy Loading Images and Videos
  48. 48.Video Streaming: HLS, DASH, Adaptive Bitrate
  49. 49.REST and GraphQL for Frontend Engineers
  50. 50.Batching, Throttling and Debouncing API Calls
  51. 51.Handling API Errors in the Frontend
  52. 52.Authentication Flows: OAuth and JWT
  53. 53.Global Error Boundaries in React
  54. 54.Error Reporting: Sentry and LogRocket
  55. 55.Graceful Degradation and Fallback UIs
  56. 56.Real User Monitoring (RUM)
  57. 57.CORS: Cross-Origin Resource Sharing
  58. 58.Preventing XSS (Cross-Site Scripting)
  59. 59.CSRF Protection for Frontend Apps
  60. 60.Content Security Policy (CSP)
  61. 61.Storing Auth Tokens: Cookies vs localStorage
  62. 62.PWA Manifest and Installability
  63. 63.Service Workers: Lifecycle and Caching Strategies
  64. 64.Offline Support and Background Sync
  65. 65.Push Notifications for Web Apps
  66. 66.Webpack vs Vite vs Parcel
  67. 67.Code Splitting, Tree Shaking, Asset Optimization
  68. 68.Module Resolution and Hot Module Replacement
  69. 69.Monorepo Tooling: Nx and Lerna
  70. 70.Atomic Design for Component Libraries
  71. 71.Theming and Customization in Design Systems
  72. 72.Accessibility in Component Libraries
  73. 73.Component Library Docs, Testing and Versioning
  74. 74.Publishing and Consuming a Component Library
  75. 75.Full-Text Search: Levenshtein and Fuzzy Matching
  76. 76.Indexing Large Datasets in the Browser
  77. 77.Search Performance: Debouncing and Web Workers
  78. 78.Integrating Backend Search Services
  79. 79.Choosing a Chart Library: D3, Chart.js, Recharts
  80. 80.Rendering Large Datasets: Canvas vs SVG
  81. 81.Interactive, Responsive Data Visualizations
  82. 82.Data Fetching and Aggregation for Dashboards
  83. 83.A/B Test Design and Rollout Strategies
  84. 84.Client-Side vs Server-Side A/B Testing
  85. 85.Feature Flags and Remote Configuration
  86. 86.Experiment Data Collection and Analytics
Frontend System Design Roadmap
76

Frontend System Design · Lesson 76 of 86 · 5 min

Indexing Large Datasets in the Browser

This topic separates candidates who can build a UI from those who can design a performant data-driven application.

When you're asked to design a feature like a searchable product grid or a filterable dashboard, the naive solution is to fetch an array and use Array.prototype.filter. This works for a few hundred items. It fails completely when you have 10,000, 50,000, or more. A linear scan that takes 50ms will drop frames and make your app feel broken.

The goal is to answer queries without iterating. You do this by pre-processing the data into specialized lookup structures called indexes. With the right index, you can find any item or group of items in sub-millisecond time, regardless of dataset size.

When Array.filter Is Too Slow

A linear scan over an array has a time complexity of O(N). For 50,000 items, every single keystroke in a search bar could trigger a 50,000-iteration loop. This can easily exceed the 16ms budget for a 60fps refresh rate, causing UI jank.

Indexing is a trade-off. You accept an upfront cost—CPU time to build the index and memory to store it—in exchange for near-instantaneous reads. You only pay this cost when the data is first loaded and whenever it changes.

If your dataset is small (under ~1,000 items) or doesn't require fast, repetitive lookups, you don't need an index. Don't over-engineer. But once you cross a certain threshold, indexing becomes non-negotiable for a good user experience.

The Two Indexes You Actually Need

For most frontend use cases, you only need to master two types of indexes. Don't get lost in complex database theory.

1. Hash Index (for O(1) exact lookups) This is for finding a single item by a unique identifier. The perfect tool for this in JavaScript is a Map. You map a unique key (like a userId or productId) to the object itself or its location.

Use it for: getItemById(someId).

2. Sorted Index (for O(log N) range queries) This is for finding items that fall within a range (numbers, dates, alphabet) or for retrieving items in a specific order. The structure is a simple sorted array of values or pointers, which you can query with a binary search algorithm.

Use it for: getItemsByPriceRange(min, max), getUsersRegisteredAfter(date).

The trap: A common interview mistake is using the wrong tool for the job. Using a sorted array to find an item by ID forces a slow O(log N) search when you could have had an O(1) map lookup. Conversely, trying to query a price range with a hash map is impossible without iterating over all the values.

Building and Using an Index

Let's build both index types from a single dataset. We'll create a hash index for product IDs and a sorted index for prices.

Notice the sorted index stores only the price and id. This is a memory optimization: the index is small, and we use the id to retrieve the full object from the id index, which acts as our single source of truth.

interface Product {
  id: string;
  name:string;
  price: number;
}

// The two index structures
const idIndex = new Map<string, Product>();
const priceIndex: { price: number; id: string }[] = [];

// The raw data array
const products: Product[] = [
  { id: 'a1', name: 'Keyboard', price: 150 },
  { id: 'b2', name: 'Mouse', price: 75 },
  { id: 'c3', name: 'Monitor', price: 400 },
  // ... thousands more
];

// Build the indexes once, on load
function buildIndexes(data: Product[]) {
  for (const product of data) {
    // Hash index for O(1) ID lookup
    idIndex.set(product.id, product);
    // Data for the sorted index
    priceIndex.push({ price: product.price, id: product.id });
  }
  // Sort the price index for O(log N) range queries
  priceIndex.sort((a, b) => a.price - b.price);
}

buildIndexes(products);

// --- Querying ---

// O(1) lookup using the hash index
function getProductById(id: string): Product | undefined {
  return idIndex.get(id);
}

// O(log N) + O(K) lookup using the sorted index
// (K is the number of items in the result)
function getProductsInPriceRange(min: number, max: number): Product[] {
  // In a real implementation, you'd use binary search here to find
  // the start and end indices efficiently.
  const ids = priceIndex
    .filter(item => item.price >= min && item.price <= max)
    .map(item => item.id);
  
  // Retrieve full objects using the fast ID index
  return ids.map(id => idIndex.get(id)).filter(Boolean) as Product[];
}

In an interview, you must talk about the trade-offs. Building an index consumes memory. For a dataset of 100,000 objects, an index mapping IDs to objects effectively duplicates your data in memory. Storing only pointers (like the original array index or the object ID) in your secondary indexes is a critical optimization.

The Cost of Updates

Indexes are fast for reads, but they make writes more expensive. If you add, remove, or update an item in your original dataset, your indexes become stale.

You have two main strategies for keeping them in sync:

  1. Rebuild: The simplest way. Throw away the old indexes and build new ones from the updated dataset. This is inefficient for frequent, small changes but acceptable for infrequent bulk updates.
  2. Incremental Update: Manually add, remove, or modify entries in your index structures. This is more complex to implement but far more performant for frequent changes. For example, adding a product requires one idIndex.set() and a sorted insertion into the priceIndex.

The trap: Candidates often design a perfect read-only system. The interviewer will then ask, "What happens when the data changes?" Be ready to discuss the cost of writes and the trade-off between rebuilding and incremental updates.

Don't Block the Main Thread

Building an index for 100,000 items is a heavy computation. It can take hundreds of milliseconds. If you do this on the main thread, your UI will freeze.

This is a non-starter.

The solution is to perform all index construction and maintenance inside a Web Worker. The main thread's only job is to send the raw data to the worker, and later, send queries and receive results. This keeps the UI perfectly responsive, even during the initial, expensive indexing phase.

This diagram shows the decision flow for when and what to index. For any large dataset, the process should be offloaded to a worker.

The communication is asynchronous. The main thread sends the data and can immediately render a loading state. When the worker messages back that the indexes are ready, the UI can enable the search and filter controls.

Try it

Find a public JSON API that returns an array of at least 1,000 items (e.g., a list of countries, cities, or products). Fetch the data and build two indexes in your browser's console: a hash index on a unique string field and a sorted index on a numeric field. Use console.time and console.timeEnd to measure how long it takes to find an item using your index versus using Array.find or Array.filter.

Next: Search Performance: Debouncing and Web Workers.

Lock it in — apply this lesson under interview conditions.

Try a free mock interviewPractice system design problems
Previous

Full-Text Search: Levenshtein and Fuzzy Matching

Next

Search Performance: Debouncing and Web Workers