Semrush AI Visibility Index

As search shifts from ranked links to AI-generated answers, brands face a new and critical challenge: are AI models mentioning them at all?

To answer this, Semrush set out to establish the category’s first definitive yardstick. Tracking brand prominence across large language models meant solving an unprecedented constraint from day one: the platform had to be completely transparent to the very AI scrapers it was measuring.

What began as a product definition exercise evolved into a core engine of Semrush’s marketing infrastructure.

Tracking brand prominence across large language models meant solving an unprecedented constraint from day one: the platform had to be completely transparent to the very AI scrapers it was measuring.

What began as a product definition exercise evolved into a core engine of Semrush’s marketing infrastructure.

The rise of AI answer engines has upended technical assumptions that have shaped web development for over a decade. For years, teams relied on client-side React, knowing Google’s crawlers easily rendered JavaScript. However, LLM scrapers behind ChatGPT, Claude and Perplexity parse raw HTML instantly without waiting to execute complex client-side scripts.

This creates a massive blind spot. A site can rank perfectly on legacy search engines while remaining completely invisible to AI models — a gap most digital teams have yet to notice. For a platform tracking AI visibility, falling into this gap was not an option.

We chose Next.js, leveraging server-side rendering and static site generation. By ensuring all data arrived fully formed in the raw HTML, we guaranteed absolute discoverability for AI crawlers.

This focus on data integrity extended to the frontend. Using Apache ECharts, we developed interactive visualisations designed to convey clear narrative trends without obscuring the underlying metrics. Because data analysts require raw information rather than just summaries, we designed an integrated CSV export for every chart. The interface functions as both an intuitive visual story and an open, operational dataset.

We took ownership of the platform’s interaction model and product design, translating complex data structures into clean, intuitive user journeys anchored in cognitive clarity.

Replacing rotating cards with a structured grid layout resolved readability barriers, reducing the cognitive load required to parse dense and comparative brand metrics. Page hero sections utilised dynamic, animated search prompts to mimic real-time AI engine behaviours, contextualising the platform’s core purpose for new users. Aligning the chart experience with Semrush’s existing visual language ensured the new product felt instantly authoritative and native to their ecosystem.

What originated as a standalone report has matured into a business-critical publishing system. To embed these insights into Semrush’s core footprint, we built a dedicated API endpoint exposing live index data directly to the flagship semrush.com homepage. This decoupled architecture safely shared a unified data feed while keeping both codebases independent.

As Semrush expanded pipelines to track complex, multi-model metrics, we refactored the application’s core data logic to handle this increased scale. Concurrently, when Semrush rolled out an enterprise-wide rebrand, we integrated the new visual identity across the report platform, ensuring brand directives actively drove every interactive layout decision.

To future-proof editorial velocity, we shifted the site architecture from hand-built static routes to a dynamic, JSON-driven content framework. This optimisation directly unlocked business agility: the editorial team can now launch deeply detailed, multi-model reports instantly to match the market pace, entirely eliminating ongoing technical overhead.

Through this partnership, Semrush established definitive market authority in an emerging category. By operationalising that authority through a highly scalable, decoupled platform, Semrush continuously dictates the industry conversation around AI search dynamics, providing businesses with the empirical data needed to navigate the future of digital visibility.