Check your company's visibility in AI
Free, with no credit card required.
Key findings from Semly's research
Brand visibility in AI should be assessed in relation to the needs of the person asking a question and the course of their conversation with the model. A brand appearing in an answer to a general query does not yet show whether the system recommends it to someone for whom its offer is suitable. In Semly's method, Leon assumes the role of a user defined by a persona and builds context through conversation before asking the target purchase question.
At Semly, we are developing this approach with AI Agent Leon. For six months, we conducted research and experiments on querying ChatGPT and Gemini. The dataset provided for this report covers 1.2 million queries, 24 aggregate records and 12 matched comparisons. The analysis includes six cohorts and two content preparation approaches: without a persona and with a persona.
The average time to reach the same visibility target was 100 days without a persona and 70 days with a persona. This represents a 30% reduction, or an average of 30 days. Content relevance increased from 61.5% to 77.5% - by 16 percentage points. We observe the same direction of difference in both systems.
The findings support using personas in content planning and visibility analysis. They provide a basis for a more specific brief: whom the material is for, which selection criteria it should explain and which objections it should address. Answer repeatability and conversion impact require separate metrics; this dataset does not provide a numerical effect for either.
| Information | Value | Basis |
|---|---|---|
| Scope | Six months, ChatGPT and Gemini | Semly's research description |
| Queries in the dataset | 1 200 000 | Sum of query counters at the start and end of measurement |
| Comparison structure | 24 records, 12 pairs | 6 cohorts × 2 systems × 2 approaches |
| Average time to target | 100 vs 70 days | 30% reduction |
| Content relevance | 61.5% vs 77.5% | 16 percentage points; 1200 assessments per approach |
| Repeatability and conversion | No numerical metric in the dataset | Separate areas of evaluation |
Why a question alone does not describe a customer's needs
The query "Where can I buy comfortable running shoes?" does not specify the surface, experience, budget or intended use. A recommendation may therefore address a broad product category, whereas a brand needs to know whether it reaches a recreational forest runner, a beginner on paved routes or a competitive runner.
The statement "I mostly run recreationally on forest trails and need shoes for short training sessions" narrows the situation. In Semly's approach, Leon can introduce this context at the start of the conversation and move to the target question after interpreting the reply. The main purchase need remains the same, but the measurement conditions include the persona, intent and previous messages in that conversation. This matters when interpreting increases or decreases in visibility.
A persona defines the user's role and scenario, which Leon uses when talking to the model. The agent builds his own context in that conversation, bringing the research closer to how a person uses a chat. It does not reconstruct the history of a specific real user. It is worth starting with product-related needs and constraints. Age, gender and other demographic characteristics should be justified by the scenario under study.
Official OpenAI and Google guidance indicates that relevant context helps define a task and its constraints [1][2]. This provides a basis for designing queries.
How AI Agent Leon connects brand knowledge with user context
Leon creates personas from knowledge about the brand, its products, services and existing content. He then acts in the study as a user matching the chosen profile. He starts a conversation, introduces himself to the model and interprets its reply. On that basis, he recognises when he can ask the target prompt with context and intent. The result addresses a practical question: does AI include the brand in a conversation with someone whose needs match its offer?
Leon's role connects monitoring with content planning. Responses can reveal selection criteria, objections and information the model uses to justify recommendations. If information for a particular scenario is missing from the brand's website, the analysis can lead to a brief identifying content gaps.
Leon conducts the conversation as the user
Research conducted by Leon follows a conversation. The agent first introduces himself to the model as the user under study, in line with the selected persona. This establishes the user's situation and needs. After receiving a response, he reads it and interprets it in relation to the conversation's goal.
Interpreting the reply is part of the agent's operation. Leon assesses whether he can move to the target question. Its timing follows from the course of the conversation and the response received. The target prompt contains the user's context and intent and appears in a conversation in which the model already has earlier information about the user.
This way of querying is closer to how a person uses a chat. A user describes their situation, reads the reply and formulates the next question on that basis. Leon takes this role in the study, allowing brand recommendations to be analysed within an unfolding conversation.
Intent defines the decision the user wants to make. In the running example, the context might be recreational running on forest trails and the intent choosing and buying suitable shoes. Leon asks for a recommendation once he has interpreted the earlier response and determined that he can ask the target question.
The entire history of the conversation becomes relevant when interpreting the result. Identical wording in the final question may produce different responses if earlier messages introduce different needs. The study should therefore identify both the target prompt and the conversation in which it was asked.
The process description explains Leon's role and why the research resembles a chat interaction. However, the supplied aggregates do not isolate the effect of introducing himself, interpreting a reply or choosing when to ask the question. We do not attribute the 30% and 16-percentage-point results separately to any of these stages.
| Conversation stage | Leon's action | Research significance |
|---|---|---|
| Introduction | Assumes the user's role defined by the persona and starts the conversation | Establishes the user's situation and needs |
| Model response | Receives and interprets the reply | Accounts for the course of the conversation |
| Decision on the next question | Recognises when to ask the target prompt | Adapts the timing to the response |
| Prompt with intent | Asks the target question with context and intent within that conversation | Enables evaluation of recommendations in user context |
What a 30% reduction in time means
The dataset assumes a shared initial visibility level of 20% and a shared target of 40%. Visibility is defined as the number of responses recommending the brand divided by the number of queries. This represents an increase of 20 percentage points. The approaches describe how content is prepared; results are assessed against the same target and question panel, under the dataset's assumptions.
The average time to target is 100 days without a persona and 70 days with a persona. We calculate the reduction as (100 - 70) / 100 × 100%, or 30%. The difference is 30 days. The precise description is "30% less time to reach the same target".
For a team building visibility, this potentially means a shorter cycle from identifying needs to achieving the intended result. The operational interpretation matters: more targeted content may reduce the need for successive revisions. However, the dataset does not isolate editorial working time or the individual causes of the difference, so we do not attribute the entire effect solely to Leon's automation.
The supplied data contain baseline visibility, final visibility and time to target, without a daily visibility time series. The chart therefore shows the time to a shared result. We do not use these data to draw a curve suggesting that the effect accumulated at a constant rate.
Chart 1. Average time to the shared target of 40% visibility: 100 and 70 days. Basis: 12 matched comparisons in Semly's data. Difference: 30 days, a 30% reduction.
Content relevance and answer accuracy are different measures
In this dataset, content relevance is the share of assessed materials considered aligned with the persona's needs. Each record includes 100 assessments. Without a persona, the total is 738 relevant materials out of 1200 assessments; with a persona, 930 out of 1200. These correspond to 61.5% and 77.5%.
The difference is 16 percentage points. The relative increase over the 61.5% baseline is approximately 26.0%. These are two ways of describing the same dataset, but not the same unit. The main measure used in the text and chart remains the difference in percentage points.
Content relevance is not equivalent to the truthfulness of a model's response. A response may fit the user's needs while containing an incorrect product specification. Answer accuracy should therefore be assessed separately: does the model account for requirements, is the recommendation appropriate and are the stated facts supported by sources?
The 77.5% result shows a higher share of materials meeting users' needs in the persona approach. It supports describing content selection as more relevant within this dataset. Because only aggregate data were provided, without individual assessments or a detailed scoring rubric, the report does not claim statistical significance or equate content relevance with the factual accuracy of every AI response.
Chart 2. Content relevance: 61.5% and 77.5%, a difference of 16 percentage points. Basis: 1200 assessments per approach in Semly's data. The metric describes alignment with persona needs; it is not a conversion rate.
Results for ChatGPT and Gemini
For ChatGPT, the average time to target is 95 days without a persona and 66.5 days with a persona. For Gemini, the figures are 105 and 73.5 days. Both systems show a 30% reduction. The comparison indicates the same direction of difference in the groups analysed.
Content relevance in the ChatGPT records is 60.5% without a persona and 76.5% with a persona. For Gemini, it is 62.5% and 78.5%. The difference in each system is 16 percentage points. Each combination of system and approach includes 600 content assessments.
Separating results by system checks whether the aggregate average masks different outcomes. In this dataset, the differences between approaches are consistent across ChatGPT and Gemini. This is not a quality ranking of the models themselves: times and relevance describe the activities and content analysed.
In practice, maintaining separate visibility profiles for each system is useful. The same content and user needs may be presented differently. Leon's recommendations can then be assessed against a particular system and scenario, without reducing a brand's entire presence to a single number.
Chart 3. Average time to the shared target in ChatGPT and Gemini. Basis: 6 matched comparisons per system, Semly data. The result describes the approaches analysed, not a ranking of model quality.
| AI system | Time without / with a persona | Relevance without / with a persona |
|---|---|---|
| ChatGPT | 95 / 66.5 days | 60.5% / 76.5% |
| Gemini | 105 / 73.5 days | 62.5% / 78.5% |
Results across six cohorts
The dataset covers cohorts M1-M6. The average time to target, calculated for both systems within each cohort, is 75, 85, 95, 105, 115 and 125 days without a persona. With a persona, it is 52.5, 59.5, 66.5, 73.5, 80.5 and 87.5 days.
The difference between approaches is visible for both shorter and longer times to target. It is 22.5 days in M1 and 37.5 days in M6. In every pair in the supplied dataset, the time with a persona is 70% of the time without one. We show this fixed relationship explicitly; we do not treat it as a measure of model response repeatability.
M1-M6 describe cohorts, not six successive visibility readings for one brand. Increasing times should not be interpreted as declining effectiveness in successive months. Analysing seasonality and changes over time requires calendar dates and daily or weekly readings.
A complete time-to-target evaluation should also report cases that did not reach the threshold before observation ended. The supplied dataset does not identify such cases separately. This should be considered when designing future comparisons.
Chart 4. Time to target across six cohorts, averaged over two systems, Semly data. Cohorts M1-M6 are not a monthly visibility time series for one brand.
The same question across multiple personas
As part of Semly's work, we also studied the same base question across multiple personas. Leon acts as a user matching each profile and conducts a conversation before asking the target question. This helps identify how needs, intent and earlier context influence recommendations and information useful for a purchase decision. For a running-shoe question, scenarios could include a beginner on paved routes, a recreational forest runner and someone taking longer trail training sessions. The matrix below illustrates the method's use. It is not a transcript of study responses.
For each response, it is useful to record recommended brands, selection arguments, technical requirements, objections and cited URLs. The comparison shows which information recurs across scenarios and which is specific to a particular need. A brief can then include a common core and additional sections addressing the differences.
Semly uses observations from work with multiple personas as a basis for more relevant content recommendations. However, the supplied dataset does not include persona identifiers, full prompts or model responses. The report therefore does not attribute a numerical effect to a specific persona or rank the brands it recommends.
| Example scenario | Questions for response analysis | Possible content scope |
|---|---|---|
| Beginner, short paved routes | Are comfort and fit criteria mentioned? | Choosing a first pair, size and use |
| Recreational running on forest trails | Does the response consider surface and grip? | Terrain conditions and sole comparisons |
| Longer trail training sessions | Which constraints and specifications does the model identify? | Comparing options for longer exertion |
How responses from multiple personas become a content brief
The conversation can provide information for a brief even before the final recommendation. An earlier response may reveal criteria that need explanation, while Leon's target question connects them to the user's intent. Analysing the full conversation helps identify useful content for successive stages of decision-making.
First, selection criteria should be separated from brand names. A list of recommended companies alone does not explain why they appear. Information that a particular user needs a comparison of applications, an explanation of compatibility or clear return conditions is more useful to an editorial team.
Next, those needs should be compared with the brand's materials. A missing answer to an important question may justify updating an existing page. Differences between user groups may justify a separate section or comparison. The decision should reflect usefulness to people, rather than mechanically multiplying pages for every persona.
An example brief may include the user's scenario, the decision to make, key criteria, verifiable product data, objections and sources. If the analysis concerns forest trails, the material should explain the product's use and limitations.
Content prepared this way may be more relevant to someone arriving from an AI response. This supports the hypothesis of better conversion. Confirmation requires comparing user behaviour and sales outcomes after implementation.
Repeatability requires repeated measurements
A fixed persona description and consistent conversation rules can make measurements easier to compare. Since Leon interprets the reply before asking the target question, the conversation history and decision rules must also be controlled. Process repeatability and agreement between final responses should be measured separately.
The proposed test should retain the same model version, search mode, language, location and session-start rules. It is useful to compare both full conversations started with the same persona and goal, and responses with identically reconstructed context. The first comparison evaluates variation in the entire process; the second helps evaluate variation in the final response. The baseline panel and persona panel should run in parallel.
Possible metrics include stability of brand presence, agreement between sets of recommended brands and agreement between cited sources. For two brand sets A and B, a simple agreement measure is the share of common brands within the set of all brands from both responses. Cases in which both sets are empty require a separate reporting rule.
Repeatability within one persona and differences between personas address different questions. Stable recommendations for one profile can coexist with clear differences between profiles. Leon's structured persona descriptions help maintain more consistent querying conditions. The supplied dataset, however, has no measures of agreement between successive responses, so we do not quantify an improvement in repeatability.
How to verify the impact on conversion
Business measurement should distinguish brand presence in an answer, citation of its website, the user's visit to the site and completion of an action. Each stage requires its own metric. Recommendation counts are not visit counts, and visit counts are not purchase counts.
After implementing content, it is useful to track sessions from identifiable AI sources, engagement, enquiries, transactions and revenue. Some traffic may have no unambiguous source label, so the scope of identified sessions should be described. Conversion rate must use a clearly defined denominator, such as qualified sessions.
Comparing similar pages or content groups, with a consistent conversion definition and observation window, supports a more reliable evaluation. Other changes, such as promotions, prices, product availability and campaigns, must be recorded. Otherwise, sales growth may be wrongly attributed to a change in content strategy.
The report thus proposes a specific hypothesis: content connected to real user needs may help people make decisions. The size of its sales impact remains to be measured. The spreadsheet contains no sessions, leads or transactions.
What a methodology change means for visibility results
Introducing personas and Leon's conversations changes how models are queried. The target prompt's response also accounts for earlier messages. The result may rise when a brand better matches the stated needs, or fall when the system selects another offer. Comparisons should therefore consider both question wording and the way conversational context was prepared.
The clearest approach is to show baseline-panel and persona-panel results side by side. If a combined average is needed, persona shares should follow explicit assumptions about the audience. Equal weights are a simple analytical option, but do not prove that each group accounts for an equal share of real demand.
Alignment with needs also matters when selecting personas. A profile built solely from the offer's perspective may omit purchase barriers or situations in which the product is unsuitable. It is useful to include demanding scenarios, instead of designing a panel only around users for whom the brand is easy to recommend.
Report summary
Semly's research shows that including personas helps align content with user needs and reduce the time required to build visibility in AI. In the analysed dataset, the average time to the shared visibility target was 100 days without a persona and 70 days with a persona - 30% less. Content relevance increased from 61.5% to 77.5%, or 16 percentage points. The same direction of difference was observed in ChatGPT and Gemini.
Leon gives this approach a conversational form. The agent acts as the user: he introduces himself according to the persona, interprets the model's reply and recognises when to ask the target question with context and intent. This allows Semly to analyse recommendations in conditions closer to how people use a chat.
Querying the same base question across multiple personas identifies different selection criteria, objections and information needs. These observations can inform more specific briefs, comparisons and materials that help users decide. Higher content relevance supports this approach; conversion impact requires separate measurement of user behaviour and business outcomes.
Consistent personas and conversation rules provide a basis for comparable monitoring. Answer repeatability must nevertheless be assessed through further measurements accounting for conversation history and querying conditions. The results presented do not quantify an improvement in repeatability or an increase in conversion.
The practical implication for brands is clear: study whom AI recommends an offer to and in what situation, then use that information to develop content. Semly thereby connects a visibility monitoring tool with research into context, intent and model recommendations.
Sources
Semly material: "Over a million queries, six months of research. Why Semly is introducing personas to AI visibility monitoring", supplemented by Semly's description of Leon's conversations - sources for the agent and research description.
[1] OpenAI, Prompt engineering. Guidance on providing relevant context in a query. https://developers.openai.com/api/docs/guides/prompt-engineering
[2] Google AI for Developers, Prompt design strategies. Guidance on context and task constraints. https://ai.google.dev/gemini-api/docs/prompting-strategies
[3] Franziska Weeber, Vera Neplenbroek, Jan Batzner, Sebastian Padó, One Persona, Many Cues, Different Results: How Sociodemographic Cues Impact LLM Personalization, ACL 2026. https://aclanthology.org/2026.acl-long.2079/
Share:
