1. Rewriting a question while keeping its old identity
“Which CRM is easy for a small sales team?” and “Which CRM fits a regulated enterprise?” ask for different things. More subtle edits can matter too: adding a price ceiling or an integration requirement changes the conditions under which a recommendation is useful.
Keep the old wording and give the revised question a new version. Record why it changed. To understand the transition, consider a short overlap period in which you observe both versions under the same collection settings. Keep their results separate so you can see how the revised question behaves.
Do not assume that the same row name makes two answers comparable. The reader needs the question text behind it.
2. Adding difficult questions and calling the lower total a decline
Imagine a team testing 20 buyer questions in the same AI service in two reporting periods. This is an invented example, with exactly one usable answer per question in each period. Your brand appears in ten: 50%. Next period, those same questions again produce ten appearances. You also add ten new questions and appear in two of their answers.
The combined panel now shows 12 appearances in 30 answers, or 40%. The original panel stayed at 50%. Its visibility did not fall in this example; the report expanded into questions where the brand appeared less often.
Show both views: the unchanged set for the comparison and the expanded set for current coverage. Keep the new group visible at two of ten rather than letting it disappear inside the total.
3. Mixing discovery questions with branded checks
A question that includes your company name is different from an unbranded request for options. Adding many branded checks can make a mention metric look healthier without showing that more discovery answers recommend the brand.
Separate the tasks before averaging. A branded check might test whether an answer describes your cancellation policy accurately. An unbranded question might ask which product suits a particular buyer. Both can matter; they should not quietly replace each other in a trend.
Store the reason each question exists. Was it an observed customer question, an analyst’s formulation or a generated suggestion? The source tells you how the panel was built. It does not turn that panel into a measure of market demand.
4. Adding markets or engines inside the old total
A new language, city or AI surface expands the scope. Keep it identifiable when it enters the report. Even identical wording can produce a different comparison when the collection method or settings change.
Record what is known about locale and what remains unknown. Naming Dallas in a prompt does not establish a Dallas device location. A product label such as “Google” is also too broad if the underlying surfaces differ.
For an existing trend, retain the original question-and-surface combinations where the settings remain comparable. Put the additional coverage beside them with its own baseline and any unverified settings clearly marked.
5. Removing unanswered or inconvenient rows
A collection failure is not an answer without a brand recommendation. Dropping failures without showing their count can make a report look more complete than it is. Retiring questions where a brand performs poorly can also improve a score simply by changing its denominator.
Keep status separate from outcome. Preserve the reason for retirement and the date it took effect. If a failed record is later recovered, record that correction and whether it falls inside the reporting window; do not quietly rewrite the original observation.
When coverage changes enough to undermine a comparison, label the trend incomplete. A smaller defensible comparison is more useful than a confident total whose population cannot be explained.
Keep a change log small enough to maintain
For each change, record the old and new question versions, the reason, effective date and reporting treatment. Add the question’s evidence source, collection settings and owner. An unknown setting should remain unknown.
The illustrative panel above needs two report lines: “Existing 20-question set: 50% in both periods” and “Expanded 30-question set: 40% this period.” That preserves the stable observation while showing the harder territory the team has started to examine.
Better coverage is useful. It simply needs a label that prevents the new research question from being mistaken for an old trend.
Related: Build the weekly report around comparable evidence and create a manual tracking record.
Start with a clearly scoped snapshot. Run a free Gist Quick Scan for a topic relevant to your brand. Treat it as a starting observation when planning ongoing tracking, rather than a trend on its own.