# Audit what AI says about you: a repeatable method How to audit what AI says about you: design the prompt set, repeat runs, log answers, score each claim, and decide what to do with every finding. --- To audit what AI says about you, write a fixed set of questions a stranger would ask, run each one several times on several assistants under controlled conditions, log every answer with its sources, and score each claim for accuracy and for how often it appears. The result is not an opinion about your reputation. It is a table of claims, each with a rate, a source and a next action. A single search for your own name tells you almost nothing, because the same question can return a different answer the next day. An audit uses a sample large enough to separate a claim that appears every time from one that appeared once, so you do not spend a week chasing noise. This guide is the method only: how to build the query set, how many runs to do, what to log, how to score and what each finding calls for. It does not explain how assistants source facts or how to file a correction, which are separate topics. Where the method touches a product feature, it follows the provider's own help page, linked where it is used. The rest rests on how repeated sampling works and on our experience on files, written as tendencies and never as promises.

The short version

Use 12 to 30 fixed prompts in six families: identity, role, background, trust, decision and disambiguation. Keep the wording identical on every run.
Run each prompt at least three times, on at least two assistants, on more than one day. Treat the answer as a draw from a distribution, not as a fact.
Log the full text, the date, the assistant, the settings and every cited source. Without the log, no finding can be repeated or checked.
Score each claim twice: is it correct, and how often does it appear. Severity beats frequency for damaging false claims.
Each finding type has a different next action, from fixing a source page to doing nothing. Re-run the same set after every change.

## How to audit what AI says about you: what the method measures An audit measures claims, not impressions. A claim is one checkable statement in an answer, such as "founded a company in a given year", "holds a given degree" or "has been criticized for a given thing". One answer usually contains five to fifteen claims, and each one gets its own row in the log. For each claim the audit produces three measurements. The first is accuracy: correct, outdated, wrong, unsupported or about someone else. The second is the appearance rate: in how many of your runs the claim showed up. The third is the source: which page, if any, the assistant cited for it. Those three together tell you whether a claim is a problem, how big it is and where the fix probably sits. The audit also measures what is missing. If assistants never mention your current role or your best-known work, that is a finding even though no sentence is false. The log therefore holds a second list: facts you expect to see and do not. Run this audit on yourself, on your own company, or on a client who has asked you to. It is not a way to collect information about a private individual. ## Build a query set in six families A good query set copies the questions people actually ask, in their wording, and covers different intents. Write each prompt once, with square-bracket placeholders, and then fix it. If you change the wording between runs you are no longer measuring the assistant, you are measuring your own phrasing.

Family	Template prompt	What it reveals	Watch for
Identity	Who is [full name]?	Whether the assistant knows you, and which person it picks	Another person with your name; a refusal to answer
Role	Who is [full name], [role] at [company]?	Whether the assistant ties you to your work	Wrong employer, old title, no mention of your company
Background	What is [full name]'s career history and education?	Credentials and timeline claims	Invented degrees, shifted dates, merged careers
Trust	Is [full name] reliable? Are there complaints or controversies?	Negative claims and their sources	Old disputes, claims about someone else, vague warnings
Decision	Who are good [roles] in [city] for [need]? Would you hire [full name] for [need]?	Whether you appear when a buyer asks without naming you	Absence, competitors listed instead, thin reasons
Disambiguation	Is [full name] in [field] the same person as [other name]?	How well the assistant separates you from a namesake	Confident merges; only useful if you know a namesake exists

Use three to five prompts per family, with variations in wording that a real person would use: with and without a job title, with a city, with a company name, with a nickname or a common misspelling. That gives 12 to 30 prompts. The workload section below shows why more than 30 becomes unmanageable. For a company, use brand equivalents: who is [company], what does it sell, what do customers say, how does it compare with [competitor], which providers would you recommend for [need]. The decision family matters most here, because a buyer asking for recommendations never types your name. ### Keep a separate pressure set Leading questions, such as "Is [full name] a fraud?", push an assistant toward the premise. Run them once, because a hostile reader will ask them, but score them in a separate table and never pool them with the neutral set. ## Hold the conditions steady so runs can be compared Two runs only compare if the conditions match. The audit therefore fixes the variables that are under your control and records the ones that are not. Fix these four. First, the assistant and its mode: the same product in its search-enabled mode and its plain mode can behave differently, so name the mode in the log. Second, memory and personalization: OpenAI's memory page describes saved memories, a setting that lets ChatGPT draw on past conversations, and Temporary Chat, which it says does not use existing memories or create new ones. An assistant that remembers you will answer with your own history rather than what a stranger would see, so check the equivalent settings of every assistant you use. Third, the account state: a logged-out or fresh account is the closest to a stranger. Fourth, the session: start a new conversation for every run, so that the previous answer does not shape the next. Record these three, which you cannot fully control: date and time, approximate location and language. OpenAI's ChatGPT search page says search uses general location inferred from your IP address, so if your audience is in another country, run part of the audit there and note it. A sensible minimum of surfaces is two chat assistants, one answer engine that shows citations, and the AI summary on a regular search results page. Google's AI Overviews page says an overview is not shown for every search, so when none appears, log "no summary shown" rather than skipping the row. Where a product shows citations, such as inline links or a Sources list, copy them into the log. ## Repeat every prompt, because the answer is a draw, not a fact An assistant generates its answer fresh each time, so the same prompt can return different wording, different claims and different sources. That is why the audit repeats each prompt. A claim seen once is a data point. A claim seen in most runs is a property of how that assistant describes you. Run each prompt at least three times per assistant, spread over at least two days. The spread protects you from a temporary quirk, such as one retrieval result that happened to rank high that morning. Three runs is a floor for spotting patterns, not a precise instrument. If a claim matters a lot, add runs until the picture is stable. Decide the size before you start: prompts times assistants times runs equals answers to read, log and score.

Three audit sizes, in answers to logPrompts times assistants times runs. Assumed effort: about 2 minutes per answer including logging, which is an estimate, not a measurement

Full: 24 prompts, 3 assistants, 3 runs216 answers

Core: 12 prompts, 3 assistants, 3 runs108 answers

Quick: 6 prompts, 2 assistants, 3 runs36 answers

Computed from the stated assumptions. At 2 minutes per answer, the sizes come to about 7.2 hours, 3.6 hours and 1.2 hours. Widths are scaled to the largest value.

For a first baseline the core audit is usually right: it covers every family and gives nine answers per prompt. Use the quick audit for re-tests, limited to the prompts where you found problems, and the full audit before a funding round or launch. ## Log each answer so a stranger could rerun it The log is the audit. Without it you have impressions, and impressions drift: after reading thirty answers you will remember the three that upset you. A spreadsheet with one row per answer and one row per claim is enough. Keep two sheets, one for answers and one for claims, linked by an answer number.

Field	What to record	Why it matters	Sheet
Answer ID	A running number, for example A-0001	Links claims to the answer they came from	Answers
Date and time	With time zone	Answers change; a reviewer needs the moment	Answers
Surface and mode	Assistant name, mode, logged in or out	Separates assistant effects from prompt effects	Answers
Prompt ID and exact text	Pasted word for word	A paraphrase cannot be rerun	Answers
Full answer text	Pasted as text, plus a screenshot	Text can be searched and counted; the image proves the context	Answers
Cited sources	Each link, in order shown, and whether it opened	Where most fixes begin	Answers
Claim text	One checkable statement, in the assistant's words	The unit you score	Claims
Verdict	Correct, outdated, wrong, unsupported, other person	Drives the next action	Claims
Evidence	The public document that proves the true fact	Without it you cannot correct anything	Claims

The procedure for a single session keeps the log honest. Do not score while you run. Capture first, judge later, because judging as you go makes you change the next prompt.

One audit session, start to finish

Open a fresh sessionNew conversation, memory and history settings set as recorded, logged-out or fresh account if that is your protocol.
Paste the prompt unchangedCopy it from your prompt sheet. Never retype it, and do not add "please" or a clarification halfway.
Capture the whole answerPaste the text into the Answers sheet and save a screenshot. Copy every citation link in the order shown.
Do not follow upDo not ask "are you sure" or "where did you get that". A follow-up creates a different conversation and contaminates the run. Follow-ups belong in the tracing stage.
Close and repeatClose the session and start the next run in a new one. Stop for the day when you reach your planned count.
Split into claims laterOn a different day, read each answer and write one row per claim. Then score.

Keep the log private: publishing a screenshot of a wrong claim about yourself can give it one more page on the web. ## Score every claim, then score how often it appears Scoring has two axes, and a claim needs both. Accuracy tells you whether the statement is right. Frequency tells you how much of the picture it makes up. Together they decide priority. For accuracy, use five verdicts. **Correct** means it matches a public document. **Outdated** means it was once true. **Wrong** means it is false and you can prove it. **Unsupported** means you can find no source either way. **Other person** means it is true of someone else. Add a framing note when a correct fact is presented with a loaded emphasis. Framing is not an error, but it is a finding. For frequency, count the share of runs in which the claim appears, per assistant and in total. As a working convention, not a standard, we group claims into three bands: stable when a claim appears in about two thirds of runs or more, recurring when it appears in roughly a third to two thirds, and sporadic when it appears in fewer than a third. The bands are only a way of deciding what to look at first. Change the cut-offs if your run count is larger, and keep them the same across audits so that results compare. Two rules stop the scoring from misleading you. First, severity overrides frequency. A sporadic claim that you committed a crime you did not commit, or that you hold a licence you do not hold, still gets documented the first time it appears, and it is not ignored because it showed up once. Second, an absence is scored like a claim: write "current role not mentioned" as a row, with its own rate. If your correct current role appears in a third of runs, that is the number to improve. ## A worked scoring example with invented numbers This scenario is invented and uses a generic subject: an independent consultant. We ran one prompt, "Who is [full name], and what do they do?", on three assistants, three runs each, so nine answers. The rates below are computed from those assumed counts.

Claim	Verdict	Appeared in	Rate and band
Runs an independent consultancy in the field	Correct	9 of 9	100 percent, stable
Is employed by a former employer	Outdated	6 of 9	67 percent, stable
Current consultancy named	Correct but missing	3 of 9	33 percent, recurring
Won an industry award	Unsupported	2 of 9	22 percent, sporadic
Linked to a lawsuit	Other person	1 of 9	11 percent, sporadic but severe

The outdated employer is the biggest problem by volume: two thirds of answers tell a stranger that the subject works somewhere they left. The missing consultancy name is the second problem and is linked to the first, because the stale page is probably crowding out the current one. The lawsuit claim appeared once, but it concerns another person and it is serious, so it is documented the first time, re-tested within a week and reported if it returns. The unsupported award is watched and not chased: with nothing to disprove, the better response is to publish the real list of achievements. With one run, the answer would have looked either fine or alarming, depending on the draw. Nine answers show the pattern. ## What to do with each finding The verdict and the band together give the next action. This table is the decision aid for the whole audit. It names a route in general terms; how each route works in detail is beyond this guide.

Finding	First action	Speed	Risk or limit
Outdated, stable	Find the cited stale page and ask its owner to update it; publish a current page of your own	Weeks, depending on the page owner	Owners may ignore you; the assistant may keep older memory
Wrong, stable	Trace the citation; fix the source if it is wrong; use the provider's feedback or request route for that exact answer	Days to weeks	Provider routes are narrow and per case
Other person, any band	Add distinguishing facts to a page you own and to matching profiles; document each appearance	Weeks	A namesake with a stronger footprint can keep winning
Missing, recurring	Strengthen the pages that state your current role and work; align profiles	Weeks to months	No assistant is obliged to mention you
Accurate but harsh	Add context and stronger accurate sources; do not try to hide it	Months	Lawful, accurate reporting is not an error to correct
Damaging and false	Capture it, report it, and take legal advice if the harm is serious	Varies	A lawyer, not an audit, decides what is actionable

Tracing sits between scoring and action. For each wrong, outdated or other-person claim, open the cited page and compare it with the claim. If the page says it, the problem lives on the web. If the page says something milder or different, the problem lives in the reading. If nothing is cited, the claim may come from memory or may be invented, and you check whether it recurs across assistants. A repeat on one assistant only points to that assistant's behavior; a repeat on all of them points to a widely available source or to something most systems learned. Sourcing mechanics and reporting routes are topics of their own and are not repeated here. One boundary applies to every row. The audit finds inaccurate, outdated and unlawful material. It is never a reason to hide accurate, lawful reporting about real wrongdoing, and no row above is meant for that. If an answer repeats something true and well sourced, the honest options are context, later developments and your own accurate account. ## Mistakes that make an audit mislead you Most bad audits fail the same few ways. Each one pushes the result in a predictable direction. - **One run per prompt.** You will overreact to a bad draw or relax at a good one. The fix is the three-run floor. - **Asking the assistant to describe its sources.** A description of where it got something is another generated answer. Check the citations the product actually shows and open them. - **Following up inside the session.** "Are you sure?" often changes the answer, which tells you about the assistant's reaction to doubt, not about what it says by default. - **Using only your full name.** People also search with a role, a company, a city or a misspelling. Without these variations the audit has blind spots. - **Mixing neutral and leading prompts.** The results answer different questions and cannot be pooled. - **Trying to fix with pressure tactics.** Fake profiles, paid placements or mass reporting are against platform rules and tend to produce a second problem. The audit exists to find what is true and strengthen it. Worry also biases verdicts: ambiguous wording gets read as false. Mark a claim unsupported, not wrong, when you cannot prove it either way. ## A realistic timeline from baseline to re-test Timing depends on how fast the underlying pages change and on the assistants, neither of which you control. The schedule below is a planning sketch from experience, written as ranges. It is not a promise and no assistant publishes a timetable we could confirm.

When	What you do	What usually happens
Days 1 to 2	Build the prompt sheet; fix settings; run the first round of the core audit	First answers logged; first patterns visible
Days 3 to 5	Run rounds two and three on different days; split answers into claims	A clear first table of stable, recurring and sporadic claims
Week 2	Trace every wrong, outdated and other-person claim to its source; write the action list	A short list of pages to fix and facts to publish
Weeks 2 to 6	Update pages, publish current facts, file requests where a route applies	Some source pages change within days; others take weeks or never change
Weeks 6 to 8	Run a quick re-test on the problem prompts, same settings	Some claims fall in rate; stubborn ones remain and need another action
Every quarter	Re-run the core audit, compare with the baseline	New drift is caught before a stranger finds it

A re-test only means something against the same prompt set and the same conditions as the baseline. Compare rates, not single answers. If the stale employer fell from 6 of 9 to 2 of 9, that is progress. If it fell from 6 to 5, it is within the normal wobble and you should not declare success. ## Next step: run the baseline this week Start small: six prompts, one per family, two sheets, and a quick audit of 36 answers over three days. That is enough to learn whether your picture is mostly fine, mostly missing or actually wrong.

If this is happening to you now

If an assistant is saying something damaging about you and you do not have the time to run a baseline, a private audit by specialists will sample the answers, trace the sources and give you a ranked action list. Reputation Partners runs these for people and companies, and the request starts on the audit page: request a private audit.

When the baseline is done, decide whether you will repeat it yourself every quarter or hand it over. If your answers change often, a monthly monitoring routine keeps the log current without your time. One last note on the provider pages. Google's page also warns that AI Overviews can make mistakes and tells readers to check important details against the linked sources, which is what the log is for. Provider pages change, so reread them before each audit and note any change in your conditions. ## Questions people ask ### How many prompts do I need? Twelve is a workable minimum for a person, one or two per intent family plus variations in wording. Thirty is a practical ceiling, because every prompt multiplies by assistants and runs. If you only have time for six, cover each family once and accept that the picture is coarse. Add prompts later, but start a new baseline when you do. ### Can I just ask the assistant what it knows about me? You can, and that is one of the prompts. But one question, once, is a single draw. Asking the assistant to list its sources or confirm its own accuracy gives you another generated answer, not a record. Use the citations the product shows, open each one, and compare them with the claim. ### Should I turn memory off before auditing? Usually yes, or use a fresh or logged-out session. An assistant that remembers earlier conversations may answer from what you told it rather than from what it would tell a stranger. OpenAI's memory page says Temporary Chat does not use existing memories, which makes it one option. Check the settings in each product, since names and defaults differ, and write down what you chose so the next audit uses the same conditions. ### Can I audit someone else with this method? Only with their knowledge, for example a client or a colleague who asked. The method is designed to show what a stranger would learn about you or about your organization. It is not a way to build a profile of a private individual, and prompts about someone's address, family or health are outside its purpose. ### How often should I repeat the audit? A quarterly core audit is a reasonable default for a person with a stable profile. Run a quick re-test about six to eight weeks after you change source pages. Increase the frequency around events that draw attention, such as a launch, a funding round or press coverage, when what people find matters most.