Repository avatar
AI Tools
v1.0.0
active

visibilityos-study-reader

io.github.PodPitch/visibilityos-study-reader

Read and compare the published AI answer study from VisibilityOS by PodPitch.

Documentation

VisibilityOS AI answer audit template

A repeatable way to see whether AI assistants mention or cite your brand when buyers ask for recommendations. Start with five prompts, capture two rounds, and compare the same prompt under the same conditions.

Use the VisibilityOS AI Audit Sample Size Planner to choose a collection target for the precision you want. Draft questions with the VisibilityOS Buyer Prompt Builder. Record the questions and matching settings in a two-round CSV with the VisibilityOS Audit Plan Builder. Use the AI Prompt Drift Checker to confirm the question text stayed the same before comparing the answers.

Try the interactive audit for two rounds in your browser, or use the CSV below to keep a record across several dates.

Use the AI Citation Evidence Checker to review whether an AI answer’s cited page supports a specific claim and export your notes. To compare which citation URLs changed between two runs, try the AI Citation Diff.

Use the VisibilityOS AI Citation Evidence Record to package one claim, its exact source excerpt, your review decision, and the next check in a plain-text file.

The VisibilityOS Citation Capture guide explains how to save a selected passage and its URL as source evidence.

The VisibilityOS Citation Diff npm package provides a local Node.js command for the same comparison. Its usage guide is also available on Yarn Classic and Ecosyste.ms.

Read the usage notes on Socket or npmx. Check release metadata in Google's Open Source Insights or Snyk. Browse the published package on jsDelivr or UNPKG.

Use the AI Brand Match Audit to check whether a name in an AI answer refers to your company or a similarly named business. The Brand Identity Audit guide walks through a same-name example. The branded Google search audit applies that identity check to the results people see when they search for a product by name. The branded search results log records results from each check and exports a CSV.

Use the Google indexing status checker to read Google's indexed report, live test and sitemap status before interpreting branded search results.

Use the AI Citation Source Mix Calculator to see how citations split by source type. If a mention or link disappears, follow the AI Search Drop Checklist to work out what changed.

Use the VisibilityOS AI Search Control Group Calculator to compare a before-and-after change in brand mentions against a separate set of prompts.

The reporting claims guide gives sample wording for three audit designs. The prior trends check shows how the groups were moving before an update.

The VisibilityOS Audit Assistant packages the audit steps for compatible assistants and includes a fictional answer review.

Run your audit

  1. Copy example.csv and replace the fictional examples with questions your sales team hears. Include a shortlist question, a use-case question, a geographic question if location matters, a comparison question, and a buying-criteria question. Leave your company name out of open-ended recommendation prompts.
  2. Ask each question in a fresh conversation. Use the same wording, engine, model, search setting, country, language, location, and brand definition in both rounds. Save the answer and source links in your working copy. Use one row per question and configuration per UTC date.
  3. Mark mentioned yes when the answer clearly names your brand. Mark linked yes when the answer or its source panel links to your official domain. Verify the identity when another business uses the same name. Leave out failed or incomplete runs instead of recording them as no.
  4. Repeat on another day. Run python3 compare.py your-audit.csv to report changes between adjacent recorded dates for each matching configuration. Requires Python 3.10+ and no extra packages. Your data stays on your computer.

If an assistant changes versions, use the VisibilityOS Model Upgrade Audit to compare old and new outputs on the same questions.

When more than one person labels answers, use the VisibilityOS Reviewer Agreement worksheet to check that they apply the same rules for brand mentions and official links. Use the reviewer agreement calculator to compare raw agreement and Cohen’s kappa from the same label counts.

Use the VisibilityOS Audit CSV Checker to check a file for missing fields, duplicate observations, and unpaired rows before you compare.

When a pair changes, use the VisibilityOS AI Answer Change Review to see which sentences appeared, disappeared, or matched across the two captured answers.

Use the VisibilityOS AI Shortlist Comparison to see which brands appear in both sets of recommendations and whether a specific brand appeared in either answer.

Use the VisibilityOS Make template to compare brand mentions and official links separately and flag losses for review.

Try it with python3 compare.py example.csv. The fictional example produces five matched comparisons, two presence changes, and two link changes. Use the VisibilityOS sample coverage calculator to report completed comparisons and change rates together.

If you keep the audit CSV in a GitHub repository, run VisibilityOS Audit Compare to put those three counts in the workflow summary.

Presence means either a mention or a link. A gain or loss is a prompt for review. Open the two answers and cited pages to see whether the product disappeared, the sources changed, or the answer confused it with another brand. Treat these counts as observations from your selected prompts and dates. See the AI Search Metrics Glossary for definitions and a worked example. For comparisons across countries or languages, see the VisibilityOS guide to comparing AI search across markets.

Published by VisibilityOS by PodPitch. See the published study and measurement method for an example using real monitoring data. The VisibilityOS research methods overview explains the study counts and what to record when reproducing the method. The sensitivity analysis shows how the pooled rate changes when you leave out one engine view at a time.

Download the aggregate study dataset and field definitions.

Run the VisibilityOS AI Answer Study Reproduction image to reproduce all eight engine results and the pooled rates from those files.

Browse the archived audit template in Software Heritage for a fixed copy of the November 19, 2026 source snapshot.

No installation packages available.