Two first-party studies and a fact-checked statistics ledger, as CSV and JSON
Two of the datasets are our own measurements: one on how many business records in our license database have a website of their own, one on who is really behind the AI crawler traffic in our server logs. The third is a ledger of other organisations’ statistics, each checked against its primary source. The files hold the aggregates each study prints, with the denominator next to every percentage. Reuse them with credit.
License-database website-presence dataset (July 2026, corrected October 2026)
Corrected 18 October 2026. The July files are the first version of this study. Their no-website shares are the share of rows our discovery step labelled “no working website”, and an audit showed that label overstates absence. The audited estimate is that about 49% of the 1,705,674 business records in our license database have no website of their own (95% CI 44% to 54%), from 385 random records and a 100-record control checked on 18 October 2026 by an AI assistant running web searches under a written rule. The audit aggregates are in their own file; the first-version files are kept as a historical record, each affected note flagged.
Preview of the first-version file: table entity_buckets (31 rows in the full file across 3 tables)
| Bucket | Rows | Share | Labelled “no working site” |
|---|---|---|---|
| Business entities | 1,705,674 | 41.90% | 95.60% |
| Individuals | 1,262,510 | 31.01% | 99.29% |
| Undetermined | 1,102,709 | 27.09% | 96.23% |
| All 56-state rows | 4,070,893 | 100% | 96.91% |
Method in three filters
- Keep only rows where our pipeline completed a website check (4,542,400 rows). A blank website field on an unchecked row is not a measured absence.
- Keep only the 56 genuine US state and territory codes (4,070,893 rows). The field held 199 distinct values; federal contractor registrations from SAM.gov and scraper source tags are not state licenses. A state code does not make a row a state license: 22% of the audited business records came from federal datasets and 11% from city permit lists.
- Classify each row by entity and trade tokens in the business name (
LLC,INC,PLUMBING) against names that simply equal the holder’s own name. Rows that match neither stay undetermined. - Audit (18 October 2026): search 385 random records labelled “no website” and a 100-record control under a written rule, then weight the two groups back to all 1,705,674 business records.
Limits. This is an audit by web search, not a visit to each business: the checker was an AI assistant following a written rule and reading search results, and a second independent read of 62 mostly borderline records agreed on whether a site was found for 90% of them. The records are the contents of our database (66% state license boards, 22% federal datasets such as SBA loans and FMCSA carrier files, 11% city permit lists), not every licensed business in the US. Trade samples hold 30 records each, so their intervals are wide and the trades cannot be ranked. Verification coverage runs from 20.1% in New York to 99.8% in Alaska, and the two largest states are the least checked. The individual share (31.01%) is a floor, because blank owner-name fields cannot be classified. The first-version state rates (a band of 93% to 99.6%, with Michigan at 85.97%) are unaudited labels.
Full write-up with the correction and the reasoning behind each filter: About 49% of business records in our license database have no website of their own.
AI crawler request log summary (19–26 July 2026)
Between 19 and 26 July 2026, 3,727 requests to vantaweb.io carried an AI-crawler or search-engine user agent. 3,109 of them, 83.4%, asked for a path we have never published. Counting user agents alone overstated real crawler activity by about six times (3,727 divided by 618 requests for real paths is 6.03).
Preview: table legitimate_fetches_per_day (13 rows in the full file across 3 tables)
| Crawler | Operator | Legitimate fetches per day |
|---|---|---|
| ClaudeBot | Anthropic | 43 |
| OAI-SearchBot | OpenAI | 13 |
| ChatGPT-User | OpenAI | 8 |
| Googlebot | 13 |
Method
- Every HTTP request to vantaweb.io for the window, pulled per day from the Cloudflare GraphQL Analytics API (
httpRequestsAdaptiveGroups, dimensionsuserAgentandclientRequestPath). - Each requested path compared with the 105 URLs in our published sitemap plus robots.txt, sitemap.xml, llms.txt, llms-full.txt, rss.xml and our .well-known files. A request for any other path counts as spoofed, whatever its user agent claimed.
Limits. One site, eight days, modest traffic: a single scanning campaign moves the percentages more than it would on a large site. We make no claim about who sent the spoofed requests. The 83.4% is specific to us.
Full write-up: 83% of our AI crawler traffic was fake.
AI receptionist statistics ledger (October 2026)
This one is different from the two studies above: it is a ledger of other organisations’ figures, not our own measurement. It lists 41 statistics on lead response, missed calls, caller behaviour, receptionist wages, pet services and AI adoption, each with a link to its primary source, the source date, the day we re-read it (18 October 2026) and a one-line method. A further 17 widely quoted claims are rated untraceable, misattributed or out of date, with the reason.
The CSV has nine columns: claim, value, unit, source_url, publisher, source_date, checked_date, method and status (verified, untraceable, misattributed or superseded). There is no JSON copy. Licensed CC BY 4.0 like the studies above; the figures themselves belong to their publishers. Full write-up with the fact-check table and an ROI method: AI receptionist statistics 2026.
How the files are laid out
Each study CSV is in long format with six columns: table (which table of the study the row belongs to), item (the row label), metric (what was counted), value, unit and note. The JSON file carries the same rows plus the dataset name, period, source and licence. Values are printed as in the study, without thousands separators. The statistics ledger has its own nine columns, listed in its section above.
Only the aggregates the studies print are published. The underlying license records and request logs are not in these files.
Licence, citation and corrections
All three datasets are licensed under Creative Commons Attribution 4.0. Use them, change them, publish with them, and credit VantaWeb with a link to this page. In the statistics ledger the figures themselves belong to their publishers.
VantaWeb. “Open data from VantaWeb: two first-party studies and a fact-checked statistics ledger.” https://vantaweb.io/data/. Hunter Spence, founder. CC BY 4.0.
If a figure looks wrong, tell us. When we correct a number we correct it on the study page and in these files together, and the study page carries a dated note. Related reading: our AI receptionist statistics, each figure with a named source and a date.