Sign up and connect your code host

Setup in five steps

1. Sign in

   Your first sign-in creates a personal workspace with you as its owner.

2. Connect a code host

   In Settings, under Code hosts, pick the host, paste a token with the scopes below and press Connect.

3. Verify the connection

   Press Test. visualdiff makes one read with the token and tells you who it signed in as.

4. Add the webhook

   Copy the payload URL and the secret, which are shown once after saving, into the repository or organisation.

5. Push to a pull request

   The review comment appears on the pull request, and the report in your inbox.

Sign in

Sign in with the button below or the one at the top of the page. visualdiff uses OpenID Connect with the Authorization Code flow and PKCE. A self-hosted deployment signs in through whichever standards-compliant provider its operator configured, such as Auth0, Okta, Keycloak or Google.

• The code is exchanged for tokens on the server; the client secret never reaches your browser. visualdiff checks the state, the nonce, and the ID token’s issuer, audience and expiry before it opens a session.
• A session lasts 30 days. The cookie is HTTP-only and holds a random token; the database stores only its SHA-256 hash.
• Your first sign-in creates a personal workspace named after you, such as “Ada’s workspace”, and makes you its owner. Later sign-ins refresh your name and email from the provider.
• Sign out from Settings, Account.

Sign in

Workspaces and roles

Everything you connect and every report you store belongs to a workspace, and every database query is filtered by it. A person’s role in the workspace decides what they can do. API keys carry scopes instead of a role; see API keys.

The app has no screen for inviting people to a workspace yet, so today each person who signs in works in their own personal workspace. To give a pipeline or a script access, create an API key.

Connect a code host

Open Settings, Code hosts as an owner or admin. Pick the host, fill in the fields and press Connect. The token is stored encrypted and used for three things: to read pull requests on private repositories, to post the review comment and to set the commit status.

GitHub

• Where: create a fine-grained personal access token at github.com/settings/personal-access-tokens/new (Settings, Developer settings, Personal access tokens, Fine-grained tokens). On GitHub Enterprise Server, use the same page on your server.
• Resource owner and repositories: choose the organisation that owns the repositories and only the repositories you want reviewed. Some organisations approve fine-grained tokens before they work.
• Permissions: Fine-grained token: Pull requests (read & write to comment), Contents (read), Commit statuses (read & write for status checks).
• Server address: only for GitHub Enterprise Server. visualdiff calls its API under /api/v3.

GitLab

• Where: the project’s Settings, Access tokens. A group access token, from the group’s Settings, Access tokens, covers every project in the group.
• Scopes and role: Project or group access token with read_api (api to post notes and statuses), role Reporter or above.
• Server address: your self-managed GitLab, such as https://gitlab.acme.com. Leave it empty for gitlab.com.

Bitbucket Cloud

• Where: Repository settings, Security, Access tokens, for a repository access token. Or an Atlassian API token from your Atlassian account’s security settings, with your email in Username or email.
• Permissions: Repository access token with Pull requests: Write and Repositories: Read; or an Atlassian API token with your email as the username.
• Limit to: a workspace or workspace/repository. With a limit, Test reads that workspace or repository; without one, it reads the account behind the token.
• Bitbucket Data Center cannot be connected yet. Its adapter is Next; until then, run the CLI in your pipeline.

Azure DevOps

• Where: User settings, Personal access tokens, New token, in the organisation that holds the repositories.
• Scopes: Personal access token with Code (Read), plus Code (Status) and Pull Request Threads (Read & write) to report back.
• Organisation: required. Pull request links on dev.azure.com/<organisation> and <organisation>.visualstudio.com are both recognised. Azure DevOps Server, the on-premises edition, is not supported.

Gitea and Forgejo

• Where: your user Settings, Applications, Generate new token. Codeberg works the same way.
• Scopes: Access token with read:repository (write:issue to comment, write:repository for statuses).
• Server address: your instance, such as https://git.acme.com. Leave it empty for codeberg.org.

Verify the connection

Press Test next to the connection. visualdiff makes one cheap read with the stored token:

The answer appears at the bottom of the page:

• Signed in as … The token works.
• The token was refused (HTTP 401) or (HTTP 403). The token is wrong, expired or missing a scope.
• The host answered HTTP … The server is reachable but returned an error, for example because the server address is wrong.
• Add the Azure DevOps organisation name. The Organisation field is empty.
• No token stored: public repositories only. Fetching public pull requests still works; commenting and statuses do not.

Under each connection, Settings shows when it was last used or its last error, updated by Test and by reviews you start from the inbox or the API. Press Test again whenever you replace a token.

Add the webhook

When you save a connection with a token, Settings shows a payload URL and a secret once. Copy both before you press Done: the secret is not shown again. The URL has this form, where {host} is github, gitlab, bitbucket, azure or gitea:

https://visualdiff.ai/api/webhooks/{host}/{connectionId}

The connection list keeps showing the URL. To get a new secret, save the connection again with its token. The old secret stops working at once, so update the webhook in the host as well.

Which events start a review

• GitHub: pull request opened, synchronize (new commits), reopened and ready for review. Ping events are answered.
• GitLab: merge request opened, reopened, and updated with new commits. Updates that change only the title, labels or other details are ignored.
• Bitbucket Cloud: pull request created and updated. The diagnostics ping is answered.
• Azure DevOps: pull request created, and pull request updated when the source branch was pushed. Other updates, such as votes or reviewer changes, are ignored.
• Gitea and Forgejo: pull request opened, synchronized and reopened.

What the host sees in its delivery log

What happens on every push

1. The code host sends the event to the payload URL.
2. visualdiff finds the connection named in the URL, checks that its host matches and verifies the signature or secret before reading anything else.
3. Events that are not a new or updated pull request are answered 202 with the reason they were ignored.
4. A push already reviewed in the last ten minutes, for the same pull request and the same head commit, is answered 202 and skipped. Hosts that redeliver do not cause a second review.
5. visualdiff answers 202 straight away and reviews in the background, so the host never times out waiting.
6. It fetches the change with the connection’s token: the diff, the commits, the repository tree and dependency manifests at the base, .visualdiff.json from the base revision, and the contents of up to 40 changed files.
7. It analyses the change under your policy, matches it to your ArchCanvas map if one is connected, checks new dependencies against npm and PyPI, and writes the summary.
8. It stores the report in your workspace, with a share link.
9. With a token on the connection, it posts or edits its one comment, and sets the commit status if you turned statuses on.
10. It notifies Slack or your endpoint if the review is held, or on every review if you chose that.

If a review fails, the failure is logged with the trace id, and the next delivery of the same commit is reviewed again rather than skipped.

The comment and the status on each host

The comment starts with a hidden marker, <!-- visualdiff-review -->. visualdiff looks for it and edits that comment, so a pull request has one review comment however often it is pushed. The comment gives the verdict and summary, the review card linking to the full report, the reasons for the verdict, the hunks to read first with links to the exact lines (on GitHub, GitLab, Bitbucket, Gitea and Forgejo), what is safe to skip, and notes on intent, provenance, dependencies, tests and infrastructure.

The status gives the verdict, the risk out of 100 and the minutes the plan needs, and links to the report. It succeeds unless the verdict is hold and Fail the status on hold is on. To block merging on hold, turn on both status settings under Review settings and make the status required in your host, for example as a required status check named visualdiff in GitHub branch protection, or a status policy for visualdiff/review in an Azure DevOps branch policy.

Inbox, reports and share links

The inbox lists your workspace’s stored reviews, newest first, with the verdict, title, repository and number, whether an agent wrote it, intent drift, risk and age. Reviews from webhooks, from the inbox and from CI with --store all land here.

• Review by link: paste a pull request link. visualdiff uses your connected token for that host automatically. For a host you have not connected, choose “Use a token once”: the token fetches that pull request and is never stored.
• Review a diff: paste the output of git diff, with a title and what the change is meant to do, which the intent check compares against.
• Budget: set the minutes a reviewer has; the plan fits the read lane into it.

A report opens on the review plan. Its tabs are Review plan, Change map, Findings, Intent, Tests & deps, Provenance, Infrastructure and Files. At the top, Copy PR comment copies the same Markdown the comment uses, Review card downloads the card image, and Share copies the share link.

• Share links end in ?t=…. Anyone with the link can open that one report without signing in. The live ArchCanvas map inside it shows only to members of your workspace.
• The card is also an image at /api/reports/{id}/card?t=…, for chat and comments.
• Each workspace keeps its newest 2,000 reports. Delete one through the API with DELETE /api/v1/reports/{id} (write scope).

Sanity index

The sanity index answers one question each week: is review keeping up with the code being written? It is built from every stored review, for one repository or all of them, over 8, 12, 26 or 52 weeks. 100 is calm.

sanity = 100
  − 0.45 × average risk
  − 25 × share of reviews held
  − 20 × share of reviews with intent drift
  − up to 10 for critical and high findings per review

Beside it you see the number of reviews, the share written by agents, the signal (the share of changed lines worth reading) and the average risk, each as a weekly line. The same series is at GET /api/v1/sanity?repo=&weeks= with the read scope, for 4 to 52 weeks.

ArchCanvas

ArchCanvas maps your cloud accounts into one resource graph. With it connected, every review matches the change to the running resources it names: Terraform resources, Kubernetes objects, environment variables and hostnames. The Infrastructure tab lists them with their neighbours one and two hops away.

1. In ArchCanvas, create an API key with the read scope. It starts with ack_.
2. In Settings, ArchCanvas, keep the address (https://archcanvas.cloud, or your own ArchCanvas), paste the key and press Connect ArchCanvas. The map starts as merged: the newest snapshot of every source.
3. To use another map, pick it from the list that appears once connected: the newest document, or one document by name. Then press Save ArchCanvas.
4. Press Test. It reports how many documents the key can read.

visualdiff only reads from ArchCanvas. See the docs for how matching works.

Notifications

In Settings, Notifications, add a Slack incoming webhook (its URL starts with https://hooks.slack.com/) or any HTTPS endpoint of your own, and choose Held reviews or Every review. Notifications are sent for reviews that came from a code host webhook, not for inbox or CI runs.

• Slack receives the verdict, the risk, the repository and number, the title, the summary and a link to open the review plan.
• Your endpoint receives a JSON event signed with X-Visualdiff-Signature: sha256=<hmac>. The signing secret is shown once when you add it.

{
  "event": "review.completed",
  "verdict": "hold",
  "risk": 72,
  "pr": { "host": "github", "repo": "acme/api", "number": 482, "title": "…", "url": "…" },
  "summary": "…",
  "url": "https://visualdiff.ai/r/…?t=…",
  "findings": 6
}

import { createHmac, timingSafeEqual } from 'node:crypto';

// rawBody: the request body exactly as received, before JSON.parse.
function fromVisualdiff(rawBody, header, signingSecret) {
  const expected = 'sha256=' + createHmac('sha256', signingSecret).update(rawBody).digest('hex');
  const a = Buffer.from(header || '');
  const b = Buffer.from(expected);
  return a.length === b.length && timingSafeEqual(a, b);
}

Review settings and policy

Owners and admins change these in Settings, Review; everyone else sees them read-only.

Workspace policy

Under Settings, Policy you edit the workspace policy as JSON. visualdiff picks the first policy it finds, in this order:

1. .visualdiff.json from the base revision of the pull request. Because it is read from the base, a pull request cannot loosen the rules it is judged by.
2. The workspace policy.
3. The defaults below.

Keys you leave out keep their defaults; a key you set replaces the default entirely, so a protected list replaces the default list. Reset to defaults removes the workspace policy. The policy reference explains each key.

{
  "version": 1,
  "maxChurn": 1200,
  "maxChurnAI": 600,
  "maxFiles": 60,
  "requireTestsFor": [
    "src/**",
    "lib/**",
    "app/**",
    "pkg/**",
    "internal/**"
  ],
  "protected": [
    {
      "path": "**/auth/**",
      "reason": "Authentication changes get a security review.",
      "level": "warn"
    },
    {
      "path": "**/migrations/**",
      "reason": "Schema migrations are reviewed by the data owner.",
      "level": "warn"
    },
    {
      "path": ".github/workflows/**",
      "reason": "Pipeline changes can leak secrets.",
      "level": "warn",
      "aiOnly": true
    }
  ],
  "forbidNewDependencies": false,
  "blockOn": [
    "critical"
  ],
  "requireHumanCommitForAI": false,
  "blockDriftForAI": false,
  "allowedHosts": [],
  "blockUnknownHosts": false,
  "approvedVendors": [],
  "blockUnapprovedVendors": false,
  "ignore": []
}

API keys

Keys let the CLI in CI and your own scripts act for the workspace. Owners and admins create them in Settings, API keys, from a signed-in session: a key cannot create another key.

• Name the key after where it will live, such as “GitHub Actions: acme/api”. Choose when it expires: in 30 days, 90 days, a year, or never.
• The key is shown once. Afterwards Settings shows its prefix, scopes, when it was last used and when it expires.
• Revoke stops it at once.
• Send it as Authorization: Bearer vdk_…. Each key is limited to 30 requests per 10 seconds.

curl -s "https://visualdiff.ai/api/v1/reports?limit=5" \
  -H "Authorization: Bearer $VISUALDIFF_KEY" \
  | jq '.reports[] | {repo, prNumber, verdict, risk}'

Every endpoint is listed in the API reference. To use a key from CI, see guest mode or an API key.

Security and data

• visualdiff reads pull requests and writes one comment and one status. It never pushes, merges or changes settings in your code host.
• Host tokens, the ArchCanvas key, webhook secrets, notification URLs and signing secrets are encrypted at rest with AES-256-GCM. The API never returns them; the webhook and signing secrets visualdiff generates are shown once, when created.
• Sessions and API keys are stored only as SHA-256 hashes.
• Webhooks are verified by HMAC signature or shared secret before anything in them is read.
• Addresses you enter, such as a self-managed server or ArchCanvas, are fetched over HTTPS only, and private or local network addresses are refused at every redirect.
• The diff and the files needed to resolve imports are fetched and analysed in memory; the report is stored in your workspace.
• The verdict comes from a deterministic engine. If the deployment has a model configured, it writes the summary from counts, paths and finding titles only. The model never sees your code or excerpts from it, and never sets the verdict.
• Logs are one structured line per event. Anything that looks like a secret is reduced to its length and last four characters.
• To keep everything on your own servers, self-host visualdiff; the Enterprise plan on the pricing page covers it.

Troubleshooting

Read next