Product

Audit findings

When a scan of your code drafts the specification of a feature or a component, Productmind checks the draft as soon as it arrives. Each problem the check finds is an audit finding, listed above the draft, for example 2 audit findings, 1 kept this draft from applying itself. This page explains what each finding means, why some of them keep a draft from applying itself, what to do about each one, and how to check what a scan did.

Key concepts

Draft. The specification a scan proposes for a feature or a component. It replaces the whole document. You compare it with the current document and apply it, unless it applies itself. The API calls a draft a description proposal.

Audit. The check Productmind runs on every draft that a scan or a coding session submits, as it arrives. It never refuses a draft: it records what it found, for whoever reviews the draft.

Audit finding. One problem the audit found, described in one sentence. It has nothing to do with the Audit log, which records who changed what.

Blocking finding. A finding with a red blocking badge. It keeps the draft from applying itself, so the draft waits for your review. It never stops you from applying the draft yourself. Every other finding is advice for the person who reviews the draft.

Applies itself. Said of a draft that becomes the specification with nobody reviewing it first. The app calls this auto-apply. Only some drafts can, and only when nothing blocks them. See When a draft applies itself.

Link. The connection between a feature (or a component) and one of its linked applications: an application that implements it. You make one with Link applications, and scans make them too. A link also holds the feature's code surface in that application, so a finding that says "the link should be corrected" means the code surface should be corrected.

Code surface. The files and entry points that implement a feature in one linked application, shown under that application in the feature's Applications panel. Scans record them, and you can add paths of your own, or correct one a scan recorded, which are marked Manual. Findings call it the surface, the recorded surface or the stored surface.

Recorded code paths. The files and folders on a code surface: the ones scans recorded and the ones added by hand.

Entry points. The ways into a feature's code that a scan recorded on its code surface, such as a screen, an API route or a scheduled job, each with the file and line where it starts.

The link-feature-application call. How a scan records a code surface. link-feature-application is the API operation Link a feature to an application it depends on, which AI assistants you connect over MCP can use too. A scan of a component uses it, before it submits its draft, to record the files it found belong to the component. When a finding says ownership "was settled by the link-feature-application call", it means the scan had already recorded those files as the component's.

Composition. A feature's specification written from its components' specifications, when you choose Compose from components. See Features.

A scan also writes some sections into its own draft. These are part of the draft, not audit findings:

Findings section. The Findings heading under the Summary of a component's draft. The Component specification template asks the scan to report five checks there, one line each:

  • the TODO, FIXME, HACK and XXX comments in the component's code, quoted;
  • the fields whose length, default, or whether they can be empty, differ between the places in the code that define them;
  • each limit that a comment, a message or a label states, and whether the code enforces it;
  • the Code surface line, which counts the files the draft was written from, for example: Code surface — 12 path(s) across 2 application(s), as link-feature-application now records them. Audit findings call it the Code surface receipt;
  • the branches that no code can reach.

A line that starts with Attributed elsewhere names a file the scan decided belongs to another component.

Open questions. What the scan couldn't determine from the code, such as a value set only while the product runs, or code in a repository the scan wasn't given. Each one says what the scan searched, and what you should check.

Decisions for the product owner. Things the code settles today that a person should confirm. Each one says what happens today, the decision to make, and which rule the answer would change.

Documented-but-false. A statement the code contradicts: in the current specification, a code comment, a document, an error message or a label. A scan is asked to report each one it finds, in its summary and in the draft, rather than quietly drop it.

Where audit findings appear

In the comparison

When a scan's draft is waiting for review, Review proposed changes opens over the feature's document when you open the feature. To open it again after you close it, choose Review changes in the header. From the top, it shows:

  • Review proposed changes, and under it the scan's summary of what it did. Only the first two lines of the summary show. To read all of it, see To see the findings of a draft that has already applied, which works for every draft.
  • How long the scan ran and how many tokens it used, for example 9m 41s · 1,204,377 tokens.
  • When someone changed the document after the draft arrived, This document changed since the scan — the comparison below is against the current version. The comparison is always against the document as it is now.
  • The findings banner, when the audit found anything. It says how many findings the draft has, and how many of them are blocking, for example 2 audit findings, 1 kept this draft from applying itself. Choose Show details to list the findings, one sentence each, and What do these findings mean? to open this page. A blocking finding starts with a red blocking badge.
  • The changes: the current document on the left, the draft on the right.

When the draft writes business rules into its text, a Business rules panel beside the changes lists them.

In the header, after a draft applies

When a draft applies, by itself or because you applied it, the feature's header shows audit: clean when the audit found nothing, or audit: N findings, such as audit: 3 findings. Point at it to see what it means.

The badge shows only when the draft applied while you had Productmind open, and it's gone once you reload the page or come back to the feature later. The findings are kept with the draft: see To see the findings of a draft that has already applied.

In the Features list

A row whose draft is waiting for review has a review button. Point at it to see A proposed specification is waiting for your review, and choose it to open the feature and its comparison. To list every feature in that state, choose Filters, then Deep scan › Waiting for review. See Filtering, sorting and grouping features.

Reviewing a scan's draft

To read a scan's audit findings

  1. Open the feature or the component.
  2. In Review proposed changes, find the banner under the summary, such as 2 audit findings, 1 kept this draft from applying itself, and choose Show details.
  3. Look up each finding in The findings, one by one, by how its sentence starts.

The list stays open until you choose Hide details. A finding with a red blocking badge is why the draft didn't apply itself.

To resolve a blocking finding

A blocking finding doesn't stop you from applying the draft: it hands the decision to you.

  1. In the list of findings, look up the blocking finding in The findings, one by one.
  2. Fix what it points at. Choose Edit to change the draft, or Reject on a change you don't want. When the finding doesn't matter for this draft, change nothing.
  3. Choose Apply N changes, for example Apply 3 changes, or Apply once you've edited the draft.

The draft becomes the specification, with the changes you kept. Productmind doesn't check your edited version again: the draft keeps the findings it arrived with, and the header's badge counts them.

If the draft isn't worth fixing, choose Dismiss instead, and scan again (see Features). Each scan has a cost: see What deep reads cost. When the dismissed draft was the first specification of an entry that feature discovery created, the next scan's draft can still apply itself.

When a finding is about the code surface, correct the code surface before you scan again. See To correct a code surface.

To check what a scan read

  1. In Review proposed changes, read the scan's summary: what it read and what it decided.
  2. Choose a citation in the draft, at the end of a statement, to open the code it cites in your Git provider, at the commit the scan read.
  3. Open the Applications panel, and choose each linked application to see the files and entry points recorded for the feature. Choose a path to open it in your Git provider.
  4. In the draft, read Open questions and Decisions for the product owner, and in a component's draft, the Findings section.

You know what the scan read, what it couldn't settle, and what it wants a person to decide. A statement with no citation, or whose citation doesn't show what the statement says, is the one to check first.

The paths in the Applications panel open at the commit of the last scan that applied, or on the repository's branch as it is today when none has. Point at the repository chip on the application's row to see which. A draft that waits for review may have read a newer commit: its citations name the commit it read.

To correct a code surface

  1. Open the feature or the component, and open the Applications panel.
  2. Choose the application, to show its code surface.
  3. Choose Add code path to add a file or folder that's missing. To correct a path, point at it, choose its pencil, change the path and press Enter. To remove a path marked Manual, point at it and choose its x.

The next scan reads the corrected code surface, and Productmind checks its draft against it. A path a scan recorded can be corrected but not removed: see I can't remove a path a scan recorded. If the feature doesn't use the application at all, unlink the application instead. See Code surface.

To see the findings of a draft that has already applied

Productmind keeps every draft of a feature, with its findings and its whole summary, whether it applied, was dismissed or failed. The app shows them only while a draft waits for review, and for a short time after it applies. The API lists them all.

  1. Get an access token. See API access and authentication.

  2. Find the feature's id with the List product features in a workspace operation: pass its title as search, and parentFeatureId=all to include the features and components that sit under another feature.

  3. Call List description proposals for a feature with that id:

    TerminalCode
    curl "https://api.productmind.ai/workspaces/<workspaceId>/features/<featureId>/description-proposals" \ -H "Authorization: Bearer <access_token>"

The response lists the feature's drafts, newest first. In each one:

  • auditResult.violations holds its findings, each sentence in its detail;
  • status says whether the draft was applied, dismissed, or is still waiting;
  • summary holds the scan's whole summary;
  • scannedCommits names the commit each repository was read at.

Add ?status=applied to the address to list only the drafts that applied. The API Reference documents every field, under Features.

How a scan's draft is checked

Every draft is checked, and none is refused. Productmind checks every draft a scan or a coding session submits, whatever the feature. A feature isn't picked out for an audit: its findings are what the check found in that draft. A finding never stops a draft from arriving.

The draft is compared with what Productmind holds. Productmind compares every draft with your workspace's specification template: Feature description for a feature, Component specification for a component, on the Templates tab of Agents. It compares a component's draft with its code surface and its linked applications too, as they are when the draft arrives, so a scan that corrected the code surface during its run is checked against the corrected one.

The code-surface checks run on components only. A component is where a feature's code lives, so a feature's draft isn't checked for entry points, code paths, linked applications or the Code surface line. Nor is it reported for citing a business rule it doesn't define, because a feature's rules belong to its components.

Blocking means the draft isn't the document it should be. A blocking finding says the draft is incomplete or unfinished, copied the template's example, has broken links to your code, contradicts itself, or contradicts Productmind's record of its code surface. Everything else is advice for the reviewer: wording, length, and parts of the code the draft doesn't cover. Advice never keeps a draft from applying itself, so a large scan doesn't turn into a queue of reviews.

Edits aren't checked. The findings belong to the draft as the scan submitted it. Editing the draft, or correcting the code surface afterwards, doesn't change them.

When a draft applies itself

A draft becomes the specification with nobody reviewing it only when all three of these hold:

  1. It's a first specification, or a composition. Either it's the first specification a scan writes for an entry that feature discovery created, before any scan's draft has applied to it, or it composes a feature from its components' specifications.
  2. Nobody changed the document while the scan ran.
  3. The audit found nothing blocking.

Otherwise the draft waits for your review. A refresh of a specification that already exists, a scan of an entry that a person created, and a draft from a coding session always wait.

The findings, one by one

The first column is how the finding's sentence starts in the app. ... stands for the part that changes from draft to draft, such as a section's or a file's name, and N and M for numbers. In the app, a finding writes section names with ## or ### in front, and shows some words between backquotes (`) or double asterisks (**): it is written in the same markup as the specification.

The finding starts withBlockingWhat it means, and what to do
`## ...` is missing.YesA section your specification template asks for isn't in the draft. A PRFAQ section may be left out, and is never reported. Add the section when you edit the draft, or dismiss the draft and scan again.
`## ...` is not a section of the template.NoThe draft has a section your template doesn't. Choose Reject on it, or keep it if it's useful.
`## ...` appears where `## ...` was expectedNoThe sections aren't in your template's order. Only the first one out of place is reported. Reorder them when you edit the draft, or keep the order.
Contains "etc." or Contains "various"NoThe draft gestures at a list instead of naming its members. The quoted words can also be "and so on", "among others", "a number of", "including but not limited to", "as appropriate", "where applicable" or an ellipsis. Replace them with the list, if you know it.
Contains N unfinished marker(s)YesThe draft still says TODO, TBD, FIXME or "to be confirmed": parts the scan didn't finish. Words in code formatting, and the Conflicts & Gotchas section, aren't counted. Finish or remove those parts when you edit the draft, or dismiss it and scan again.
Contains "..." from the template's worked exampleYesThe draft contains a name from the example your template uses to show its shape (a booking, a venue or a call-routing example), so it may describe the example instead of your code. If the name isn't in your code, remove that part, or dismiss the draft and scan again. If it really is in your code, for example your product uses the Europe/Amsterdam time zone, apply the draft.
Cites ... but defines no matching `### BR-n` heading.YesThe draft refers to a business rule it doesn't contain. Only a draft that defines rules of its own is checked for this. Remove the reference when you edit the draft, or dismiss it and scan again.
N rule(s) carry no `- **Evidence** ...` quote or No rule carries an `- **Evidence** ...` quoteNoRules that don't quote the line of code that enforces them. An unquoted rule is the most likely to be wrong, so check those rules against the code. When no rule has a quote, the scan most likely followed a customized Feature deep read template that predates quotes: compare it with the default on the Templates tab.
N rule(s) carry no `- **Enforced at** ...` lineNoRules that don't say where they're enforced, which reads as "enforced everywhere". Check where the code enforces them.
N rule(s) omit `server` from **Enforced at**NoRules whose Enforced at line leaves out the server, such as a rule only the screen enforces, and doesn't say what gets past the check, such as a call to your API. This can be a real gap in your product: check whether the server should enforce the rule.
N characters is below the N floorYesThe draft is too short to be a full specification: under 6,500 characters for a component, or 6,000 for a feature. Dismiss it and scan again, or apply it if the component really is that small.
`## ...` is N characters against a ~N budgetNoA section is more than one and a half times the length the default template budgets for it. The finding says itself that it's advice only. Shorten the section if you like.
N citation(s) are still the prose formNoReferences to code that couldn't become links, usually because the feature has several linked applications and the reference doesn't say which repository it means. They show as plain text. Check them by hand if you rely on them.
N `coderef:` link(s) do not parseYesLinks to your code that are broken and can't be opened. Fix or remove them when you edit the draft, or dismiss it and scan again.
Summary claims N rules; the submitted markdown defines M.YesThe scan's summary says it wrote N rules, but the draft defines M: rules may be missing from the draft, or the summary is wrong. Check the draft's rules, and scan again if some are missing.
N recorded entry point(s) appear nowhere in the specificationNoEntry points on the component's code surface that the draft never mentions. Either the draft is incomplete, or they belong to another component and the code surface is wrong. Check each one.
N of M files on this component's code surface are not mentioned in the specificationNoFiles on the component's code surface that the draft never mentions, in a citation or anywhere else. Test files aren't counted. See the troubleshooting entry.
An open question asks whether N file(s) belong to this componentYesThe draft asks whether files belong to the component, when the scan had already recorded them as the component's. See the troubleshooting entry.
`### Findings` carries no **Code surface** receipt.NoThe draft lacks the line that counts its code surface. The finding quotes the line with the right numbers: add it when you edit the draft if you want the draft to carry it. Raised only when your Component specification template asks for the line.
The **Code surface** receipt claims N path(s) across M application(s)YesThe draft's count of its code surface doesn't match what's recorded for the component, so it was written from another list of files. See the troubleshooting entry.
No rule, decision or entry point cites `...`NoThe component is linked to an application the draft never mentions. If the component doesn't use that application, unlink it. Otherwise the draft missed part of the component.

Permissions

ToYou need
See a scan's draft, its audit findings and the audit badgeFeatures: Read
Edit, apply or dismiss a draftFeatures: Update
Scan againFeatures: Update
Add, edit or remove code paths, or unlink an applicationFeatures: Update
List a feature's drafts and their findings through the APIFeatures: Read
Replace the code paths a scan recorded, through the APIFeatures: Update

A private feature's teams change this, on that feature and everything under it. See Private features.

Troubleshooting

A finding says files on the code surface are "not mentioned in the specification"

The finding reads, for example: "2 of 7 files on this component's code surface are not mentioned in the specification (test files are not counted): ... Either the specification misses part of the component, or these files don't belong to it and should come off its code surface." It doesn't block.

Those files are on the component's code surface, recorded by a scan or added by hand, but the draft never mentions them, in a citation or anywhere else. A folder counts as mentioned when the draft cites any file in it. Test files are left out: a scan records them because they show how the code behaves, but a specification doesn't need to cite them. Either the draft doesn't describe part of the component, or the files no longer belong to it. To find out which:

  1. Open the Applications panel, and choose the application to show the component's code surface.
  2. Choose each path the finding lists, to open it in your Git provider.
  3. Decide whether each file belongs to the component.

If a file belongs to the component, the draft is incomplete: add what it misses when you edit the draft, or scan the component again. A file that only declares an interface, such as a port or a contract other code implements, is often fine to leave unmentioned. If a file doesn't belong to the component, correct the code surface. See To correct a code surface.

A blocking finding says an open question asks whether files belong to this component

The finding reads, for example: "An open question asks whether 1 file(s) belong to this component, but they are already on its code surface: "invoice-service.js". Answer or remove the question, or take the files off the code surface if they don't belong here."

Before a scan of a component submits its draft, it records which files belong to the component on each of its links. The draft's Open questions then asks whether one of those same files belongs to the component. The draft contradicts its own record, so Productmind holds it for a person to decide.

The check reads each question in Open questions on its own. It's raised when one question names a recorded file, by its name without its folder, and the same question uses a whole word such as "own", "owned", "ownership", "belong", "in scope", "out of scope", "attributed to" or "this component's", or speaks of the "code surface". Words that only contain these, such as "known" or "owner", don't count, and neither does a word in a different question.

  1. Open the Applications panel, choose the application, and find the file the finding names.
  2. Read the question in Open questions, and decide whether the file belongs to the component.
  3. If it does, choose Edit, remove the question or answer it, and choose Apply. If the question isn't about which component owns the file, choose Apply N changes to apply the draft as it is.

The draft becomes the specification. If the file doesn't belong to the component, the code surface is wrong instead: correct it (see To correct a code surface), then dismiss the draft and scan the component again.

A scan's draft didn't apply itself

One of these is true:

  • It isn't a first specification or a composition: it refreshes a specification that already exists, it's a scan of an entry that a person created, or a coding session proposed it.
  • Someone changed the document while the scan ran.
  • The audit found something blocking: the banner says how many findings are blocking.

Open the feature: the draft waits in Review proposed changes. Apply it, or dismiss it. See When a draft applies itself.

The Code surface line disagrees with what's recorded

The blocking finding "The **Code surface** receipt claims N path(s) across M application(s), but ..." means the draft counts different numbers from what's recorded for the component now. The finding gives both. Usually the scan corrected the code surface during its run, then kept counting the list it started with. A path someone added or removed, or an application linked or unlinked, while the scan ran has the same effect.

Compare the line with the Applications panel, which counts the paths recorded under each linked application, for example 6 paths · 5 entry points. When you edit the draft, correct the line and any sentence about which files the component owns, then apply it. Or dismiss the draft and scan again.

I can't remove a path a scan recorded

Only a path you added, marked Manual, has a remove button. A path a scan recorded changes when the code surface is recorded again: a scan of the component is designed to record its corrected code surface before it submits its draft.

To correct it now, edit it: point at it in the Applications panel, choose its pencil, change the path and press Enter. Your path takes its place, marked Manual, and the correction stays: a later scan doesn't bring the old path back, even when it records it again. See Code surface.

To replace the whole list a scan recorded, call the API's Link a feature to an application it depends on with the full list of code paths you want for that application: the list replaces the one scans recorded, and paths marked Manual and your corrections stay. See the API Reference, under Features. If the feature doesn't use the application at all, unlink the application in the Applications panel instead.

The findings disappeared after I reloaded the page

The header's audit: clean or audit: N findings badge shows only for a short time after a draft applies. The findings are kept with the draft. See To see the findings of a draft that has already applied.

Last modified on