Security Research

Intrudify research. Targets: Fider v0.33.0 and Photoview 2.4.0. Date of our runs: 28 June 2026.

At a glance

On the two applications Doyensec tested, Fider v0.33.0 and Photoview 2.4.0, there are four genuinely impactful bugs. Here is who caught them, and what happened to the one Critical both platforms reported:

Intrudify confirmed all four impactful bugs by exploitation, and correctly ruled out the Critical SQL injection that both platforms reported and that code review signed off on, but that does not actually inject. Everything below is sourced to a scan ID so it can be re-checked.

Summary

In 2026 Doyensec published a benchmark comparing two commercial artificial-intelligence penetration-testing platforms, Aikido and XBOW, against a set of open-source applications. The headline was a true-positive rate around 96 percent. Eighty of eighty-three reported findings were validated, mostly by manual code review.

We took two of those applications, Fider and Photoview, and re-tested the proof-requiring findings the way an attacker would: by running them live and trying to break them, then running our own combined static and dynamic scan against each. Two results stand out.

First, one of the agreed Critical findings is a false positive. Both Aikido and XBOW reported a Critical SQL injection (structured query language injection) in Photoview. Code review confirmed it. It is not exploitable. The database layer safely quotes the input, and every payload we sent came back as a harmless error. Our scanner looked at the same endpoint and correctly declined to call it an injection.

Second, when we ran Intrudify's own detection engine against both apps, it produced zero false positives, caught the genuinely impactful bugs, and stayed silent on the findings the commercial platforms got wrong. On Fider it confirmed the unauthenticated account takeover that is the application's one real Critical. On Photoview it confirmed the expired-share-token bug and the real Broken Object Level Authorization (BOLA) flaw that Aikido and XBOW also found.

The point of this writeup is not that our tool found more. It is a narrower, more technical observation, and one the study's own authors gesture at when they downgrade severities: when a finding is validated primarily by reading code rather than by exploiting it, impact tends to be over-counted. A real application is built in layers, and a tainted input reaching a dangerous function is not the same thing as a working exploit. The layer in between, an Object-Relational Mapper (ORM) that quotes identifiers, a sanitizer that strips scripts, an auto-escaping template, is exactly what code review tends to miss and what dynamic testing is built to catch.

Background: the study, and what it measured

The study is Doyensec's "Comparing AI Application Security Testing Platforms: Aikido vs. XBOW", by Luca Carettoni and Anthony Trummer. Doyensec selected two open-source applications at random from a curated list, Fider and Photoview, ran each commercial platform against both, and manually validated every reported finding as a true or false positive. Each scan was a standard commercial pentest priced at about 4,000 US dollars: Aikido's "Standard" tier and XBOW's "Plus" tier. It is a careful, transparent, peer-reviewed piece of work, and the headline numbers are strong:

• Aikido reported 51 findings across the two apps (19 on Fider, 32 on Photoview). Doyensec validated 49 as true positives and 2 as false positives.
• XBOW reported 32 findings (25 on Fider, 7 on Photoview). Doyensec validated 31 as true positives and 1 as false positive.

That is 80 of 83 reported findings validated, a true-positive rate of about 96 percent, with a false-positive ratio of 4 percent for Aikido and 3 percent for XBOW. We are not disputing any of that. We wanted to look closely at one specific thing the numbers do not capture: what happens to a code-review-validated finding when you actually try to exploit it.

The validation method is the reason that question is worth asking. In the report's own words, verification was performed primarily through manual code review, with dynamic testing added when possible, and peer reviewed to reduce error. That is a rigorous process. It also has a known blind spot, and Doyensec runs into it themselves: they re-scored the severity of about twenty-five of the validated findings, almost all of them downward, and note that they changed severities more often than they disputed validity. The reason is that a large share of the findings are missing security headers, cookie-flag advisories, verbose error messages, and other information disclosure. Real, in that they were observed, but low impact.

Why we focus on the impactful bugs

That last point shapes the whole comparison, so it is worth being explicit about it. Of the 80 validated findings, the large majority are low-impact hygiene issues, the kind Doyensec itself scored as Low or Informational. Findings like these are worth fixing, but they are not where tools distinguish themselves. Essentially every scanner reports them. The commercial platforms did, and Intrudify does too. A missing header is a missing header regardless of who surfaces it, so it carries almost no weight for a team trying to decide which tool to trust.

The findings that do matter, and where detection quality genuinely differs, are the handful that let an attacker take over an account, read another user's data, or run code. Across Fider and Photoview there are four of those. So that is where we concentrate, alongside the false positives. Those are the two things that actually change the outcome of a real engagement: catching the bug that leads to a breach, and not sending a team chasing one that was never exploitable.

Method: we attacked the apps, then scanned them

We deployed the same container images Doyensec used: Fider v0.33.0 (a Go feedback application backed by PostgreSQL) and Photoview 2.4.0 (a Go photo-gallery application with a GraphQL Application Programming Interface, or API, backed by MariaDB). We then did two things for each.

We reproduced the proof-requiring findings by hand, sending live payloads against the running application and reading the exact running source to confirm what the server actually does. And we ran Intrudify in combined mode, Static Application Security Testing plus Dynamic Application Security Testing (SAST plus DAST), authenticated, so we could compare our own output against the study.

Our two main scans, by scan ID, configuration, and wall-clock duration:

• Fider: scan fider-doyensec-creds-1782646191, mode dast_sast, authenticated with the admin credential so the agentic authenticator drove and captured the real sign-in flow, unlimited crawl, source code supplied for the static pass. Duration 71.5 minutes.
• Photoview: scan photoview-doyensec-creds-r2-1782651964, mode dast_sast, authenticated with two accounts (an administrator and an ordinary user) for cross-account testing, unlimited crawl, source supplied. Duration 57.2 minutes.

Finding 1: the agreed Critical SQL injection is a false positive

Both platforms reported a Critical SQL injection in Photoview, and code review signed off on it. The candidate is the GraphQL order_by argument, which flows into the Go ORM (GORM) as an ordering column.

Live, it does not inject. GORM wraps the value in backticks as a single quoted identifier, so a payload is treated as one column name, not as query syntax. Sending title-- - returns Unknown column 'title-- -' in 'ORDER BY'. A backtick-breakout attempt is escaped. A time-based payload, SLEEP(3), returns in nine milliseconds, which means nothing slept. There is no raw query construction, no string concatenation, and no unsafe execution reachable from user input anywhere in the 2.4.0 source. What looks like injection is a verbose error message disclosing a column name. That is a Low-severity information leak, not a Critical.

Intrudify's scan reached the same endpoint and filed it correctly: an information-exposure finding, explicitly not a SQL injection. This is the single most important comparison in the study, because it is a Critical that both commercial platforms reported, that code review confirmed, and that does not exist.

Finding 2: XBOW's Fider trio inflates one real issue and two non-issues

XBOW reported Cross-Site Scripting (XSS), Server-Side Template Injection (SSTI), and Remote Code Execution (RCE) on Fider. We could not reproduce any of them as described.

The XSS candidate is post and comment text, which Fider renders through a markdown pipeline that ends in DOMPurify with a hook that strips dangerous link schemes. Stored payloads come back inert. The SSTI candidate does not evaluate: a display name of {{7*7}} is stored and returned as the literal {{7*7}}, never as 49, because user input is template data, not a template. The RCE is a mislabeled dependency issue: there is no operating-system command execution in the Fider application, only a parser library with a denial-of-service advisory. Intrudify files that same dependency as Informational rather than inflating it.

So two of the most eye-catching findings across both apps, the Photoview Critical SQL injection and the Fider XSS, are over-classifications of behavior that a downstream layer neutralizes.

Why code review over-counts: a demonstration from our own static pass

Here is the uncomfortable part, and it is the heart of the argument. Static analysis, on its own, produces exactly this kind of over-count, and our own static pass is proof.

On Fider, Intrudify's SAST stage produced 42 raw findings, including 1 Critical and 8 High. The Critical and most of the Highs are Server-Side Request Forgery and XSS, flagged because user input reaches a fetch() call or a Document Object Model (DOM) sink. That is the same reasoning a code reviewer applies: tainted input reaches a dangerous function, therefore vulnerability. On Photoview, the static pass produced 18 raw findings, including 7 High, mostly vulnerable-dependency advisories.

If we shipped that raw static output the way a code-review-validated benchmark does, we would be reporting a Critical SSRF and eight Highs on Fider that are not exploitable, because those sinks live in client-side code and test files, behind layers that defang them. We do not. Intrudify's pipeline demoted all 42 Fider static findings down to 5 surfaced findings, every one of them Informational. On Photoview, 18 raw static findings became 1 Informational. The dynamic layer never confirmed the rest, so they were not allowed to masquerade as real.

This is the structural reason a static-only tool reports high false-positive counts as true positives. An application is layers, and static analysis sees the first layer and stops. The sanitizer, the ORM quoting, the framework auto-escaping, the access check one function deeper, none of those are visible to a reader following taint to a sink. Only running the application reveals them. Code-review validation inherits the same blind spot, and so does the current generation of artificial-intelligence security tools that work by reviewing code rather than running it. They follow tainted input to a sink and stop, never executing the layers that defang it, which is why a process scored that way can read as 96 percent accurate while shipping a Critical that cannot be exploited.

What Intrudify caught, with proof

Across both applications, every confirmed Intrudify finding came from a proof-required detector, what we call a brain, and there were zero false positives.

Fider, scan fider-doyensec-creds-1782646191, four brain findings:

• Critical, proven, Common Vulnerability Scoring System (CVSS) 9.9: account takeover via a client-controlled redemption secret. Fider's sign-in flow lets the caller supply the secret that is supposed to be emailed to the user. Our authflow brain drove the real flow, supplied its own secret on the issuance request, redeemed it, and landed a valid administrator session with no password and no inbox access. This is the application's one genuine Critical, the same unauthenticated account takeover Aikido and XBOW reported. The brain confirms it by exploitation, not by reading the binder code.
• High, proven: missing brute-force protection on /_api/signin. No throttle or lockout across repeated attempts.
• Medium, proven: username enumeration on the sign-in endpoint. The endpoint's response distinguishes a real account from a non-existent one.
• Low: missing security headers.

Photoview, scan photoview-doyensec-creds-r2-1782651964, four brain findings:

• High, proven: missing brute-force protection on /signin.
• Medium, proven: share-token expiry not enforced. A token minted with a past expiry still resolves, because the expiry column is written but never compared against the current time.
• Medium: Cross-Site Request Forgery on a settings form.
• Low: missing security headers.

And, importantly, silence where the commercial tools were wrong: no SQL injection, no real XSS, no SSTI, no confirmed SSRF.

The Photoview cross-account BOLA

Intrudify confirmed the real Broken Object Level Authorization bug in Photoview, the favoriteMedia flaw that Aikido and XBOW both reported (as IDOR, Insecure Direct Object Reference). With two accounts, an administrator who owns everything and an ordinary user who owns only their own album, the scan exercised the GraphQL favoriteMedia mutation across accounts and confirmed cross-account access by exploitation. The result is a clean, textbook confirmation:

Denied on the direct read, allowed on the mutation, control rejected. That triad is a confirmed cross-account BOLA, Common Weakness Enumeration (CWE) 639. A low-privilege user both writes a flag onto and exfiltrates the metadata of a photo it has no access to. The favoriteMedia resolver is gated only by @isAuthorized, meaning "are you logged in", and never checks ownership, while the media(id) query does. That mismatch is the bug.

This is the same impactful bug Aikido and XBOW found, confirmed the way Intrudify confirms everything in this writeup: by exploitation, not by reading the resolver.

Scorecard: the real impactful bugs, by impact not by count

Setting aside the misconfiguration noise on both sides, the genuinely impactful bugs across these two applications are few. Here is who catches what.

Two things fall out of that table. We match the impactful real bugs both commercial platforms found. And we avoid the false positives they shipped, including a Critical that code review validated.

What this does and does not prove

The honest limits. This is a benchmark on two open-source applications, not a head-to-head on a live client estate, and the comparison is asymmetric: we have full access to our own engine and only the public report for Aikido and XBOW, so we compare by vulnerability class, not line by line, where their findings are redacted. Username enumeration and the Cross-Site Request Forgery finding are real but modest.

A fair note on timing, and on why we did this. Doyensec ran Aikido and XBOW in March and April 2026. Both are actively developed platforms, and it is entirely possible that either has improved since then and would close some of the gaps above on a fresh run. Our goal was never to rank vendors or crown a winner. We wanted a fair, public, reproducible way to measure our own platform against two comparable commercial tools, on the same two applications, using findings the industry has already scrutinized. Read this as a snapshot in time. We would expect all three tools, ours included, to keep getting better, which is exactly why we think the discipline that matters most is not how many findings a tool produces, but whether each one survives an actual exploit.

What it does prove is the thing that matters for anyone choosing a tool. Validation by code review over-counts, because applications are layered and a sink is not an exploit. Our own static pass proves the failure mode, 42 Fider static findings including a Critical, demoted to five Informational once nothing confirmed them dynamically. Proof-required dynamic detection is what separates a real finding from a plausible one. On these two apps it gave us zero false positives, the genuine account takeover, the genuine cross-account access bug, and silence on the Critical that was never real.

A scanner that calls a non-exploitable bug Critical costs a team a weekend. A scanner that misses the real one costs them the breach. The discipline that avoids both is the same: report only what you can break, and show the break.

← Back to all posts