Assessing code security tools for domain specialists’ work

An oil and gas company hires specialists for their knowledge of oil and gas. Some use code to support their day jobs without formal developer training. Their expertise gives the work its value; security tooling needs to fit how that work is done and provide a practical route to help when a finding needs investigation.

The company had no large-scale publicly available sites. Some B2B sites supported wholesale-gas purchases, alongside specialist analysis and automation. Internal tools and B2B services can still handle sensitive information and have commercial importance. The assessment needed to account for those uses when evaluating security scanning.

A principal engineer connected repository inventory, local analysis and AI-assisted research into an assessment system. The result was a repeatable basis for comparing security tools and preparing decision material.

The file footprint made fit a harder question

Repository reconnaissance identified languages, package managers, lock files, frameworks and delivery configuration. The assessment expansion added build patterns, infrastructure definitions and a view of recognised file sizes. The 15 May snapshot had this footprint:

Recognised format	Share of recognised-file bytes
JSON	64.62%
JavaScript	10.68%
Python	10.45%
TypeScript	5.24%
SQL	4.04%
YAML	1.73%
Other recognised formats	About 3.2%

JSON and YAML together represented 66.35% of recognised-file bytes. This was a data/configuration-heavy footprint alongside scripting, application and SQL source.

The measure sums file-tree sizes by recognised extension. It excludes unknown extensions and does not filter generated or vendor files. It describes stored-file footprint, not business-logic share, runtime exposure, quality or criticality. A JSON file is not automatically infrastructure configuration requiring a security check.

That variety made fit difficult. Evaluation needed to distinguish the work represented by the files and decide which checks would be meaningful for each type.

A practical sequence from inventory to assessment

The work grew from an existing tool through a concentrated expansion and a documentation follow-up.

Date	Engineering focus	Capability added
13 February 2026	Repository reconnaissance	Inventory of languages, packages and delivery configuration.
15 May	Estate analysis and local scanning	Richer profiles, controlled local mirroring, scanner adapters and common findings storage.
16 May	Research and decision outputs	Research briefs, result ingestion, ranking, report and deck generation, and dashboard preparation.
25 May	Operating documentation	A consolidated guide to the commands, inputs and outputs.

The principal engineering contribution was connecting discovery, analysis and reporting into a sequence whose inputs could be interpreted and reused.

One reporting path for different scanners

Local mirroring provided source copies for deeper analysis. Scanner adapters translated different tools’ output into a common finding structure, retaining the tool, rule, severity and source location. Shared storage supported consistent summaries while allowing engineering review to return to the detail.

This gave different checks a common reporting path. Their relevance still depended on the material being assessed: dependency and secrets checks, infrastructure checks and source analysis each answer different questions.

Repository access, clone limits, missing scanner binaries and execution failures can leave coverage gaps. Review needs to consider what actually ran before interpreting a quiet result.

Zoom outZoom inReset100%Close

Reference assessment workflow. Measured inputs remain separate from research claims; review considers security-tool fit for specialists’ work, with shared data supporting the shortlist, report, deck and dashboard preparation.

Fit required more than a language checklist

Language support was a starting point. Proposed pilot requirements included testing each candidate’s analysers, framework assumptions and build prerequisites against representative scripts, analysis work and applications. Website-oriented checks needed an appropriate target and scope for the smaller B2B footprint.

A pilot should also test whether findings explain the problem and offer useful help, how much irrelevant output needs triage, and how a code author reaches engineering or security support. Ownership matters: who investigates an uncertain finding, approves an exception or maintains a shared script? Those arrangements should support specialists in their work and accommodate differing development experience.

The implemented comparison excluded selected data/configuration categories, including JSON and YAML, then weighted the remaining recognised formats by bytes. That remainder still included markup, styles and infrastructure formats. The resulting fit and ranking scores were heuristics for review. They needed these workflow and ownership questions alongside the calculations.

AI research with a defined contribution

AI supplied research into capabilities, deployment options, integration and limitations. The dispatcher prepared structured briefs and ingested returned results; a separately run external agent performed the research.

The response structure carried sources and qualifications into the comparison. Ingestion selected research fields while preserving locally calculated coverage values. Product claims therefore remained distinguishable from repository-derived inputs.

Human review remains responsible for checking sources and applicability, including whether documented features help this mix of work. The companion article on bounded AI agents and verifiable results explains the handoff method.

Security within the assessment

Source copies and scanner results create sensitive assessment material. Their handling matters across internal automation and commercially important B2B services.

The implementation restricted the mirror directory to its owner and applied repository-size and free-space limits. Preserving local coverage values during research ingestion maintained the separation between measured inputs and generated claims.

Operating the workflow requires agreed read access and a private workspace. Scanner settings need review for outbound transfers. Research should receive approved, minimised context, with source, credentials and raw findings kept inside the agreed boundary. Report distribution also needs review. These operating responsibilities extend beyond the implemented local safeguards.

A decision package with a clear next step

The reporting layer brought estate summaries, findings and shortlist inputs into a written report and presentation deck. Shared summaries supplied dashboard data snapshots, with an interactive dashboard scaffold prepared for further integration.

A reviewed shortlist should explain which work a candidate suits, what support its users need and what a pilot must demonstrate. Engineering priorities need business context alongside severity and finding volume, especially where analysis or automation supports an important operational or commercial decision.

The delivered capability was an assessment foundation for that evaluation. Tool selection, supported triage and rollout each required further decisions. Related work on enterprise AI release engineering shows how defined checks can support subsequent delivery.

An Enterprise AI Delivery Assessment can define the decision, evaluation and responsibilities for your initiative. It is a paid engagement with scope and a fixed fee agreed after qualification.