Part 5 of 6 — Series: Generative AI in a Regulated Financial Institution
The previous part addressed where data should pass: a gateway under the institution's control, infrastructure located in Brazil, fail-closed behavior, and an audit trail with retention defined by evidence category.
Architecture, however, does not determine how code should be generated. That is where many programs applying AI to development leave a gap: the model produces plausible code, but plausibility is not correctness. When context is incomplete, the model fills gaps with assumptions, which may reach the repository without a link to a documented business decision.
The premise of this article is that, in a regulated environment, the specification does more than document. It can function as a technical control and as part of the process evidence.
AI-generated code is not exempt from regulatory responsibility#
Using AI does not transfer responsibility for the software to the model vendor. The institution remains responsible for the system placed into production, while developers, reviewers, and other professionals are accountable for their roles in development, validation, and approval. CMN Resolution 4,893/2021, amended by CMN Resolution 5,274/2025, reinforces the need for procedures and controls for secure information-system development and the adoption of new technologies. When external tools become part of that process, controls related to procurement, third-party management, access, traceability, and data protection also apply. The exact classification depends on the architecture and service-delivery arrangement. In practice, the institution must define how the tool may be used, which instructions and restrictions should guide it, and how its results will be reviewed before reaching production.
The point is not to prevent AI from being used in the development lifecycle, but to avoid its use without process, criteria, and evidence. The same principle that justifies an explicit critique and validation stage in investment assistants also applies to code generation: “the AI generated it on its own” does not transfer responsibility or explain which controls were applied. Without documented instructions, acceptance criteria, and a record of output review, it is difficult to demonstrate to Compliance, auditors, or regulators that the tool operated within the institution's defined parameters. That is where the spec makes the difference: it turns intent into a verifiable rule and generation into a traceable process.
The five anti-patterns an auditor will find in your process#
1. Vibe coding: accepting the first output and committing it#
Andrej Karpathy named the practice in February 2025: accepting changes without reading diffs, feeding error messages back to the model, and working around bugs rather than investigating them. In the end, the developer assumes authorship of code they have not reviewed.
In a Veracode evaluation of more than one hundred models, 45% of the samples generated for the study's tasks failed security tests. It is a relevant result, but limited to the scenarios evaluated by a security vendor—not a universal vulnerability rate for AI-generated code. A research note from the CSA (Cloud Security Alliance) also cited an earlier study reporting a 37% increase in critical vulnerabilities after successive iterations without structured human review. Because the reference is indirect, the figure needs to be interpreted cautiously. Even so, the warning remains: iterating without a human gate does not guarantee improvement and can increase risk. The mismatch between perception and productivity also appears in the data. In a METR study conducted with early-2025 tools, 16 experienced developers took an average of 19% longer to complete tasks with AI, although they believed they had gained approximately 20% in speed. The result is useful as a warning about perceived productivity, but should not be generalized to all teams or generations of tools. METR itself published an update to the experiment in February 2026, indicating that this situation may be changing.
Without reading the diff, there is no evidence that the generated output was actually checked before entering the software lifecycle. In regulated environments, this review may be required by the secure-development policy, segregation of duties, and the change's risk level.
2. Vague prompts: "build a payment endpoint"#
A prompt such as “build a payment endpoint” communicates intent but does not define the contract. Without types, business rules, security constraints, and edge cases, the model has to fill gaps with assumptions. In a financial product, those gaps may involve fees, limits, rounding, idempotency, and error-handling conventions. The resulting code may appear coherent and technically correct, even when it implements a rule that the business never defined.
The same problem may go unnoticed in tests. If the model invents a behavior and then generates a unit test using that same assumption as the expected result, the test passes. It proves only that the code matches the premise created by the model itself, not the product's actual rule. The divergence may appear only during integration or in production.
A recent review of spec-driven development, published in 2026, brings together still-early studies indicating improved generated-code quality when human-refined specifications replace vague prompts. Some cited studies reported error reductions of up to 50% in controlled scenarios. Because the evidence is still emerging, that figure should be interpreted as an experimental result, not a guarantee applicable to every project.
In a regulated financial product, code based on presumed business rules may have test coverage but lacks process coverage: there is no documented decision explaining where that rule came from.
3. Excessive context: pasting an entire production file#
The intuition is simple: the more context the model receives, the better the response. In practice, useful context is not synonymous with abundant context. A paper on context engineering published in August 2025 proposes targeted selection and injection of relevant information to improve coding assistants' adherence to complex projects. The practical conclusion is that filling the context window is not enough. Entire files can add redundant information, conflicting instructions, and details unrelated to the task, reducing the model's ability to identify what really matters without necessarily signaling that loss of quality.
The safer approach is to provide the minimum sufficient context and retrieve additional information only when necessary. The risk, however, is not limited to output quality: pasting production files into a prompt can expose proprietary code, credentials, secrets, personal data, and information protected by internal policies or contracts. If the content reveals client operations, services, or information, it may also involve the duty of confidentiality under Complementary Law 105/2001. The corporate hook presented in Part 4 of the series acts as a preventive barrier: before submission, it applies rules to detect personal data, secrets, credentials, protected files, and code patterns classified by the institution.
This anti-pattern therefore creates two risks at once: silent degradation of the result and improper exposure of information that should not leave the corporate perimeter.
4. Auto-merge: a PR with green automated gates and no human review#
The SonarQube AI Code Assurance feature identifies projects containing AI-generated code and subjects them to specific quality standards, with static analysis and stricter quality gates. These controls matter, but they only check what can be translated into automated rules. A passing pipeline can demonstrate that code compiled, passed existing tests, and did not violate configured rules. It does not prove, by itself, that the implementation matches the expected business rule. A semantic error in a credit operation—such as inadequate precision in interest calculations, rounding different from the contract, or incorrect handling of an exception condition—may pass SAST and all available tests when the correct behavior is not represented in contracts, analyzable rules, or test cases.
Treating a green pipeline as automatic merge authorization therefore removes an important point for checking business logic. The need for this review should be defined according to the change's criticality, the secure-development policy, and the institution's segregation-of-duties rules. The NIST Secure Software Development Framework, for example, organizes security practices that must be integrated throughout the development lifecycle, rather than concentrated in a single tool or automated stage. Technical gates and human approval are also distinct controls within the repository workflow itself. On platforms such as GitHub, status checks and mandatory pull-request reviews can be configured separately in branch-protection rules.
In an audit, the institution may need to demonstrate how higher-risk changes were analyzed and approved. A green pipeline records approval by automated controls; it does not replace evidence of business-logic review when that review is required by risk.
5. Blind delegation to the agent: “resolve every Sonar finding”#
Asking an agent to automatically analyze and fix all SAST findings seems like an efficient way to eliminate technical debt. The problem arises when fixes are applied in bulk, without a defined scope for each item or individual review of impacts. In the financial domain, behavior that looks incorrect in isolation may be intentional: silent error handling, for example, may be part of an idempotency strategy to prevent a transaction from being reprocessed. By “fixing” that behavior based solely on a generic rule, the agent can produce a technically convincing solution while introducing a subtle regression in business logic.
The OWASP Top 10 for Agentic Applications 2026 classifies as ASI09 Human-Agent Trust Exploitation the risk that confident, well-presented responses lead human operators to approve harmful actions. Automatically fixing a SAST finding is one possible example of that mechanism: the change appears safe, satisfies the analysis tool, and may even improve code locally, but changes a behavior the agent does not fully understand. ISO/IEC 42001, in Clause 10, addresses continual improvement and the handling of nonconformities and corrective actions. In the development process, this means a finding must be analyzed, treated, and closed with evidence proportionate to risk, not merely removed from the tool's list. Bulk fixes without per-item justification and review weaken that traceability. The scanner may turn green, but the most important question remains unanswered: what changed, why was the solution appropriate, and how did the institution verify that it did not introduce a regression?
The problem is not allowing the agent to propose fixes, but turning the proposal into an approved change without preserving the link between the finding, the decision, the review, and the test results.
The five anti-patterns in brief:
Enlarge ↗Seven stages with explicit gates: making generation traceable#
In the 2025 Technology Radar, Thoughtworks described spec-driven development as an approach in which well-crafted requirement specifications guide AI agents in generating executable code. The analysis itself, however, includes a caveat: when the specification becomes excessively detailed and tries to define all the work before implementation begins, the process may lose agility and approach a waterfall model.
The seven-stage flow presented below reduces this risk by limiting each spec to a unit of work: a function, endpoint, or business rule. The specification guides the next change without trying to anticipate the whole system.
Tools such as Kiro, GitHub Spec Kit, and Tessl already offer specification-driven workflows covering requirements, technical design, planning, and implementation. These tools, however, do not automatically incorporate the controls, evidence, and responsibilities specific to a Brazilian financial institution. Access rules, data protection, independent review, segregation of duties, traceability, and evidence retention still need to be defined by the organization itself.
The following flow adds this governance layer to the spec-first process.
Enlarge ↗
Enlarge ↗Diff review, in Stage 4, is the practical application of the control that Part 3 of this series related to LLM05 Improper Output Handling in the OWASP Top 10 for LLM Applications. The risk described by OWASP lies in using model outputs without adequate validation, treatment, or checking before they are passed to other components. In software development, this means not automatically accepting or executing generated code. Reading the diff, understanding the changes, and checking their impacts is what turns model output into a change consciously taken on by the developer.
Even so, the author's review does not eliminate the need for Stage 6. Human code review and automated gates serve different purposes.
Tools such as SonarQube AI Code Assurance check properties that can be represented by rules: known vulnerabilities, bugs, complexity, duplication, coverage, and other quality and security indicators. These controls are essential, but they work on what has been encoded in rules, tests, or analyzable contracts. The human reviewer evaluates another dimension: whether the implementation matches the behavior expected by the business. That includes checking whether types are appropriate to the financial domain, calculations follow product rules, rounding is correct, and error handling reflects a documented decision rather than a plausible model assumption.
Stage 6 therefore does not repeat Stage 5: the pipeline demonstrates that code passed the automated controls defined by the institution, while code review demonstrates that someone other than the author assessed the meaning and impacts of the change before approving it.
The spec as a model instruction and a demonstrability artifact#
In June 2025, Andrej Karpathy popularized the concept of context engineering: the discipline of selecting and organizing exactly the information a model needs to perform its next step. In AI-assisted development, the spec serves that role: it bounds context, makes rules explicit, and reduces the scope for the model to fill gaps with assumptions. At the same time, it becomes part of the process evidence: it records which parameters, restrictions, and criteria should guide generation.
The spec alone does not prove that the model received those exact instructions. For that, it needs to be correlated with the session identifier, gateway records, diff, PR, and automated-control results. Together, these artifacts help demonstrate how generation was conducted and reviewed.
In this model, the spec has five essential components:
1. Technical context#
Language, framework, versions, authorized libraries, and project conventions: without this information, the model tends to assume generic defaults that may diverge from the institution's architecture, supported versions, or security practices.
2. Explicit contract#
Inputs, outputs, types, possible errors, and expected behavior: in Java, for example, monetary values should use , with a defined scale and rounding mode, rather than floating-point types such as . An intent can be interpreted; a contract can be verified.
3. Enumerated business rules#
Fees, limits, rounding, idempotency, priorities, and exception conditions must be explicit: the model does not automatically know a financial product's proprietary rules, and when they are absent from the spec, it may replace them with plausible behaviors that do not exist in the business.
4. Security restrictions#
The spec must also record what code may not do: log personal data, include credentials in code, ignore input validation, or use error handling outside the defined standard. These controls relate to the risk of LLM02 Sensitive Information Disclosure in the OWASP Top 10 for LLM Applications, and to secure-development and new-technology-adoption requirements reinforced by CMN Resolution 5,274/2025.
5. Test cases and acceptance criteria#
The spec should cover the success path, null or invalid inputs, boundary values, exceptions, and out-of-domain behavior: the study “Tests as Prompt: A Test-Driven-Development Benchmark for LLM Code Generation” explores tests as part of the specification provided to the model and, at the same time, as a mechanism for verifying generated code. There is also a practical benefit for diff review: when expected results are defined before generation, the developer knows what to check. The analysis stops relying solely on how code looks and begins comparing it with previously documented criteria.
Example spec with all five components:
Enlarge ↗The spec is not a system-architecture document; it is granular and covers a task. Each iteration begins with a new spec, preventing context invented in one session from contaminating the next.
Process artifacts as evidence of control#
Parts 2 and 3 of this series mapped the applicable regulations and frameworks. Here, the question is more practical: which artifacts help demonstrate that AI-assisted development followed the controls defined by the institution?
The CMN Resolution 4,893/2021, amended by CMN Resolution 5,274/2025, determines that security procedures and controls also cover information-system development and the adoption of new technologies. When an external tool begins participating in that lifecycle, the institution must assess controls related to procurement, identity and access, data protection, traceability, and third-party management according to the architecture and service-delivery arrangement. In this process, the spec records the rules, restrictions, and acceptance criteria that should guide generation, but does not satisfy a regulatory obligation in isolation or prove on its own that those instructions were actually sent to the model. To function as evidence, it must be correlated with the generation session, diff, PR, commit, and records produced by the gateway and engineering tools.
In 2025, ANBIMA published a non-binding Secure Application Development Policy template, guided by the NIST Secure Software Development Framework SSDF v1.1, reinforcing that security controls must accompany the entire software lifecycle. AI-assisted code is also part of that lifecycle. The spec, session identifier, commit attribution, scanner results, PR approval, branch-protection rules, and gateway logs are complementary evidence. None of them alone proves that the entire process was performed properly. Correlation between these records is what allows reconstruction of what was requested, generated, changed, checked, and approved.
There is also an important distinction concerning the LGPD. When software placed into production makes decisions based solely on automated processing of personal data that affect data subjects' interests, Article 20 guarantees the right to request review of the decision. Code review is another control: it contributes to software quality and governance, but does not replace the automated-decision review process provided for by law.
The ISO/IEC 42001, in turn, structures a management system to establish, implement, maintain, and continually improve AI-related processes. In software development, this means a nonconformity should not result only in an isolated fix: it can also inform new tests, rules, policies, or controls capable of preventing the problem from recurring.
The retention period should be defined by evidence category. Specs, PR approvals, gateway logs, scanner results, and incident records may have different retention periods according to their purpose, criticality, internal policy, and applicable obligation.
Without a spec, it becomes harder to demonstrate which instructions, restrictions, and criteria guided generation. Without diff review, evidence that the output was checked is missing; without recorded code review, there is no evidence of independent assessment when risk requires it.
Artifacts do not individually prove compliance. Together, they help demonstrate that the engineering process was defined, executed, reviewed, and controlled.
The spec does not replace the trail an auditor may request. It organizes the intent, criteria, and restrictions that give that trail meaning.
The seven-stage flow names each component by its immediate function: the project's CLAUDE.md, hooks, CI/CD, and gateway. In 2026, the market gained more precise terms for what each component does in the context of an AI agent in production: CLAUDE.md is a guide, hooks are sensors, and CI/CD is orchestration. The discipline that organizes these components into a coherent architecture has a name. It is what closes the series.
This is Part 5 of 6 in the series Generative AI in a Regulated Financial Institution. Part 4 — Architecture with an on-premises gateway, fail-closed behavior, and an audit trail is here. Part 6 closes with harness engineering.
Links and references#
Regulations, standards, and institutional documents#
CMN Resolution 5,274/2025 — Changes to cybersecurity requirements
Complementary Law 105/2001 — Confidentiality of financial institutions' operations
ISO/IEC 42001:2023 — Artificial intelligence management systems
NIST SP 800-218 — Secure Software Development Framework, version 1.1
AI application security#
Veracode — Insights from the 2025 GenAI Code Security Report
Cloud Security Alliance — Vibe Coding Security Debt: AI-Generated Vulnerabilities at Scale
Studies and research#
METR — Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
Spec-Driven Development: From Code to Contract in the Age of AI Coding Assistants — arXiv 2602.00180
Context Engineering for Multi-Agent LLM Code Assistants — arXiv 2508.08322
Tests as Prompt: A Test-Driven-Development Benchmark for LLM Code Generation — arXiv 2505.09027