A payment can be legitimate according to its credentials and still be deliberately shaped to evade a fraud model. A credit decision can be generated by approved software and still depend on corrupted data. Neither failure requires an attacker to steal a model’s source code. This is why bank security teams need to examine not only who can access an AI system, but also how adversaries can influence what it learns and what it decides.

Adversarial machine learning is the deliberate manipulation of an AI system’s inputs, training process, model, or surrounding workflow to make it behave contrary to its intended purpose. For banks, the principal concerns are fraud evasion, contaminated training data, exposure of sensitive information, and manipulation of AI-enabled processes. The right response combines conventional security controls, model risk management, adversarial testing, and operational safeguards—not a single robustness product (Vassilev et al., 2025).

For bank CISOs: Start with customer-facing fraud models, credit decisioning, transaction monitoring, and external AI services. Establish who can influence model inputs and feedback, test attacks that would be feasible in the real banking workflow, and document the residual exposure and response plan. Ordinary accuracy tests cannot establish adversarial resilience.

What is adversarial machine learning in banking?

Adversarial machine learning (Adversarial ML) concerns deliberate attempts to undermine machine learning systems. In banking, an attacker may change transaction behavior to evade fraud alerts or influence the data used to retrain a model. Other attacks may reveal sensitive information through model responses or manipulate an AI assistant connected to bank workflows. NIST’s adversarial ML taxonomy groups attacks on predictive AI into evasion, poisoning, and privacy attacks, with additional misuse considerations for generative AI (Vassilev et al., 2025).

The acronym AML already means anti-money laundering in banking. This article uses Adversarial ML for the security discipline and AML compliance for anti-money laundering programs. The distinction matters when writing controls, incidents, and examination evidence.

Adversarial ML differs from an ordinary model error. A model that deteriorates because customer behavior changes may suffer concept drift; that is not, on its own, evidence of an attack. A model deliberately deceived by feasible changes to attacker-controlled inputs presents an adversarial risk. In practice, both can occur together, and defenders need evidence before attributing cause.

Four attack paths bank security teams should prioritize

These are distinct failure modes. For example, a phishing message written with a generative model is AI-enabled fraud, but is not necessarily an attack against the bank’s machine learning model. Likewise, prompt injection is an important issue for AI assistants, but should not be presented as the same technique as manipulating tabular fraud-detection inputs (Vassilev et al., 2025; MITRE, n.d.).

Fraud detection: the most immediate exposure

Transaction-monitoring models use patterns such as payment amounts, merchant categories, device signals, velocity, and prior behavior to support fraud decisions. An attacker rarely controls all these features. A realistic assessment asks which fields the attacker can change directly, which outcomes can be influenced indirectly through behavior, and which are controlled by the bank or payment network.

Fursov et al. (2021) evaluated black-box attacks on deep models processing financial transaction sequences and reported that limited additional transactions could undermine model predictions in their experimental setting. More recently, Lunghi et al. (2026) published a reinforcement-learning attack, Fraud-RLA, against credit-card fraud detection, using a threat model designed around constrained attacker knowledge and behavior. These are peer-reviewed demonstrations of potential vulnerabilities, not public evidence that particular banks were breached using those techniques.

A bank should therefore test not just whether a classifier recognizes fraudulent examples in a static dataset, but whether its complete decision process remains effective when an adversary adapts transaction behavior over time. This includes rules, model scores, step-up authentication, manual review, and the feedback used to update future models.

Credit scoring: integrity and explanation are both at stake

Credit models may be affected by manipulated application data, compromised upstream information, identity fraud, or poisoning of datasets used for development and monitoring. Some of these attacks are better characterized as document or identity fraud that influences a model rather than direct compromise of its algorithm. The threat model should state which.

A security test must also protect lawful applicants. A bank could improve resistance to certain manipulations while increasing false declines or creating unreliable explanations. This is a governance problem as well as a technical one: the institution must evaluate model performance, customer outcomes, data integrity, and the quality of reasons given for consequential decisions.

Anti-money laundering compliance: do not confuse model risk with reporting obligations

Banks may use machine learning to prioritize alerts or support transaction monitoring under their AML compliance programs. Attackers may try to shape activity to reduce alerting, while flawed labels or fragmented entity data can degrade monitoring. A missed alert may involve an adversarial attack, ordinary model limitations, inadequate rules, or weaknesses elsewhere in the program.

Treat model scores as one element of a broader financial-crime control process. Define escalation, sampling, analyst oversight, and review of suspicious patterns that fall outside expected model behavior. The revised 2026 U.S. model risk guidance superseded not only SR 11-7 but also the prior interagency model-risk statement for Bank Secrecy Act/AML systems (Federal Reserve, OCC, & FDIC, 2026).

What does the evidence actually show?

Established in research: attacks can undermine financial transaction classifiers under specified experimental conditions, including black-box or restricted-information settings (Fursov et al., 2021; Lunghi et al., 2026).

Established in official risk reporting: the European Banking Authority identifies AI adoption, third-party dependence, cyber threats, and fraud as material operational-risk concerns for banks. Its June 2026 report calls for risk-sensitive AI governance and security testing; it does not establish a sector-wide rate of successful adversarial ML attacks (EBA, 2026).

Not established by those sources: that adversarial ML caused a documented breach at JPMorgan Chase, HSBC, or Deutsche Bank; that most bank fraud models are already compromised; or that a particular library provides comprehensive protection. None of these claims should appear in a defensible risk assessment without case-specific evidence.

The broader review by Pelekis et al. (2025) surveys methods, defenses, and tooling across several critical sectors, but does not measure the frequency of attacks against banks. Research results also depend on dataset quality, target model, attack budget, and the realism of permitted changes. A test that modifies a bank-controlled field which a fraudster cannot influence may demonstrate a mathematical weakness without establishing a feasible operational exploit. Conversely, perfect accuracy on a historical holdout set says little about an adaptive attacker.

How should a bank threat-model its AI systems?

Use one concrete model-to-business-workflow record for every material AI use. It should identify the model’s business owner, risk owner, version, provider, downstream decisions, affected customers, data sources, retraining route, and fallback procedure.

Then examine five trust boundaries:

1. Data collection: What can a customer, merchant, counterparty, broker, or external provider submit or influence?
2. Feature processing: Which transformations, feature stores, and third-party feeds can change what the model sees?
3. Model development: Who can change training sets, labels, code, hyperparameters, or deployed artifacts?
4. Inference and decisions: Who can query the model, observe scores or decisions, and alter downstream actions?
5. Feedback and recovery: Which analyst outcomes, chargebacks, investigations, and customer corrections return to training—and how would the bank detect and reverse contamination?

Map plausible adversary behaviors to the NIST taxonomy and the MITRE ATLAS knowledge base, then document attacker access, objective, affected assets, feasible manipulations, detection points, and business impact. This is more useful than creating a long catalog of attacks without showing whether they apply to the bank’s architecture.

A practical adversarial testing program for bank CISOs

1. Set a baseline that reflects banking outcomes

Record fraud capture at a specified false-positive or customer-decline rate, investigation workload, monetary losses where measurable, alert latency, and performance across relevant customer or transaction segments. For credit decisions, include stability and appropriate outcome measures, reason-code validity, and customer-impact review. Overall accuracy can be deceptive when harmful events are rare.

2. Generate feasible adversarial scenarios

Test realistic changes to attacker-influenced inputs, sequences, and behavioral patterns. Separate white-box, black-box, and limited-query evaluations. Preserve immutable test samples and explicitly reject invalid changes to protected or bank-controlled fields. Use controlled or synthetic data where possible, and never conduct experiments against live payment rails without formal authorization and safeguards.

3. Test the end-to-end system, not only the classifier

A model may output a vulnerable score but be protected by additional rules, limits, review, or transaction holds. The reverse is also possible: a model may look robust while a compromised enrichment feed, threshold configuration, or analyst queue undermines the final decision. Run tests through the same feature transformations and decision logic used in production, within an approved test environment.

4. Evaluate several defenses and their costs

Adversarial training may improve resistance to attacks represented in training, but it is not a universal defense. Input validation, data and label provenance, secure pipelines, query controls, dual-control for material model changes, independent checks, and business-level safeguards often address different parts of the risk. Measure any increase in false positives, computational load, latency, and customer friction before approving a defense.

For sensitive data, techniques such as differential privacy and federated learning may be relevant to a separately defined privacy or data-sharing problem. They do not, by themselves, establish resistance to evasion or poisoning. Differential privacy also involves utility and implementation trade-offs (Papernot & Thakurta, 2021).

5. Document evidence and make an explicit go/no-go decision

The validation record should include model and dataset versions, attack assumptions, permitted feature changes, test code, observed outcomes, limitations, compensating controls, independent challenge, approval, and the planned retest trigger. A red-team report without an accountable decision and follow-up is incomplete assurance.

Tools: Adversarial Robustness Toolbox (ART) supports attacks and defenses across tabular data and several common ML frameworks. CleverHans supplies reference adversarial-example implementations. Both are useful for appropriate tests; neither replaces banking-specific constraints, manual analysis, or validation of the complete decision workflow. An image-classification benchmark should not be treated as direct evidence that a payments model is secure.

What should a CISO measure after deployment?

A useful AI security dashboard connects model risk to customer harm, fraud losses, investigation capacity, and change control. There is no universal pass/fail threshold; banks should set risk-based limits for each model and use case.

Track exceptions until closure, preserve prior model versions, and require reassessment after material data changes, supplier changes, model updates, new attack intelligence, or major fraud-pattern changes.

Explainable AI cannot substitute for adversarial security

Explainable AI (XAI) helps analysts and affected parties understand how a model’s outputs relate to inputs. It can support model validation, investigations, and credit-decision communications. But a plausible explanation is not evidence that a model is secure, accurate, or causally faithful.

Slack et al. (2020) demonstrated that post-hoc explanation methods including LIME and SHAP can themselves be manipulated under particular conditions. A bank should therefore test explanation stability, the relationship between explanations and actual model behavior, and whether reason codes accurately describe the decision. Explainability, robustness, privacy, and fairness require separate assessments.

For U.S. lending, the Equal Credit Opportunity Act and Regulation B retain adverse-action notice requirements, including specific principal reasons where applicable. The CFPB withdrew its 2022 circular on complex algorithms in May 2025, but the withdrawal did not repeal the underlying notice duties (CFPB, 2025; 12 C.F.R. § 1002.9). Avoid citing that withdrawn circular as current supervisory guidance.

AI model security and banking regulation in 2026

Different regimes address different risks. They should be mapped to the bank’s activities and legal entities rather than combined into a fictitious universal AI compliance requirement.

The April 2026 U.S. interagency guidance concerns the covered model-risk discipline; the OCC states that generative and agentic AI models are outside its scope. A bank may still need to secure those systems under other applicable obligations and its own security framework. Do not describe the 2026 model guidance as a complete governance standard for banking AI (OCC, 2026).

The EBA’s June 2026 risk assessment specifically calls for adequate governance, enhanced data and cyber security, and regular security testing in bank AI adoption. In July 2026, the three European Supervisory Authorities also called for risk-based governance to address ICT risks from frontier AI models, including third-party dependencies (EBA, 2026; EBA, EIOPA, & ESMA, 2026).

Third-party AI: what to require before procurement or renewal

Many banks cannot inspect a vendor’s full training corpus or weights. That does not make vendor assurance optional, nor does it make a supplier’s generic certification conclusive. Ask for the information necessary to evaluate the specific service, version, data flows, and operating environment your institution will use.

At minimum, require an intended-use statement, data provenance, and retraining arrangements. Obtain security-test summaries with limitations, incident and investigation commitments, model-change notices, and access and logging details. Examine hosting and subprocessor dependencies and contract for continuity, rollback, and exit. Where source-level testing is restricted, arrange agreed black-box tests and compensate with independent monitoring and business-level controls.

A vendor may provide strong technical assurance while the bank’s own integration introduces a new weakness. Assess both sides of that boundary. The revised U.S. guidance explicitly addresses vendor and other third-party products, while DORA adds ICT third-party obligations for in-scope EU institutions (Federal Reserve et al., 2026; EBA, 2026). See the InfoSecured.ai analysis of third-party AI accountability under ISO/IEC 42001 for a companion treatment of shared supplier responsibilities.

An incident scenario: suspected poisoning of fraud labels

Illustrative scenario—not a reported banking incident. A bank notices that chargeback outcomes from one upstream feed are arriving with inconsistent labels. A newly retrained fraud model shows better historical accuracy but a rising rate of subsequently confirmed fraud in a particular channel. There is no evidence yet of an adversary.

The incident team should preserve the training dataset and label history, model artifact, approval trail, and relevant feed logs. Data engineering checks whether the labels changed during ingestion. Model risk management reproduces the validation and compares the new model with the prior approved version. Fraud operations reviews real customer losses and missed alerts. Security investigates whether data or access controls were breached. The bank may suspend retraining, revert to a validated model, apply compensating rules, or increase review, based on its documented decision criteria.

The finding should distinguish confirmed malicious poisoning, data-quality failure, and unresolved anomaly. Reporting and customer-notification decisions depend on facts and applicable law, not simply on an unusual model metric.

A 90-day CISO roadmap for adversarial ML assurance

This is a proposed practitioner roadmap, not a regulatory checklist or a bank-tested certification method. Its objective is to establish ownership and defensible evidence for two or three selected high-consequence AI workflows in 90 days—not to declare the entire bank’s AI estate secure. A CISO should sponsor the work jointly with the business owners, fraud/financial-crime teams, and independent model risk management. Institutions without the necessary data access, validation capability, or vendor cooperation may need more time (Federal Reserve et al., 2026; NIST, 2023; EBA, 2026).

Days 1–30: Scope and establish control

Accountable teams: The CISO’s delegate, business and model owners, model risk management, data engineering, and vendor risk.

Work: Identify material model-connected decisions, including critical external services. Rank them by financial and customer consequences, adversary influence, and fallback readiness. Select two or three priority workflows. Map the trust boundaries for input data, labels, models, downstream decisions, and suppliers.

Decision gate: Obtain a signed priority inventory with named owners, baseline metrics, current model and data versions, attack hypotheses, and documented rollback or manual fallback. Record missing evidence as an open issue, never as a pass.

Days 31–60: Challenge the controls

Accountable teams: Independent model validation, fraud or credit operations, security testing, and data engineering.

Work: In an authorized test environment, assess feasible evasion and data or label manipulation. Compare clean and adversarial outcomes at a defined review or decline rate. Check pipeline integrity, logging, supplier evidence, customer impact, and downstream decision controls.

Decision gate: Produce reproducible test records: attacker knowledge, allowed changes, test samples, model versions, attack success, fraud or credit effects, false positives, limitations, and independent challenge. Escalate serious exposures when discovered rather than waiting until day 60.

Days 61–90: Decide and operationalize

Accountable teams: The business risk owner and designated approval forum, supported by the CISO, independent validation, compliance, and operations.

Work: Remediate or constrain serious weaknesses. Set deadlines for open findings, implement proportionate monitoring and retest triggers, and rehearse rollback, manual handling, and vendor escalation.

Decision gate: Record signed residual-risk decisions, evidence of controls, monitoring ownership, recovery-test results, and open exceptions with deadlines. Present the executive risk report. Deployment and expansion decisions remain subject to the bank’s own policies and authorities.

The day-90 output is an assurance dossier, not a blanket approval. For each scoped workflow, record its intended use, owners, data and model versions, attack assumptions, and authorized test procedures. Include normal and adversarial results, customer and operational consequences, independent validation, vendor dependencies, unresolved limitations, residual-risk decisions, and retest triggers. Preserve the underlying artifacts rather than relying on a slide-deck summary.

What should stop expansion? Escalate credible attacks that may cause financial or customer harm without effective safeguards. Also escalate when testing cannot be reproduced, critical data or label provenance is unknown, or recovery has not been tested. Whether to suspend, restrict, or accept exposure is context- and policy-dependent; InfoSecured.ai cannot make that institution-specific decision.

The bank should set its own thresholds for fraud capture, loss, false-positive burden, test coverage, recovery performance, and risk acceptance. No universal adversarial-security pass score exists. The InfoSecured.ai evidence library offers companion documentation resources, and its ISO/IEC 42001 guide provides management-system context. Neither substitutes for bank-led validation.

Frequently asked questions

Can attackers fool bank fraud-detection AI without accessing its code?

Yes. Peer-reviewed studies demonstrate black-box and restricted-information adversarial attacks on financial transaction models. Whether an attack works in a particular bank depends on attacker control, model design, downstream controls, and monitoring (Fursov et al., 2021; Lunghi et al., 2026).

Is adversarial machine learning the same as anti-money laundering?

No. Adversarial ML describes attacks against machine learning systems. Anti-money laundering describes legal and operational measures for detecting and preventing money laundering. Banks may use machine learning within AML compliance systems, so both terms can appear in the same risk assessment.

Does the EU AI Act classify fraud-detection systems as high-risk?

Not merely because they detect financial fraud. Annex III’s creditworthiness and credit-scoring category expressly excludes AI used for financial fraud detection. A system may still fall within another applicable category or legal obligation; classification depends on actual purpose and use (Regulation (EU) 2024/1689).

Does passing an AI red-team test prove a banking model is secure?

No. A test supports a conclusion about defined models, attack assumptions, data, and time periods. New attacker behavior, changed data, or modified integrations can invalidate that conclusion. Banks need ongoing monitoring, retesting, and operational safeguards.

Which tools can a bank use to begin adversarial ML testing?

ART is a useful starting point for compatible predictive models, including tabular classifiers. CleverHans provides adversarial-example implementations. Tool selection should follow the model architecture and test objective, and results should be checked against the banking workflow rather than accepted as a universal security score.

What can a bank CISO realistically accomplish in 90 days?

A 90-day effort can establish a risk-ranked inventory, challenge two or three high-consequence workflows, and produce accountable risk decisions with monitoring and recovery steps. It cannot guarantee adversarial resilience across every bank model.

Does the revised 2026 U.S. model risk guidance require adversarial red teaming?

No. The April 2026 interagency guidance describes risk-based model management principles, not an enforceable adversarial-testing prescription. The OCC also states that generative and agentic AI models fall outside this guidance (Federal Reserve et al., 2026; OCC, 2026).

Final Thoughts

Banks already know how to demand evidence before trusting a financial model. Adversarial ML adds a necessary question: What happens when someone deliberately tries to make the system reach the wrong conclusion?

The answer cannot come from a high accuracy score, an attractive explanation, or a supplier certificate alone. It requires realistic attack assumptions, independent testing, controlled data pipelines, measurable customer and financial outcomes, and a documented ability to detect and recover from failure. For bank CISOs, that is the practical meaning of defensible AI assurance.

References

View all 19 sources

12 C.F.R. § 1002.9. (2026). Notifications; Equal Credit Opportunity (Regulation B). Regulation B § 1002.9

Consumer Financial Protection Bureau. (2025). Withdrawn guidance [including Circular 2022-03 withdrawn May 12, 2025]. CFPB withdrawn-guidance register

European Banking Authority. (2026, June). Risk assessment report—June 2026. EBA risk assessment

European Banking Authority, European Insurance and Occupational Pensions Authority, & European Securities and Markets Authority. (2026, July 31). EBA, EIOPA and ESMA call for enhanced governance and consistent supervision to mitigate ICT risks from frontier AI models in the EU financial sector. Joint ESA statement

European Commission. (2026). AI Act: Regulatory framework for artificial intelligence. European Commission AI Act overview

Federal Reserve, Office of the Comptroller of the Currency, & Federal Deposit Insurance Corporation. (2026, April 17). Revised guidance on model risk management (SR 26-2). Federal Reserve SR 26-2

Fursov, I., Morozov, M., Kaploukhaya, N., Kovtun, E., Rivera-Castro, R., Gusev, G., Babaev, D., Kireev, I., Zaytsev, A., & Burnaev, E. (2021). Adversarial attacks on deep models for financial transaction records. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (pp. 2868–2878). doi:10.1145/3447548.3467145

International Organization for Standardization & International Electrotechnical Commission. (2023). Information technology—Artificial intelligence—Management system (ISO/IEC 42001:2023). ISO/IEC 42001

Lunghi, D., Molinghen, Y., Simitsis, A., Lenaerts, T., & Bontempi, G. (2026). Fraud-RLA: A reinforcement learning adversarial attack against credit card fraud detection. IEEE Transactions on Dependable and Secure Computing, 23(3), 7468–7481. doi:10.1109/TDSC.2026.3673847

MITRE. (n.d.). MITRE ATLAS. MITRE ATLAS

National Institute of Standards and Technology. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI RMF 1.0

Office of the Comptroller of the Currency. (2026, April 17). OCC issues updated model risk management guidance. OCC release: scope and exclusions

Papernot, N., & Thakurta, A. (2021, December 21). How to deploy machine learning with differential privacy. National Institute of Standards and Technology. NIST guidance

Pelekis, S., Koutroubas, T., Blika, A., Berdelis, A., Karakolis, E., Ntanos, C., Spiliotis, E., & Askounis, D. (2025). Adversarial machine learning: A review of methods, tools, and critical industry sectors. Artificial Intelligence Review, 58, Article 226. doi:10.1007/s10462-025-11147-4

Prudential Regulation Authority. (2026, April 23). Model risk management principles for banks (Supervisory Statement SS1/23; April 2026 edition). PRA SS1/23

Regulation (EU) 2022/2554 of the European Parliament and of the Council on digital operational resilience for the financial sector (DORA). (2022). Official Journal of the European Union. DORA official legal text

Regulation (EU) 2024/1689 of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). (2024). Official Journal of the European Union (consolidated 27 July 2026). Consolidated AI Act, 27 July 2026

Slack, D., Hilgard, S., Jia, E., Singh, S., & Lakkaraju, H. (2020). Fooling LIME and SHAP: Adversarial attacks on post hoc explanation methods. In Proceedings of the 2020 AAAI/ACM Conference on AI, Ethics, and Society (pp. 180–186). doi:10.1145/3375627.3375830

Vassilev, A., Oprea, A., Fordyce, A., Anderson, H., Davies, X., & Hamin, M. (2025). Adversarial machine learning: A taxonomy and terminology of attacks and mitigations (NIST AI 100-2e2025). National Institute of Standards and Technology. NIST AI 100-2e2025