Oztas, B., 2025. Enhancing Transaction Monitoring Controls to Detect Money Laundering Using Machine Learning. Doctoral Thesis (Doctoral). Bournemouth University.
Full text available as:
Preview |
PDF
OZTAS, Berkan_Ph.D._2024.pdf Available under License Creative Commons Attribution Non-commercial. 8MB |
Copyright to original material in this document is with the original owner(s). Access to this content through BURO is granted on condition that you use it only for research, scholarly or other non-commercial purposes. If you wish to use it for any other purposes, you must contact BU via BURO@bournemouth.ac.uk. Any third party copyright material in this document remains the property of its respective owner(s). BU grants no licence for further use of that third party material. |
Abstract
Money laundering poses a significant threat to global financial systems, with estimates sug- gesting that 2-5% of global Gross Domestic Product (GDP) is laundered annually. Despite the pivotal role of financial institutions in combating these illicit activities through anti-money laundering mechanisms, existing rule-based transaction monitoring systems are inefficient, most notably due to false positive rate exceeding 95%. These inefficiencies result in considerable op- erational costs and complicate the effective identification of genuine financial crimes. This research aimed to address the limitations inherent in traditional transaction monitoring approaches by investigating advanced machine learning techniques to enhance the proficiency of detection. A systematic literature review forms the foundation of the study, encompassing an analysis of current transaction monitoring methods. It evaluates data pre-processing tech- niques, dataset characteristics, feature selection processes, and the application of various ma- chine learning algorithms. This review highlights significant gaps in the application of advanced methods, such as deep learning and graph analysis, and underscores the lack of industry-wide collaboration along with the challenges posed by inconsistent and unavailable datasets. In response to these challenges, the study explores the perspectives of anti-money laundering specialists through semi-structured interviews. These discussions reveal current shortcomings in transaction monitoring and the potential of machine learning, addressing the disconnect between the industry and academicians. Innovative solutions utilising anomaly detection and graph analysis to identify complex local and global patterns of transactions are identified as crucial for enhancing rule-based systems. The requirements for a successful approach, include the need for explainability, incorporating a feedback loop, and enhanced risk identification capabilities, are also thoroughly analysed. Further, the thesis introduces SAML-D, a novel synthetic dataset created through agent- based and typology-based simulations designed to reflect the intricacies of money laundering schemes. The typologies incorporated within SAML-D were created based on insights gathered from specialists, existing datasets, and through comprehensive literature reviews. SAML-D in- cludes diverse elements like geographic variables and high-risk payment types, providing a more robust platform for testing Anti-Money Laundering (AML) systems than currently available datasets. Building on these insights, Tab-AML is developed. It employs a dual-masked transformer structure enhanced by a residual attention mechanism and a shared embedding approach. This model is evaluated using the SAML-D dataset against both conventional and deep learning models, such as TabTransformer and TabNet, demonstrating superior capabilities. Specifi- cally, Tab-AML achieved a 93.01% ROC-AUC score, significantly reducing the false positive rate by 17% while maintaining a high true positive rate of 98%. The importance of model selection tailored to the task and dataset was underscored by additional tests on a real trans- action monitoring dataset, where XGBoost was the best-performing model. Transformer-based models excelled at identifying suspicious behaviour in the initial monitoring stage with un- processed transaction data, while XGBoost performed best in the later monitoring stage with pre-processed, case-structured data. These comparisons also underlined the Tab-AML model’s adaptability and strengths, emphasising the benefits of the residual attention and shared em- bedding mechanisms where inter-transaction connections are critical. This research advances the field of anti-money laundering by demonstrating the effectiveness of transformer-based architectures for transaction monitoring. The novel SAML-D dataset provides a robust testing platform, while Tab-AML shows superior performance in detecting suspicious activities and reducing false positive rates. Comparative analysis with other models underscores the necessity of tailored model selection while demonstrating how impactful deep learning models can be. This thesis contributes to both academic and practical domains of financial security, advocating for more intelligent and adaptive systems to meet the complex demands of the modern financial landscape.
Item Type: | Thesis (Doctoral) |
---|---|
Additional Information: | If you feel that this work infringes your copyright please contact the BURO Manager. |
Uncontrolled Keywords: | Deep Learning; Machine Learning; Artificial Intelligence; Transformers; Anti-Money Laundering (AML); Transaction Monitoring |
Group: | Faculty of Science & Technology |
ID Code: | 41326 |
Deposited By: | Symplectic RT2 |
Deposited On: | 04 Sep 2025 14:15 |
Last Modified: | 04 Sep 2025 14:15 |
Downloads
Downloads per month over past year
Repository Staff Only - |