Cloud-Native Data Engineering at Scale for AI-Enabled Financial Risk, Compliance, and Accounting Operations
DOI:
https://doi.org/10.64137/31078699/IJETET-V2I3P102Keywords:
Cloud-Native Data Engineering, Financial Risk Management, Regulatory Compliance Frameworks, Enterprise Data Pipelines, Risk Taxonomy Models, Data Governance Gates, Audit-Ready Architectures, Quality-Driven Data Systems, AI-Driven Financial Pipelines, Scalable Data WorkflowsAbstract
The financial services industry has gradually embraced AI to automate business processes and augment decision-making. However, cloud-native data engineering, a critical prerequisite to AI, is still nascent, especially in mission-critical areas such as accounting and regulatory compliance, where years of regulatory scrutiny have made CEOs and Boards wary of reckless data quality and compliance risks. Cloud-native data pipelines for risk management are typically built, operated, and maintained independently, with quality and governance applied in hindsight. An end-to-end enterprise-grade cloud-native data-engineering framework is presented, specifically addressing the requirements of a regulated financial institution. The framework is driven by a risk taxonomy and the regulatory compliance library, with cloud-native concepts applied at each layer. Governance gates ensure quality, testing, validation, traceability, and audit-readiness without stifling development velocity. By establishing an open, extensible architecture and implementing core components and data workflows with a stringent quality-first mindset, the framework serves as a solid foundation for accelerated development and live operation of use-case-specific, AI-driven data pipelines.
References
[1] S. Russell, and P. Norvig, Artificial intelligence: A modern approach, 5th ed., Pearson, 2026.
[2] M. V. K. Kumar, S. H. Kolla, V. Pamisetty, L. Pandiri, U. S. Yandamuri, and D. Valiki, "Enterprise-Scale Generative AI Agents for Secure and Governed Automation in Insurance and Public Financial Management," 2026 International Conference on Computational Robotics, Testing and Engineering Evaluation (ICCRTEE), pp. 1–6, May 2026, doi: 10.1109/iccrtee68719.2026.11566439.
[3] R. Loganathan, K. Amistapuram, and A. R. Aitha, "Letter to the Editor re Annual Updates of the European Association of Urology – European Society for Pediatric Urology (EAU-ESPU) Pediatric Urology Guidelines: Are Large-Language Models (LLMs) Better Than the Usual Structured Methodology?," Journal of Pediatric Urology, p. 106057, June 2026, doi: 10.1016/j.jpurol.2026.106057.
[4] K. C. Nagabhyru, S. Singireddy, A. L. Gadi, G. K. Sheelam, and D. Kapila, "Toward Secure and Usable Communication-Centric Authorization Models for Smart Homes," Lecture Notes in Electrical Engineering, pp. 355–366, 2026, doi: 10.1007/978-3-032-20235-2_32.
[5] S. S. Kumar, R. S. Garapati, A. R. Segireddy, S. Kalisetty, R. Inala, and K. C. Nagabhyru, "Hybrid Deep Neural Network–DevOps Pipeline Optimization for Risk Prediction in Cloud-Native Workers' Compensation Platforms," 2026 IEEE International Conference for Convergence in Computing Technology (I3CTCON), pp. 1–6, Mar. 2026, doi: 10.1109/i3ctcon68242.2026.11507219.
[6] T. Nguyen, M. Lopez, and J. Carter, “Autonomous compliance validation using explainable AI pipelines in cloud banking systems,” IEEE Transactions on Cloud Computing, vol. 14, no. 2, pp. 611–628, 2026.
[7] N. B. Hiremath, S. K. Kolla, R. Sunkara, S. Lal. G, and K. Sireesha, “Adaptive and Intelligent Secure Multimedia Transmission for Dynamic Communication Systems,” 2026 2nd International Conference on Intelligent Systems and Computational Networks (ICISCN), pp. 1–8, Apr. 2026, doi: 10.1109/iciscn67954.2026.11566143.
[8] M. Recharla, R. S. Garapati, V. D. V. K. Bandi, U. S. Yandamuri, and B. M. Mangalampalli, “Generative AI-Enhanced Data Engineering Pipelines for Predictive Biomarker Discovery in Alzheimer’s and Kidney Disease,” 2026 IEEE International Conference on AI Engineering and Innovations (AIEI), pp. 1–5, Mar. 2026, doi: 10.1109/aiei69164.2026.11496858.
[9] B. M. Mangalampalli, R. K. Peddi, S. K. Kolla, V. A. R. Reddy, N. Mangala, and A. Seenu, "Explainable Clinical Graph Intelligence Framework for Longitudinal Risk Modeling and Care Pathway Optimization," 2026 Third International Conference on Innovations in Cybersecurity and Data Science (ICICDS), pp. 1–6, June 2026, doi: 10.1109/icicds70526.2026.11604804.
[10] P. Sharma, and Y. Chen, “Scalable lakehouse architectures for AI-driven enterprise financial analytics,” Future Generation Computer Systems, vol. 172, pp. 201–219, 2026.
[11] M. Krishnan et al., “Engineering Intelligent Cloud-Native Data Ecosystems for Predictive Decision-Making in Industry,” Journal of European Economic History, vol. 7, no. 2, pp. 68–88, June 2026, doi: 10.61336/jeeh/26-2-7.
[12] G. Padma, V. A. R. Reddy, S. Devi. K. A, B. Buvaneswari, and K. S. S. Kumar, “Privacy-Preserved Face Recognition Biometric Authentication Using FaceNet and Zero-Knowledge Proofs for Secure, Access Control on Decentralized Blockchain Networks,” 2026 2nd International Conference on Intelligent Systems and Computational Networks (ICISCN), pp. 1–8, Apr. 2026, doi: 10.1109/iciscn67954.2026.11566528.
[13] P. R. Sudha Rani, K. Amistapuram, V. Pamisetty, S. Singireddy, D. N. Kummari, and G. K. Sheelam, "Hybrid Knowledge Graph–Deep Learning Framework for Automated Exception Handling and Investigation in Complex Insurance Claims," 2025 IEEE 3rd Global Conference on Wireless Computing and Networking (GCWCN), pp. 1–6, Nov. 2025, doi: 10.1109/gcwcn66157.2025.11448301.
[14] R. S. Garapati, S. Paleti, R. Meda, K. C. Nagabhyru, and B. S. Deepa Priya, "Physical-Unclonable-Function-Based Secure and Anonymous User Authentication for Smart Homes," Lecture Notes in Electrical Engineering, pp. 367–378, 2026, doi: 10.1007/978-3-032-20235-2_33.
[15] B. D. Bhavani, Sowmya. S. R, R. Loganathan, S. Nagaraj, and Vasukidevi. G, “Evolutionary Gravitational Neocognitron Neural Network, Snow Leopard Optimization and Deep Graph Reinforcement Learning for Routing Protocol in WSN,” 2026 2nd International Conference on Intelligent Systems and Computational Networks (ICISCN), pp. 1–8, Apr. 2026, doi: 10.1109/iciscn67954.2026.11566191.
[16] R. S. Garapati, S. Paleti, R. Meda, K. C. Nagabhyru, and B. S. Deepa Priya, "Physical-Unclonable-Function-Based Secure and Anonymous User Authentication for Smart Homes," Lecture Notes in Electrical Engineering, pp. 367–378, 2026, doi: 10.1007/978-3-032-20235-2_33.
[17] M. Rahman, J. Zhao, and X. Li, “AI-powered regulatory reporting and governance automation in financial institutions,” Computers & Security, vol. 147, 2026.
[18] D. Dawadi, U. S. Yandamuri, S. K. V, J. L. A. Stalin, and R. Naveenkumar, "Privacy-Aware Edge-Based Intelligent Video Analytics for Scalable Crowd Management in Smart Cities," 2026 3rd International Conference on Integrated Intelligence and Communication Systems (ICIICS), pp. 1–7, June 2026, doi: 10.1109/iciics67880.2026.11483444.
[19] J. Kim, R. Patel, and L. Brown, “Continuous auditability and provenance tracking in cloud-native accounting systems,” Information Systems, vol. 132, 2026.
[20] M. Krishnan, A. R. Aitha, K. Amistapuram, B. P. Nandan, P. K. Kaulwar, and J. Singireddy, "Human-in-the-Loop Hybrid Neuro-Symbolic AI Model for Reliable Data Engineering in High-Stakes Industrial Systems," 2025 IEEE 3rd Global Conference on Wireless Computing and Networking (GCWCN), pp. 1–7, Nov. 2025, doi: 10.1109/gcwcn66157.2025.11448516.
[21] R. Inala, “Cloud-Native AI and MDM Framework for Next-Generation Insurance and Retirement Data Products,” International Journal of Engineering & Extended Technologies Research, vol. 8, no. 03, May 2026, doi: 10.15662/ijeetr.2026.0803006.
[22] S. Raschka, Y. Liu, V. Mirjalili, and D. Dzhulgakov, “Large language models and generative AI for data engineering and analytics,” Information Systems, vol. 125, 2024.
[23] S. Manikandan, G. P. Singh, V. R. R. Gottimukkala, P. S. L. N. Davuluri, R. Sadagopan, and T. V. Kumar, “Human-in-the-Loop Automation Patterns for Financial Services Using Camunda + Modern Java UIs,” 2026 IEEE International Conference for Convergence in Computing Technology (I3CTCON), pp. 1–6, Mar. 2026, doi: 10.1109/i3ctcon68242.2026.11507795.
[24] T. Chen, H. Xu, and Y. Zhang, “AIOps-driven anomaly detection and root cause analysis for cloud-native enterprise platforms,” Future Generation Computer Systems, vol. 158, pp. 310–324, 2024.
[25] M. Moor et al., "Foundation models for generalist medical artificial intelligence," Nature, vol. 616, no. 7956, pp. 259–265, Apr. 2023, doi: 10.1038/s41586-023-05881-4.
[26] Rohit Gorle, “Post-Quantum Identity and Access Management for Enterprise Cloud Security,” International Journal of Special Education, vol. 41, no. 14s, pp. 932–943, 2026. Retrieved from https://internationalsped.com/index.php/ijse/article/view/4643
[27] L. Wang et al., “A survey on large language model based autonomous agents,” Frontiers of Computer Science, vol. 18, no. 6, Mar. 2024, doi: 10.1007/s11704-024-40231-1.
[28] Z. Xi et al., "The Rise and Potential of Large Language Model-Based Agents: A Survey," Sept. 2023. doi: 10.48550/arXiv.2309.07864.
[29] Supriya. R K, N. Mangala, R. Priyanka, R. A. Mohammed Sait, and P. P. Selvam, "Privacy Preserving Analytics for Intelligent in Internet of Things System in Health Care Using Graph Neural Network with Latent Graph Inference Technique," 2026 2nd International Conference on Intelligent Systems and Computational Networks (ICISCN), pp. 1–6, Apr. 2026, doi: 10.1109/iciscn67954.2026.11566407.
[30] S. Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models," Mar. 2023. doi: 10.48550/arXiv.2210.03629.
[31] P. Nayak S, R. Loganathan, Velmurugan. R, S. R. K. Sarma A, and V. Jayaraj, “Scale-Aware Dilated Lightweight Convolutional Network Improving Solar Panel Defect Classification through Efficient Electroluminescent Image Analysis Techniques,” 2026 2nd International Conference on Intelligent Systems and Computational Networks (ICISCN), pp. 1–7, Apr. 2026, doi: 10.1109/iciscn67954.2026.11566218.
[32] K. Naveen, R. Lingam, G. R. Kunduru, N. Mangala, N. N. Reddy, and P. Balaji, “Intelligent Loan Approval System: Machine Learning for Home and Education Loan Eligibility,” 2026 Contemporary Computing Innovations Conference (CCIC), pp. 1–6, Feb. 2026, doi: 10.1109/ccic68129.2026.11486013.
[33] S. H. Kolla and R. Mattaparthi, “Hybrid Gen AI Systems: Integrating Small LMs with Large Language Models for Cost-Efficient Enterprise Automation and Decision Intelligence,” International Journal of Research Publications in Engineering, Technology and Management, vol. 08, no. 06, Dec. 2025, doi: 10.15662/ijrpetm.2025.0806038.
[34] M. S. R. L. Reddy, T. Sunitha, K. Kanchana, A. R. Nagubandi, A. R. Segireddy, and S. N. Bhavanam, “AI-Enhanced Blockchain Consensus Mechanisms for Secure Transaction Validation,” 2026 International Conference on Emerging Research in Smart Electronics and Machine Informatics (ECMI), pp. 1–11, Apr. 2026, doi: 10.1109/ecmi68341.2026.11602724.
[35] N. Shinn, B. Labash, and A. Gopinath, "Reflexion: Language Agents with Verbal Reinforcement Learning," arXiv (Cornell University), vol. 1, no. 1, Mar. 2023, doi: 10.48550/arxiv.2303.11366.
[36] W. X. Zhao et al., “A Survey of Large Language Models,” arXiv (Cornell University), Mar. 2023, doi: 10.48550/arxiv.2303.18223.
[37] D. Kaur, S. Uslu, K. J. Rittichier, and A. Durresi, "Trustworthy Artificial Intelligence: A Review," ACM Computing Surveys, vol. 55, no. 2, pp. 1–38, Mar. 2023, doi: 10.1145/3491209.
[38] B. M. Mangalampalli, R. K. Peddi, S. K. Kolla, V. A. R. Reddy, N. Mangala, and A. Seenu, "Explainable Clinical Graph Intelligence Framework for Longitudinal Risk Modeling and Care Pathway Optimization," 2026 Third International Conference on Innovations in Cybersecurity and Data Science (ICICDS), pp. 1–6, June 2026, doi: 10.1109/icicds70526.2026.11604804.
[39] R. Kuppuswami, U. S. Yandamuri, M. Bhatt, M. Patel, and Nagendran. R, "A Cognitive Clustering Framework for Wireless Sensor Networks Using Quantum Machine Learning," 2026 3rd International Conference on Integrated Intelligence and Communication Systems (ICIICS), pp. 1–6, June 2026, doi: 10.1109/iciics67880.2026.11483591.
[40] R. S. Garapati, A. R. Aitha, U. S. Yandamuri, V. R. R. Gottimukkala, A. R. Nagubandi, and S. H. Kolla, “Cloud-Native Orchestration of Multi-Counterparty Derivatives and Collateral in Manufacturing Enterprises via AI-Assisted Financial Audit Engines,” 2026 IEEE International Conference on AI Engineering and Innovations (AIEI), pp. 1–6, Mar. 2026, doi: 10.1109/aiei69164.2026.11497364.
[41] J. Mökander, J. Schuett, H. R. Kirk, and L. Floridi, "Auditing large language models: a three-layered approach," May 2023, doi: 10.1007/s43681-023-00289-2.
[42] Tarun Vakkalagadda, and Vijayanandh Rajamanickam, “Agentic AI Architectures for Next-Generation Investment Advisory and Portfolio Intelligence,” International Journal of Computer Information Systems and Industrial Management Applications, vol. 18, no. 9s, pp. 1206–1219, 2026. Doi: https://doi.org/10.70917/ijcisim-2026-3548
[43] R. Paramasivam, S. S. Reddy, V. A. R. Reddy, K. Sandra, and M. V. S. Narayana, “JellyFish Search Optimization with Arctic Puffin Optimization Algorithm for Efficient Routing Based on the Software Defined Communication Network,” 2026 2nd International Conference on Intelligent Systems and Computational Networks (ICISCN), pp. 1–6, Apr. 2026, doi: 10.1109/iciscn67954.2026.11566144.
[44] S. Lapologang and S. Zhao, "The impact of environmental policy mechanisms on green innovation performance: the roles of environmental disclosure and political ties," Technology in Society, vol. 75, p. 102332, Nov. 2023, doi: 10.1016/j.techsoc.2023.102332.
[45] S. J. Sheppard, V. R. R. Gottimukkala, S. K. Rongali, K. Kulkarni, G. K. Sheelam, and A. L. Gadi, "Repairability in the Circular Economy: Extending Product Lifecycles for Environmental and Economic Gains," Circular Economy and Sustainability, pp. 167–182, 2026, doi: 10.1007/978-3-032-24891-6_7.
[46] B. Bedi, U. S. Yandamuri, D. N. Kummari, A. R. Nagubandi, K. Amistapuram, and A. D, “AI-Driven Sentiment and Behavior Analysis for Sustainable Business Growth,” 2026 International Conference on Emerging Research in Smart Electronics and Machine Informatics (ECMI), pp. 1–11, Apr. 2026, doi: 10.1109/ecmi68341.2026.11603189.
[47] "Natural stone test methods - Petrographic examination," doi: 10.3403/30342862.
[48] K. Jyoshna, R. K. Peddi, P. Nayak. S. Dhanamalar, M, and S. Punitha, "Spatio-Temporal Graph Neural Network with Global Spatio-Temporal Network for the Traffic Flow Prediction," 2026 2nd International Conference on Intelligent Systems and Computational Networks (ICISCN), pp. 1–6, Apr. 2026, doi: 10.1109/iciscn67954.2026.11566462.
[49] S. Raj, P. Prashanthi, S. K. Kolla, S. Bandaru, N. Bhardwaj, and S. P, “Lightweight Cryptographic Schemes for Resource-Constrained IoT and Healthcare Devices,” 2026 International Conference on Multidisciplinary Innovations For Smart & Sustainable Future (MISSF), pp. 1–6, Apr. 2026, doi: 10.1109/missf68264.2026.11521999.
[50] Cloud Security Alliance, “Security Guidance for Critical Areas of Focus in Cloud Computing,” July 2017. [Online]. Available: https://cloudsecurityalliance.org/artifacts/security-guidance-v4
[51] P S L Narasimharao Davuluri, “Autonomous Compliance Systems: AI, Event Streaming, and the Future of Financial Crime Prevention,” Journal of Informatics Education and Research, vol. 6, no. 1, pp. 1281-1294, 2026.
[52] U. S. Yandamuri et al., “Adaptive Intelligence Networks for Human-centered Enterprise Automation and Governance,” 2026 Third International Conference on Innovations in Cybersecurity and Data Science (ICICDS), Pathum Thani, Thailand, pp. 678-683, 2026. Doi: https://doi.org/10.1109/ICICDS70526.2026.11604640
[53] S. Amershi et al., "Software Engineering for Machine Learning: A Case Study," 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), May 2019, doi: 10.1109/icse-seip.2019.00042.
[54] Ch. S. Rao, V. D. V. K. Bandi, L. M. K. Brahmandam, S. Gantikota, and K. Kannan, “Topology-Aware Hypergraph Neural Network Framework for Intelligent Decision Support Systems in Network Operations,” 2026 2nd International Conference on Intelligent Systems and Computational Networks (ICISCN), pp. 1–7, Apr. 2026, doi: 10.1109/iciscn67954.2026.11566638.
[55] T. Kalita, S. Prasad Tiwari, A. Reddy Aitha, and A. Garg, "Evolutionary and Swarm-Based Metaheuristics for Neural Architecture Search and Hyperparameter Tuning," SSRN Electronic Journal, 2026, doi: 10.2139/ssrn.6708638.
[56] A. Rajkomar, J. Dean, and I. Kohane, "Machine Learning in Medicine," New England Journal of Medicine, vol. 380, no. 14, pp. 1347–1358, 2019, doi: 10.1056/nejmra1814259.
[57] A. R. Aitha, “Explainable Agentic AI Framework for Automated Insurance Fraud Detection and Predictive Risk Intelligence,” International Journal of Advanced Research in Computer Science & Technology, vol. 9, no. 03, May 2026, doi: 10.15662/ijarcst.2026.0903009.
[58] O. Vinyals, M. Fortunato, and N. Jaitly, "Pointer Networks," Jan. 2017. doi: 10.48550/arXiv.1506.03134.
[59] G. Kim, P. Debois, J. Willis, J. Humble, and J. Allspaw, The DevOps handbook : how to create world-class agility, reliability, and security in technology organizations. Portland, Or: It Revolution Press, Llc, 2017.
[60] A. Gopinath, P. S. L. N. Davuluri, U. B. Kumar, Sahana. M P, and G. Yamini, “Communication Aware Malware Detection Using Network Flow Bytecode Fusion With Adaptive Focal Optimization,” 2026 2nd International Conference on Intelligent Systems and Computational Networks (ICISCN), pp. 1–8, Apr. 2026, doi: 10.1109/iciscn67954.2026.11566323.
[61] W. van der Aalst, Process Mining. Berlin, Heidelberg: Springer Berlin Heidelberg, 2016. doi: 10.1007/978-3-662-49851-4.
[62] R. Mattaparthi, "GenAI-Augmented Diagnostic Reasoning for Diesel Engine Fault Triage: A Large Language Model Framework for Technician Decision Support at Scale," Journal of Material Sciences & Manufacturing Research, vol. 6, no. 12, p. 1, Dec. 2025, doi: 10.47363/jmsmr/2025(6)226.
[63] N. Jaakkola and F. van der Ploeg, "Non-Cooperative and Cooperative Climate Policies with Anticipated Breakthrough Technology," SSRN Electronic Journal, 2018, doi: 10.2139/ssrn.3193873.
[64] SUNKARA, S. K. (2025). LEVERAGING AI, IoT, AND BLOCKCHAIN FOR SCALABLE DIGITAL TRANSFORMATION IN POST-HARVEST SUPPLY CHAINS: A MULTI-SECTOR APPROACH TO ENHANCING EFFICIENCY AND TRACEABILITY (Vol. 26, Issue 7, pp. 2757–2766).
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
