Convex Optimization Methods for Privacy-Preserving Federated Learning
DOI:
https://doi.org/10.64137/3108-2637/IJMAR-V2I2P104Keywords:
Federated Learning, Convex Optimization, Differential Privacy, Proximal Gradient, Convergence Analysis, Privacy-Preserving Machine LearningAbstract
Federated Learning (FL) has emerged as a revolutionary paradigm in distributed machine learning, enabling multiple decentralized clients to collaboratively train models without sharing their local raw data. Despite its inherent privacy-centric design, FL remains vulnerable to sophisticated privacy attacks, such as gradient leakage and membership inference, which can reconstruct sensitive user data from communicated model updates. In order to reduce these vulnerabilities, we integrate privacy-preserving mechanisms most notably Differential Privacy (DP) and Cryptographic Protocols into the training procedure. These privacy constraints, however, come with utility loss and convergence slowdown thus highlighting a basic conflict between (differential) privacy on one side and high-order model accuracy and efficiency at another. In our paper, we carefully examine how to use convex optimization methods systematically in terms of performing this rich multi-dimensional trade-off. We center around the rigorous implementation of privacy-preserving FL couched as a bounded convex optimization task, studying how traditional and state-of-the-art optimization algorithms retain strong convergence rates even under durable privacy constraints. We benchmark the performance of these primary optimization frameworks, such as FedAvg, FedProx, and Accelerated Gradient Methods, when adopted on different privacy budgets. Theoretically, we analyze the impact of differential privacy on gradient variance in algorithms and experimentally validate how adaptive optimization (Specifically by AMSGrad) and proximal regularization can account for this noise-induced increase to enable faster convergence with a tight guarantee of differential privacy. To summarize, this work provides a unified approach for aiding the design of state-of-the-art privacy-preserving distributed learning systems that are also utility-optimal and is an important step towards using such approaches in high-stakes domains like healthcare or finance.
References
[1] M. Abadi et al., "Deep Learning with Differential Privacy," Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security - CCS'16, 2016, doi: 10.1145/2976749.2978318.
[2] Li, X., Huang, K., Yang, W., Wang, S., & Zhang, Z. (2020). On the Convergence of FedAvg on Non-IID Data. In International Conference on Learning Representations (ICLR) 2020. ArXiv: https://arxiv.org/abs/1907.02189
[3] C. Dwork and A. Roth, "The Algorithmic Foundations of Differential Privacy," Foundations and Trends® in Theoretical Computer Science, vol. 9, no. 3–4, pp. 211–407, 2014, doi: 10.1561/0400000042.
[4] R. Geyer, T. Klein, and M. Nabi, "Differentially Private Federated Learning: A Client-Level Perspective," arXiv (Cornell University), Dec. 2017, doi: 10.48550/arxiv.1712.07557.
[5] Li, X., Huang, K., Yang, W., Wang, S., & Zhang, Z. (2020). On the Convergence of FedAvg on Non-IID Data. In International Conference on Learning Representations (ICLR 2020). ArXiv DOI: https://doi.org/10.48550/arXiv.1907.02189
[6] J. Konečný, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and D. Bacon, "Federated Learning: Strategies for Improving Communication Efficiency," arXiv (Cornell University), Oct. 2016, doi: 10.48550/arxiv.1610.05492.
[7] McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2017). Communication-Efficient Learning of Deep Networks from Decentralized Data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), Proceedings of Machine Learning Research, 54, 1273–1282. Official: https://proceedings.mlr.press/v54/mcmahan17a.html
[8] Bonawitz, K., Eichner, H., Grieskamp, W., Huba, D., Ingerman, A., Ivanov, V., Kiddon, C., Konečný, J., Mazzocchi, S., McMahan, H. B., Van Overveldt, T., Petrou, D., Ramage, D., & Roselander, J. (2019). Towards Federated Learning at Scale: System Design. Proceedings of Machine Learning and Systems (MLSys), 1, 374–388. Official:https://proceedings.mlsys.org/paper/2019/hash/bd686fd640be98efaae0091fa301e613-Abstract.html
[9] Mironov, "Rényi Differential Privacy," 2017 IEEE 30th Computer Security Foundations Symposium (CSF), Aug. 2017, doi: 10.1109/csf.2017.11.
[10] Y. Nesterov, Lectures on Convex Optimization. Cham: Springer International Publishing, 2018. doi: 10.1007/978-3-319-91578-4.
[11] S. Reddi et al., "Adaptive Federated Optimization," arXiv (Cornell University), Jan. 2020, doi: 10.48550/arxiv.2003.00295.
[12] R. Shokri, M. Stronati, C. Song, and V. Shmatikov, "Membership Inference Attacks Against Machine Learning Models," IEEE Xplore, May 01, 2017. https://ieeexplore.ieee.org/document/7958568 (accessed July 07, 2026).
[13] S. Truex et al., "A Hybrid Approach to Privacy-Preserving Federated Learning," Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security - AISec'19, 2019, doi: 10.1145/3338501.3357370.
[14] M. Noble, A. Bellet, and Aymeric Dieuleveut, "Differentially Private Federated Learning on Heterogeneous Data," PMLR, pp. 10110–10145, May 2022, Accessed: July 07, 2026. [Online]. Available: https://proceedings.mlr.press/v151/noble22a.html
[15] Y. Zhao, L. Liu, L. Lai, N. Suda, Damon Jay Civin, and V. Chandra, "Federated Learning with Non-IID Data," arXiv (Cornell University), June 2018, doi: 10.48550/arxiv.1806.00582.
[16] A. Suresh, "Large Language Models for Explainable and Self-Service Business Intelligence in Large Enterprises," 2026 6th International Conference on Expert Clouds and Applications (ICOECA), Bengaluru, India, 2026, pp. 1571-1578, doi: 10.1109/ICOECA68095.2026.11485058.
[17] Yachamaneni, T., Arora, A. S., & Kotadiya, U. (2026, January). Reinforcement Learning (RL) for Dynamic Credit Limits Using Java and Cloud-Based AI. In Proceedings of Fifth International Conference on Computing and Communication Networks: ICCCN 2025, Volume 9 (Vol. 9, p. 434). Springer Nature.
[18] Gali, V. K., & Jain, S. (2025). Generative AI in multimodal learning: Integration of vision, text and audio for advanced human‑computer interaction. International Journal of Research in Modern Engineering & Emerging Technology, 13(3), 1–12. https://doi.org/10.63345/ijrmeet.org.13.3.1
[19] Akinapalli, S. (2026). AN AI-POWERED DATA TRUST AND QUALITY SCORING FRAMEWORK FOR ENTERPRISE DECISION INTELLIGENCE SYSTEMS. International Journal of Data Science and IoT Management System, 5(1), 946-950.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
