IMPLEMENTASI ARSITEKTUR HIBRIDA XCEPTION-TRANSFORMER DENGAN PENDEKATAN MULTI-DOMAIN PADA DETEKSI VIDEO DEEPFAKE

IMPLEMENTATION OF A HYBRID XCEPTION-TRANSFORMER ARCHITECTURE WITH A MULTI-DOMAIN APPROACH FOR DEEPFAKE VIDEO DETECTION

Authors

  • Tubagus Muhammad Rayhan STMIK IKMI Cirebon
  • Odi Nurdiawan STMIK IKMI Cirebon
  • Nisa Dienwati Nuris STMIK IKMI Cirebon

DOI:

https://doi.org/10.52972/hoaq.vol17no2.p418-429

Keywords:

deepfake, Xception-Transformer, multi-domain, explainable AI, celeb-DF

Abstract

Penelitian ini bertujuan merancang dan mengevaluasi keandalan arsitektur hibrida Xception-Transformer berbasis fusi multi-domain untuk mendeteksi manipulasi video deepfake. Subjek penelitian menggunakan dataset sekunder Celeb-DF v2, yang memiliki tingkat ketimpangan kelas tinggi (5.639 sampel manipulasi dan 890 asli). Pemilihan sampel dilakukan melalui pembagian data pada tingkat video (video-level split) untuk mencegah kebocoran data, dengan mengekstraksi 8 bingkai gambar secara sekuensial. Pendekatan metodologis ini menggabungkan fitur domain spasial (RGB), frekuensi (FFT), dan temporal (Optical Flow) ke dalam tensor fusi awal. Teknik analisis data mengukur performa prediksi melalui metrik akurasi, kurva ROC, dan uji ablasi, didukung pengaplikasian fungsi Binary Focal Crossentropy serta pembobotan kelas dinamis. Hasil pengujian menunjukkan bahwa pada 38 siklus pelatihan, model mencapai tingkat akurasi 98% dan skor AUC 0,985. Model terbukti sukses menangani ketimpangan kelas dengan mereduksi tingkat False Positive menjadi hanya 13 kasus. Uji ablasi membuktikan bahwa penghapusan Transformer menurunkan akurasi drastis ke tingkat 81%, mengonfirmasi krusialnya mekanisme atensi temporal. Analisis Explainable AI melalui proyeksi t-SNE dan Grad-CAM mengonfirmasi pemisahan fitur yang optimal dan lokalisasi visual pada area transisi wajah. Kesimpulannya, fusi Xception-Transformer sangat andal dalam forensik digital. Penelitian selanjutnya merekomendasikan evaluasi silang dataset dan penerapan distilasi pengetahuan guna efisiensi komputasi real-time.

 

This study aims to design and evaluate the reliability of a multi-domain fusion-based Xception-Transformer hybrid architecture for deepfake video detection. The research subject utilizes the Celeb-DF v2 secondary dataset, which exhibits high class imbalance (5,639 manipulated and 890 authentic samples). Sample selection was conducted via video-level data splitting to prevent data leakage, sequentially extracting 8 frames per video. This methodological approach concatenates spatial (RGB), frequency (FFT), and temporal (Optical Flow) domain features into an early fusion tensor. The data analysis technique measures predictive performance through accuracy metrics, ROC curves, and an ablation study, supported by Binary Focal Crossentropy and dynamic class weighting. Experimental results demonstrate that after 38 training epochs, the model achieved an accuracy of 98% and an AUC score of 0.985. The model successfully handled class imbalance, significantly reducing False Positives to 13 cases. The ablation study proves that excluding the Transformer drastically drops accuracy to 81%, confirming the critical role of temporal attention mechanisms. Explainable AI analysis via t-SNE projections and Grad-CAM confirmed optimal feature separation and visual localization on facial transition areas. In conclusion, the Xception-Transformer fusion is highly reliable in digital forensics. Future research recommends cross-dataset evaluation and knowledge distillation for real-time computational efficiency.

References

G. Iglesias, E. Talavera, and A. Díaz-Álvarez, “A survey on GANs for Computer Vision: Recent Research, Analysis and Taxonomy,” Computer Science Review, vol. 48, p. 100553, May 2023, doi: 10.1016/j.cosrev.2023.100553.

A. Heidari, N. Jafari Navimipour, H. Dag, and M. Unal, “Deepfake Detection Using Deep Learning Methods: A Systematic and Comprehensive Review,” WIREs Data Mining and Knowledge Discovery, vol. 14, no. 2, pp. 1–45, Mar. 2024, doi: 10.1002/widm.1520.

T. Walczyna and Z. Piotrowski, “Quick Overview of Face Swap Deep Fakes,” Applied Sciences, vol. 13, no. 11, p. 6711, May 2023, doi: 10.3390/app13116711.

Z. Xu et al., “MobileFaceSwap: A Lightweight Framework for Video Face Swapping,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 3, pp. 2973–2981, Jun. 2022, doi: 10.1609/aaai.v36i3.20203.

M. S. H. Mukta et al., “An Investigation of the Effectiveness of Deepfake Models and Tools,” Journal of Sensor and Actuator Networks, vol. 12, no. 4, p. 61, Aug. 2023, doi: 10.3390/jsan12040061.

Z. Akhtar, “Deepfakes Generation and Detection: A Short Survey,” Journal of Imaging, vol. 9, no. 1, p. 18, Jan. 2023, doi: 10.3390/jimaging9010018.

A. Diel, T. Lalgi, F. S. Mellis, M. Teufel, and A. Bäuerle, “The Harm of DEEPFAKES: a Scoping Review of Deepfakes’ Negative Effects on Human Mind and Behavior,” AI and Society, vol. 41, no. 6, pp. 5971–5987, 2026, doi: 10.1007/s00146-025-02774-0.

A. Broinowski and F. R. Martin, “Beyond the Deepfake Problem: Benefits, Risks and Regulation of Generative AI Screen Technologies,” Media International Australia, vol. 200, no. 1, pp. 126–142, 2026, doi: 10.1177/1329878X241288034.

M. Hameleers, T. G. L. A. van der Meer, and T. Dobber, “Distorting the Truth Versus Blatant Lies: The Effects of Different Degrees of Deception in Domestic and Foreign Political Deepfakes,” Computers in Human Behavior, vol. 152, no. November 2023, p. 108096, 2024, doi: 10.1016/j.chb.2023.108096.

Furizal, A. Ma'arif, H. Maghfiroh, I. Suwarno, D. Prayogi, Kariyamin, S. Lonang, and A.-N. Sharkawy, “Social, Legal, and Ethical Implications of AI-Generated Deepfake Pornography on Digital Platforms: A Systematic Literature Review,” Social Sciences & Humanities Open, vol. 12, p. 101882, Aug. 2025, doi: 10.1016/j.ssaho.2025.101882.

F. Abbas and A. Taeihagh, “Unmasking deepfakes: A Systematic Review of Deepfake Detection and Generation Techniques Using Artificial Intelligence,” Expert Systems with Applications, vol. 252, no. Part B, p. 124260, Oct. 2024, doi: 10.1016/j.eswa.2024.124260.

Á. F. Gambín, A. Yazidi, A. Vasilakos, H. Haugerud, and Y. Djenouri, “Deepfakes: Current and Future Trends,” Artificial Intelligence Review, vol. 57, no. 3, p. 64, Feb. 2024, doi: 10.1007/s10462-023-10679-x.

A. Kaur, A. Noori Hoshyar, V. Saikrishna, S. Firmin, and F. Xia, “Deepfake Video Detection: Challenges and Opportunities,” Artificial Intelligence Review, vol. 57, no. 6, p. 159, May 2024, doi: 10.1007/s10462-024-10810-6.

L. Y. Gong and X. J. Li, “A Contemporary Survey on Deepfake Detection: Datasets, Algorithms, and Challenges,” Electronics, vol. 13, no. 3, p. 585, Jan. 2024, doi: 10.3390/electronics13030585.

L. Deng, H. Suo, and D. Li, “Deepfake Video Detection Based on EfficientNet-V2 Network,” Computational Intelligence and Neuroscience, vol. 2022, pp. 1–13, Apr. 2022, doi: 10.1155/2022/3441549.

M. Alrashoud, “Deepfake Video Detection Methods, Approaches, And Challenges,” Alexandria Engineering Journal, vol. 125, no. March, pp. 265–277, Jun. 2025, doi: 10.1016/j.aej.2025.04.007.

H.-H. Nguyen-Le, V.-T. Tran, D.-T. Nguyen, and N.-A. Le-Khac, “Deepfake Detection Across Image, Video, and Audio: A Comprehensive Survey with Empirical Evaluation of Generalization And Robustness,” Artificial Intelligence Review, vol. 59, no. 10, p. 203, Jun. 2026, doi: 10.1007/s10462-026-11608-4.

A. A. Khan, A. A. Laghari, S. A. Inam, S. Ullah, M. Shahzad, and D. Syed, “A Survey on Multimedia-Enabled Deepfake Detection: State-of-The-Art Tools and Techniques, Emerging Trends, Current Challenges & Amp; Limitations, and Future Directions,” Discover Computing, vol. 28, no. 1, p. 48, Apr. 2025, doi: 10.1007/s10791-025-09550-0.

G. Petmezas, “Cross-Dataset Video Deepfake Detection Using Transformer and CNN Architectures,” Machine Vision and Applications, vol. 37, no. 3, p. 52, May 2026, doi: 10.1007/s00138-026-01809-w.

E. AbdElfattah, H. M. Mousa, A. Elsisi, and N. Mahmoud, “Robust Deepfake Video Detection Using Spatio-Temporal Features and Dynamic Difference Learning,” Scientific Reports, vol. 16, no. 1, pp. 1–25, 2026, doi: 10.1038/s41598-026-53545-w.

K. Omar, R. H. Sakr, and M. F. Alrahmawy, “An ensemble of CNNs with Self-Attention Mechanism for DeepFake Video Detection,” Neural Computing and Applications, vol. 36, no. 6, pp. 2749–2765, Feb. 2024, doi: 10.1007/s00521-023-09196-3.

L. Papa, P. Russo, I. Amerini, and L. Zhou, “A Survey on Efficient Vision Transformers: Algorithms, Techniques, and Performance Benchmarking,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 12, pp. 7682–7700, Dec. 2024, doi: 10.1109/TPAMI.2024.3392941.

S. Yadav and S. S. Mangalampalli, “Deepfake defense: Combining Spatial and Temporal Cues with CNN–BiLSTM–Transformer Architecture,” PLOS One, vol. 20, no. 11, p. e0334980, Nov. 2025, doi: 10.1371/journal.pone.0334980.

B. C. Soundarya and H. L. Gururaj, “Deepfake Detection: Critical Review of State-of-The-Art Approaches and Future Perspectives,” Discover Applied Sciences, vol. 8, no. 2, p. 201, Jan. 2026, doi: 10.1007/s42452-025-08174-9.

G. Petmezas, V. Vanian, K. Konstantoudakis, E. E. I. Almaloglou, and D. Zarpalas, “Video Deepfake Detection Using a Hybrid CNN-LSTM-Transformer Model for Identity Verification,” Multimedia Tools and Applications, vol. 84, no. 33, pp. 40617–40636, Mar. 2025, doi: 10.1007/s11042-024-20548-6.

A. H. Soudy et al., “Deepfake Detection Using Convolutional Vision Transformers and Convolutional Neural Networks,” Neural Computing and Applications, vol. 36, no. 31, pp. 19759–19775, Nov. 2024, doi: 10.1007/s00521-024-10181-7.

Y. Li, X. Yang, P. Sun, H. Qi, and S. Lyu, “Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Jun. 2020, pp. 3204–3213. doi: 10.1109/CVPR42600.2020.00327.

M. S. Momin, A. Sufian, D. Barman, M. Leo, C. Distante, and N. Damer, “Explainable Deepfake Detection Across Different Modalities: An Overview of Methods and Challenges,” Image and Vision Computing, vol. 163, no. March, p. 105738, Nov. 2025, doi: 10.1016/j.imavis.2025.105738.

N. Mansoor and A. I. Iliev, “Explainable AI for DeepFake Detection,” Applied Sciences, vol. 15, no. 2, p. 725, Jan. 2025, doi: 10.3390/app15020725.

P. K. Devada, P. Muppavarapu, H. Guduru, P. P. Kasaraneni, and P. R. Gogulamudi, “Deepfake face Video Detection and Interpretation Using an Explainable XceptionNet and Long Short-Term Memory framework,” Discover Applied Sciences, vol. 8, no. 6, p. 644, May 2026, doi: 10.1007/s42452-026-08675-1.

B. Wang, L. Huang, T. Huang, and F. Ye, “Two-Stream Xception Structure Based on Feature Fusion for DeepFake Detection,” International Journal of Computational Intelligence Systems, vol. 16, no. 1, p. 134, Aug. 2023, doi: 10.1007/s44196-023-00312-8.

M. S. Al-Batah and M. S. Alzboon, “Sea Animal Image Classification Using Machine Learning Algorithms for Accurate and Scalable Prediction,” Discover Artificial Intelligence, vol. 6, no. 1, p. 332, Mar. 2026, doi: 10.1007/s44163-026-01005-9.

K. M. Ang et al., “Optimizing Image Classification: Automated Deep Learning Architecture Crafting with Network and Learning Hyperparameter Tuning,” Biomimetics, vol. 8, no. 7, p. 525, Nov. 2023, doi: 10.3390/biomimetics8070525.

N. A. Sharma, R. R. Chand, Z. Buksh, A. B. M. S. Ali, A. Hanif, and A. Beheshti, “Explainable AI Frameworks: Navigating the Present Challenges and Unveiling Innovative Applications,” Algorithms, vol. 17, no. 6, p. 227, May 2024, doi: 10.3390/a17060227.

R. Tolosana, S. Romero-Tapiador, R. Vera-Rodriguez, E. Gonzalez-Sosa, and J. Fierrez, “DeepFakes Detection Across Generations: Analysis of Facial Regions, Fusion, and Performance Evaluation,” Engineering Applications of Artificial Intelligence, vol. 110, no. February, p. 104673, Apr. 2022, doi: 10.1016/j.engappai.2022.104673.

B. Wang, Y. Li, X. Wu, Y. Ma, Z. Song, and M. Wu, “Face Forgery Detection Based on the Improved Siamese Network,” Security and Communication Networks, vol. 2022, no. Data-Driven Face Forensics and Security 2021, pp. 1–13, Feb. 2022, doi: 10.1155/2022/5169873.

S. AlMuhaideb, H. Alshaya, L. Almutairi, D. Alomran, and S. T. Alhamed, “LightFakeDetect: A Lightweight Model for Deepfake Detection in Videos That Focuses on Facial Regions,” Mathematics, vol. 13, no. 19, p. 3088, Sep. 2025, doi: 10.3390/math13193088.

M. Yu, J. Zhang, S. Li, and J. Lei, “MSFRNet: Two?Stream Deep Forgery Detector Via Multi?Scale Feature Extraction,” IET Image Processing, vol. 17, no. 2, pp. 581–596, Feb. 2023, doi: 10.1049/ipr2.12657.

A. Ismail, M. Elpeltagy, M. S. Zaki, and K. Eldahshan, “An integrated Spatiotemporal-Based Methodology for Deepfake Detection,” Neural Computing and Applications, vol. 34, no. 24, pp. 21777–21791, Dec. 2022, doi: 10.1007/s00521-022-07633-3.

C. Tan, Y. Zhao, S. Wei, G. Gu, P. Liu, and Y. Wei, “Frequency-Aware Deepfake Detection: Improving Generalizability Through Frequency Space Domain Learning,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 5, pp. 5052–5060, Mar. 2024, doi: 10.1609/aaai.v38i5.28310.

T. Lu, Y. Bao, and L. Li, “Deepfake Video Detection Based on Improved CapsNet and Temporal–Spatial Features,” Computers, Materials & Continua, vol. 75, no. 1, pp. 715–740, 2023, doi: 10.32604/cmc.2023.034963.

E. Westphal and H. Seitz, “A Machine Learning Method for Defect Detection and Visualization in Selective Laser Sintering Based on Convolutional Neural Networks,” Additive Manufacturing, vol. 41, no. March, p. 101965, May 2021, doi: 10.1016/j.addma.2021.101965.

M. Rodrigo, C. Cuevas, and N. García, “Comprehensive Comparison Between Vision Transformers and Convolutional Neural Networks for Face Recognition Tasks,” Scientific Reports, vol. 14, no. 1, p. 21392, Sep. 2024, doi: 10.1038/s41598-024-72254-w.

R. Elshawi, A. Wahab, A. Barnawi, and S. Sakr, “DLBench: a Comprehensive Experimental Evaluation of Deep Learning Frameworks,” Cluster Computing, vol. 24, no. 3, pp. 2017–2038, Sep. 2021, doi: 10.1007/s10586-021-03240-4.

H. Li, Y. Wang, Y. Hong, F. Li, and X. Ji, “Layered Mixed-Precision Training: a New Training Method for Large-Scale AI models,” Journal of King Saud University - Computer and Information Sciences, vol. 35, no. 8, p. 101656, Sep. 2023, doi: 10.1016/j.jksuci.2023.101656.

N. Jabbar, S. M. Bhatti, M. Rashid, A. Jaffar, and S. Akram, “Single-layer KAN for Deepfake Classification: Balancing Efficiency and Performance in Resource Constrained Environments,” PLOS One, vol. 20, no. 7, p. e0326565, Jul. 2025, doi: 10.1371/journal.pone.0326565.

O. Rainio, J. Teuho, and R. Klén, “Evaluation Metrics and Statistical Tests for Machine Learning,” Scientific Reports, vol. 14, no. 1, p. 6086, Mar. 2024, doi: 10.1038/s41598-024-56706-x.

S. H. Silva, M. Bethany, A. M. Votto, I. H. Scarff, N. Beebe, and P. Najafirad, “Deepfake forensics Analysis: an Explainable Hierarchical Ensemble of weakly Supervised Models,” Forensic Science International: Synergy, vol. 4, no. December 2021, p. 100217, 2022, doi: 10.1016/j.fsisyn.2022.100217.

Downloads

Published

17-11-2026

How to Cite

Rayhan, T. M., Nurdiawan, O., & Nuris, N. D. (2026). IMPLEMENTASI ARSITEKTUR HIBRIDA XCEPTION-TRANSFORMER DENGAN PENDEKATAN MULTI-DOMAIN PADA DETEKSI VIDEO DEEPFAKE: IMPLEMENTATION OF A HYBRID XCEPTION-TRANSFORMER ARCHITECTURE WITH A MULTI-DOMAIN APPROACH FOR DEEPFAKE VIDEO DETECTION. HOAQ (High Education of Organization Archive Quality) : Jurnal Teknologi Informasi, 17(2), 417–429. https://doi.org/10.52972/hoaq.vol17no2.p418-429