Handwritten Digit Recognition Using Support Vector Machine (SVM) with Radial Basis Function (RBF) Kernel

Firta Sari Panjaitan; Roma Sinta Simbolon; Juliana Batubara

Authors

Firta Sari Panjaitan Institute of Computer Science (IOCSCience), Indonesia
Roma Sinta Simbolon Institute of Computer Science (IOCSCience), Indonesia
Juliana Batubara Institute of Computer Science (IOCSCience), Indonesia

Keywords:

Handwritten Digit Recognition, Support Vector Machine (SVM), Radial Basis Function (RBF) Kernel, Machine Learning, MNIST Dataset

Abstract

Handwritten digit recognition is a fundamental task in machine learning and computer vision, with applications in fields such as postal services, banking, and automated data entry. This research explores the use of Support Vector Machine (SVM) for handwritten number recognition, focusing on the comparison of different kernel functions and their impact on classification performance. The MNIST dataset, a standard benchmark in digit recognition, was used for model evaluation. Various kernel functions, including linear, polynomial, and Radial Basis Function (RBF), were implemented and tested. The results showed that the RBF kernel outperformed the others, achieving an accuracy of approximately 98-99%, demonstrating the SVM's ability to effectively handle non-linearly separable data. A comparison with other machine learning techniques, such as Neural Networks and K-Nearest Neighbors (KNN), revealed that while Neural Networks provided higher accuracy, SVM offered a better balance of efficiency and computational cost. The study concludes that SVM with the RBF kernel is a robust and efficient method for handwritten digit recognition, suitable for medium-sized datasets and applications requiring both high accuracy and computational efficiency. This research contributes to the ongoing development of automated systems for handwritten number recognition in real-world applications.

References

Ahlawat, S., Choudhary, A., Nayyar, A., Singh, S., & Yoon, B. (2020). Improved handwritten digit recognition using convolutional neural networks (CNN). Sensors, 20(12), 3344.

Asif, A., Hannan, S. A., Perwej, Y., & Vithalrao, M. A. (2014). An overview and applications of optical character recognition. Int. J. Adv. Res. Sci. Eng, 3(7), 261–274.

Bohdal, O., Yang, Y., & Hospedales, T. (2020). Flexible dataset distillation: Learn labels instead of images. ArXiv Preprint ArXiv:2006.08572.

Chan, M., Estève, D., Fourniols, J.-Y., Escriba, C., & Campo, E. (2012). Smart wearable systems: Current status and future challenges. Artificial Intelligence in Medicine, 56(3), 137–156.

Dargan, S., Kumar, M., Ayyagari, M. R., & Kumar, G. (2020). A survey of deep learning and its applications: a new paradigm to machine learning. Archives of Computational Methods in Engineering, 27, 1071–1092.

Deka, P. C. (2014). Support vector machine applications in the field of hydrology: a review. Applied Soft Computing, 19, 372–386.

Feng, B.-Y., Ren, M., Zhang, X.-Y., & Suen, C. Y. (2014). Automatic recognition of serial numbers in bank notes. Pattern Recognition, 47(8), 2621–2634.

Fujisawa, H. (2008). Forty years of research in character and document recognition—an industrial perspective. Pattern Recognition, 41(8), 2435–2446.

Hira, Z. M., & Gillies, D. F. (2015). A review of feature selection and feature extraction methods applied on microarray data. Advances in Bioinformatics, 2015(1), 198363.

Hossin, M., & Sulaiman, M. N. (2015). A review on evaluation metrics for data classification evaluations. International Journal of Data Mining & Knowledge Management Process, 5(2), 1.

Kabra, N., Bhattacharya, P., Tanwar, S., & Tyagi, S. (2020). MudraChain: Blockchain-based framework for automated cheque clearance in financial institutions. Future Generation Computer Systems, 102, 574–587.

Kour, V. P., & Arora, S. (2019). Particle swarm optimization based support vector machine (P-SVM) for the segmentation and classification of plants. IEEE Access, 7, 29374–29385.

Liu, C.-L., Nakashima, K., Sako, H., & Fujisawa, H. (2004). Handwritten digit recognition: investigation of normalization and feature extraction techniques. Pattern Recognition, 37(2), 265–279.

Mingqiang, Y., Kidiyo, K., & Joseph, R. (2008). A survey of shape feature extraction techniques. Pattern Recognition, 15(7), 43–90.

Mohapatra, R. K., Majhi, B., & Jena, S. K. (2015). Classification performance analysis of mnist dataset utilizing a multi-resolution technique. 2015 International Conference on Computing, Communication and Security (ICCCS), 1–5.

Nagy, G. (2016). Disruptive developments in document recognition. Pattern Recognition Letters, 79, 106–112.

Nguyen, A. (2020). How Artificial Intelligence can affect postal and parcel industry.

Normanyo, E., Ayim, D., & Isaac, A. (2009). Designing of a letter sorting machine for the regional post offices in Ghana. Engineering and Applied Sciences, 4, 1–13.

Pouyanfar, S., Sadiq, S., Yan, Y., Tian, H., Tao, Y., Reyes, M. P., Shyu, M.-L., Chen, S.-C., & Iyengar, S. S. (2018). A survey on deep learning: Algorithms, techniques, and applications. ACM Computing Surveys (CSUR), 51(5), 1–36.

Saravanan, C. (2010). Color image to grayscale image conversion. 2010 Second International Conference on Computer Engineering and Applications, 2, 196–199.

Sze, V., Chen, Y.-H., Emer, J., Suleiman, A., & Zhang, Z. (2017). Hardware for machine learning: Challenges and opportunities. 2017 IEEE Custom Integrated Circuits Conference (CICC), 1–8.

Tsang, I. W., Kwok, J. T., Cheung, P.-M., & Cristianini, N. (2005). Core vector machines: Fast SVM training on very large data sets. Journal of Machine Learning Research, 6(4).

Wu, Y., Wang, H., Zhang, B., & Du, K.-L. (2012). Using radial basis function networks for function approximation and classification. International Scholarly Research Notices, 2012(1), 324194.

Zhang, S., Zhang, S., Wang, B., & Habetler, T. G. (2020). Deep learning algorithms for bearing fault diagnostics—A comprehensive review. IEEE Access, 8, 29857–29881.

Zhu, Y., Tan, Y., Hua, Y., Wang, M., Zhang, G., & Zhang, J. (2010). Feature selection and performance evaluation of support vector machine (SVM)-based classifier for differentiating benign and malignant pulmonary nodules by computed tomography. Journal of Digital Imaging, 23, 51–65.