Abstract
Handwritten signature verification techniques, which can facilitate user authentication and enable secure information exchange, are still important in property safety. However, on-line automatic handwritten signature verification usually requires dynamic handwritten patterns captured by a special device, such as a sensor-instrumented pen, a tablet or a smartwatch on the dominant hand. This paper presents SonarSign, an on-line handwritten signature verification system based on inaudible acoustic signals. The key insight is to use acoustic signals to capture the dynamic handwritten signature patterns for verification. Particularly, SonarSign exploits the built-in speakers and microphones of smartphones to transmit a specially designed training sequence and record the corresponding echo for channel impulse response (CIR) estimation, respectively. Based on the sensitivity of CIR to the tiny surrounding environment changes including handwritten signature actions, SonarSign designs an attentional multi-modal Siamese network for end-to-end signatures verification. First, multi-modal CIR streams are fused to extract representative signature pattern features from spatio-temporal dimensions. Then an attentional Siamese network is elaborated to verify whether the given two signatures are from the same signatory. Extensive experiments in real-world scenarios show that SonarSign can achieve accurate and robust signatures verification with an AUC (Area Under ROC (Receiver Operating Characteristic) Curve) of 98.02% and an EER (Equal Error Rate) of 5.79% for unseen users.
- Kian Ahrabian and Bagher BabaAli. 2019. Usage of autoencoders and Siamese networks for online handwritten signature verification. Neural Computing and Applications 31, 12 (2019), 9321--9334.Google Scholar
Cross Ref
- Gabe Alvarez, Blue Sheffer, and Morgan Bryant. 2016. Offline signature verification with convolutional neural networks. Technical report, Stanford University (2016).Google Scholar
- Rummana Bari, Roy J Adams, Md Mahbubur Rahman, Megan Battles Parsons, Eugene H Buder, and Santosh Kumar. 2018. rconverse: Moment by moment conversation detection using a mobile respiration sensor. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies 2, 1 (2018), 1--27.Google Scholar
Digital Library
- Luana Batista, Eric Granger, and Robert Sabourin. 2012. Dynamic selection of generative--discriminative ensembles for off-line signature verification. Pattern Recognition 45, 4 (2012), 1326--1340.Google Scholar
Digital Library
- Mengqi Chen, Jiawei Lin, Yongpan Zou, Rukhsana Ruby, and Kaishun Wu. 2020. SilentSign: Device-free Handwritten Signature Verification through Acoustic Sensing. In 2020 IEEE International Conference on Pervasive Computing and Communications (PerCom). IEEE, 1--10.Google Scholar
Cross Ref
- Cao Dian, Dong Wang, Qian Zhang, Run Zhao, and Yinggang Yu. 2020. Towards Domain-independent Complex and Fine-grained Gesture Recognition with RFID. Proceedings of the ACM on Human-Computer Interaction 4, ISS (2020), 1--22.Google Scholar
Digital Library
- Moises Diaz, Miguel A Ferrer, Donato Impedovo, Muhammad Imran Malik, Giuseppe Pirlo, and Réjean Plamondon. 2019. A perspective analysis of handwritten signature technology. Acm Computing Surveys 51, 6 (2019), 1--39.Google Scholar
Digital Library
- Moises Diaz, Andreas Fischer, Miguel A Ferrer, and Réjean Plamondon. 2016. Dynamic signature verification system based on one real signature. IEEE transactions on cybernetics 48, 1 (2016), 228--239.Google Scholar
- Feng Ding, Dong Wang, Qian Zhang, and Run Zhao. 2019. ASSV: handwritten signature verification using acoustic signals. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 1--22.Google Scholar
Digital Library
- Tom Fawcett. 2006. An introduction to ROC analysis. Pattern recognition letters 27, 8 (2006), 861--874.Google Scholar
- Miguel A Ferrer, Jesus B Alonso, and Carlos M Travieso. 2005. Offline geometric parameters for automatic signature verification using fixed-point arithmetic. IEEE transactions on pattern analysis and machine intelligence 27, 6 (2005), 993--997.Google Scholar
Digital Library
- Miguel A Ferrer, J Francisco Vargas, Aythami Morales, and Aaron Ordonez. 2012. Robustness of offline signature verification based on gray level features. IEEE Transactions on Information Forensics and Security 7, 3 (2012), 966--977.Google Scholar
Digital Library
- Andreas Fischer and Réjean Plamondon. 2016. Signature verification based on the kinematic theory of rapid human movements. IEEE Transactions on Human-Machine Systems 47, 2 (2016), 169--180.Google Scholar
Cross Ref
- G. H. Givens, J. R. Beveridge, P. J. Phillips, B. Draper, Y. M. Lui, and D. Bolme. 2013. Introduction to face recognition and evaluation of algorithm performance. Computational Statistics & Data Analysis 67, 11 (2013), 236--247.Google Scholar
Digital Library
- Marta Gomez-Barrero, Javier Galbally, Julian Fierrez, Javier Ortega-Garcia, and Réjean Plamondon. 2015. Enhanced on-line signature verification based on skilled forgery detection using sigma-lognormal features. In 2015 international conference on biometrics (ICB). IEEE, 501--506.Google Scholar
Cross Ref
- Donato Impedovo and Giuseppe Pirlo. 2008. Automatic signature verification: The state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 38, 5 (2008), 609--635.Google Scholar
Digital Library
- Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning. PMLR, 448--456.Google Scholar
- Nidal S Kamel, Shohel Sayeed, and Grant A Ellis. 2008. Glove-based approach to online signature verification. IEEE Transactions on Pattern Analysis and Machine Intelligence 30, 6 (2008), 1109--1113.Google Scholar
Digital Library
- Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. 2015. Siamese neural networks for one-shot image recognition. In ICML deep learning workshop, Vol. 2. Lille.Google Scholar
- Bence Kovari and Hassan Charaf. 2013. A study on the consistency and significance of local features in off-line signature verification. Pattern Recognition Letters 34, 3 (2013), 247--255.Google Scholar
Digital Library
- Songxuan Lai, Lianwen Jin, and Weixin Yang. 2017. Online signature verification using recurrent neural network and length-normalized path signature descriptor. In 2017 14th IAPR international conference on document analysis and recognition (ICDAR), Vol. 1. IEEE, 400--405.Google Scholar
Cross Ref
- Neal S Latman and Emily Herb. 2013. A field study of the accuracy and reliability of a biometric iris recognition system. Science & Justice Journal of the Forensic Science Society 53, 2 (2013), 98--102.Google Scholar
Cross Ref
- Alona Levy, Ben Nassi, Yuval Elovici, and Erez Shmueli. 2018. Handwritten signature verification using wrist-worn devices. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 1--26. Proc. ACM Hum.-Comput. Interact., Vol. 5, No. ISS, Article 499. Publication date: November 2021. Smartphone-based Handwritten Signature Verification using Acoustic Signals 499:25Google Scholar
Digital Library
- Dong Li, Jialin Liu, Sunghoon Ivan Lee, and Jie Xiong. 2020. FM-track: pushing the limits of contactless multi-target tracking using acoustic signals. In Proceedings of the 18th Conference on Embedded Networked Sensor Systems. 150--163.Google Scholar
Digital Library
- Kang Ling, Haipeng Dai, Yuntang Liu, Alex X Liu, Wei Wang, and Qing Gu. 2020. Ultragesture: Fine-grained gesture sensing and recognition. IEEE Transactions on Mobile Computing (2020).Google Scholar
- Jialin Liu, Dong Li, Lei Wang, and Jie Xiong. 2021. BlinkListener: " Listen" to Your Eye Blink Using Your Smartphone. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2 (2021), 1--27.Google Scholar
Digital Library
- Li Lu, Jiadi Yu, Yingying Chen, Hongbo Liu, Yanmin Zhu, Yunfei Liu, and Minglu Li. 2018. LipPass: Lip Readingbased User Authentication on Smartphones Leveraging Acoustic Signals. In 2018 IEEE Conference on Computer Communications. IEEE, 1466--1474.Google Scholar
- Abhishek Malviya and Himanshu Yadav. 2020. An Assessment on Credit Card Fraud Detection: Survey. International Journal of Advanced Computer Technology 9, 5 (2020), 12--15.Google Scholar
- Wenguang Mao, Jian He, and Lili Qiu. 2016. Cat: high-precision acoustic motion tracking. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking. 69--81.Google Scholar
Digital Library
- Wenguang Mao, Wei Sun, Mei Wang, and Lili Qiu. 2020. DeepRange: Acoustic Ranging via Deep Learning. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 4 (2020), 1--23.Google Scholar
Digital Library
- Javier Ortega-Garcia, Julian Fierrez, Fernando Alonso-Fernandez, Javier Galbally, Manuel R Freire, Joaquin Gonzalez- Rodriguez, Carmen Garcia-Mateo, Jose-Luis Alba-Castro, Elisardo Gonzalez-Agulla, Enrique Otero-Muras, et al. 2009. The multiscenario multienvironment biosecure multimodal database (bmdb). IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 6 (2009), 1097--1111.Google Scholar
Digital Library
- Dominique A Potvin, Kirsten M Parris, and Raoul A Mulder. 2011. Geographically pervasive effects of urban noise on frequency and syllable rate of songs and calls in silvereyes (Zosterops lateralis). Proceedings of the Royal Society B: Biological Sciences 278, 1717 (2011), 2464--2469.Google Scholar
Cross Ref
- Markku Pukkila. 2000. Channel estimation modeling. Nokia Research Center 17 (2000), 66.Google Scholar
- Yingyong Qi and Bobby R Hunt. 1994. Signature verification using global and grid features. Pattern Recognition 27, 12 (1994), 1621--1629.Google Scholar
Cross Ref
- A Rodríguez Valiente, A Trinidad, JR García Berrocal, C Górriz, and R Ramírez Camacho. 2014. Extended high-frequency (9--20 kHz) audiometry reference thresholds in 645 healthy subjects. International journal of audiology 53, 8 (2014), 531--545.Google Scholar
- Shibani Santurkar, Dimitris Tsipras, Andrew Ilyas, and Aleksander Madry. 2018. How does batch normalization help optimization?. In Advances in Neural Information Processing Systems. 2483--2493.Google Scholar
- Abhishek Sharma and Suresh Sundaram. 2016. An enhanced contextual DTW based system for online signature verification using vector quantization. Pattern Recognition Letters 84 (2016), 22--28.Google Scholar
Digital Library
- Michael Stauffer, Paul Maergner, Andreas Fischer, Rolf Ingold, and Kaspar Riesen. 2019. Offline signature verification using structural dynamic time warping. In 2019 International Conference on Document Analysis and Recognition (ICDAR). IEEE, 1117--1124.Google Scholar
Cross Ref
- Michael Stauffer, Paul Maergner, Andreas Fischer, and Kaspar Riesen. 2020. A survey of state of the art methods employed in the offline signature verification process. In New Trends in Business Information Systems and Technology. Springer, 17--30.Google Scholar
- Ke Sun and Xinyu Zhang. 2021. UltraSE: single-channel speech enhancement using ultrasound. In Proceedings of the 27th Annual International Conference on Mobile Computing and Networking. 160--173.Google Scholar
Digital Library
- Ke Sun, Ting Zhao, Wei Wang, and Lei Xie. 2018. Vskin: Sensing touch gestures on surfaces of mobile devices using acoustic signals. In Proceedings of the 24th Annual International Conference on Mobile Computing and Networking. 591--605.Google Scholar
Digital Library
- Kalaivani Sundararajan and Damon LWoodard. 2018. Deep learning for biometrics: A survey. ACM Computing Surveys (CSUR) 51, 3 (2018), 1--34.Google Scholar
Digital Library
- Jiayao Tan, Cam-Tu Nguyen, and Xiaoliang Wang. 2017. SilentTalk: Lip reading through ultrasonic sensing on mobile phones. In IEEE INFOCOM 2017-IEEE Conference on Computer Communications. IEEE, 1--9.Google Scholar
Cross Ref
- Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, 11 (2008).Google Scholar
- Dangxiao Wang, Yuru Zhang, Chong Yao, Jun Wu, Huimin Jiao, and Muli Liu. 2009. Toward force-based signature verification: A pen-type sensor and preliminary validation. IEEE Transactions on Instrumentation and Measurement 59, 4 (2009), 752--762.Google Scholar
Cross Ref
- LeiWang, Xiang Zhang, Yuanshuang Jiang, Yong Zhang, Chenren Xu, Ruiyang Gao, and Daqing Zhang. 2021. Watching Your Phone's Back: Gesture Recognition by Sensing Acoustical Structure-borne Propagation. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2 (2021), 1--26.Google Scholar
Digital Library
- Yanwen Wang, Jiaxing Shen, and Yuanqing Zheng. 2020. Push the Limit of Acoustic Gesture Recognition. IEEE Transactions on Mobile Computing (2020). Proc. ACM Hum.-Comput. Interact., Vol. 5, No. ISS, Article 499. Publication date: November 2021. 499:26 Run Zhao et al.Google Scholar
- Reid Kress Weisbord and David Horton. 2019. Inheritance Forgery. Duke LJ 69 (2019), 855.Google Scholar
- Wei Xu, ZhiWen Yu, ZhuWang, Bin Guo, and Qi Han. 2019. Acousticid: gait-based human identification using acoustic signal. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 1--25.Google Scholar
Digital Library
- Mohammad E Yahyatabar and Jamal Ghasemi. 2017. Online signature verification using double-stage feature extraction modelled by dynamic feature stability experiment. IET Biometrics 6, 6 (2017), 393--401.Google Scholar
Cross Ref
- E Yakhnich. 2001. Channel estimation for EGPRS modems. In IEEE VTS 53rd Vehicular Technology Conference, Spring 2001. Proceedings (Cat. No. 01CH37202), Vol. 1. IEEE, 419--422.Google Scholar
Cross Ref
- Jiadi Yu, Li Lu, Yingying Chen, Yanmin Zhu, and Linghe Kong. 2019. An indirect eavesdropping attack of keystrokes on touch screen through acoustic sensing. IEEE Transactions on Mobile Computing (2019).Google Scholar
Digital Library
- Sangki Yun, Yi-Chao Chen, Huihuang Zheng, Lili Qiu, and Wenguang Mao. 2017. Strata: Fine-grained acoustic-based device-free tracking. In Proceedings of the 15th annual international conference on mobile systems, applications, and services. 15--28.Google Scholar
Digital Library
- Cheng Zhang, AbdelKareem Bedri, Gabriel Reyes, Bailey Bercik, Omer T Inan, Thad E Starner, and Gregory D Abowd. 2016. TapSkin: Recognizing on-skin input for smartwatches. In Proceedings of the 2016 ACM International Conference on Interactive Surfaces and Spaces. 13--22.Google Scholar
Digital Library
- David Zhang. 2002. Biometric Solutions. 263--288 pages.Google Scholar
- Fusang Zhang, Zhi Wang, Beihong Jin, Jie Xiong, and Daqing Zhang. 2020. Your Smart Speaker Can" Hear" Your Heartbeat! Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 4 (2020), 1--24.Google Scholar
Digital Library
- Maotian Zhang, Qian Dai, Panlong Yang, Jie Xiong, Chang Tian, and Chaocan Xiang. 2018. idial: Enabling a virtual dial plate on the hand back for around-device interaction. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (2018), 1--20.Google Scholar
Digital Library
- Qian Zhang, Dong Wang, Run Zhao, and Yinggang Yu. 2021. SoundLip: Enabling Word and Sentence-level Lip Interaction for Smart Devices. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 1 (2021), 1--28.Google Scholar
Digital Library
- Qian Zhang, Dong Wang, Run Zhao, Yinggang Yu, and Junjie Shen. 2021. Sensing to Hear: Speech Enhancement for Mobile Devices Using Acoustic Signals. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 3 (2021), 1--30.Google Scholar
Digital Library
- Yongzhao Zhang, Wei-Hsiang Huang, Chih-Yun Yang, Wen-Ping Wang, Yi-Chao Chen, Chuang-Wen You, Da-Yuan Huang, Guangtao Xue, and Jiadi Yu. 2020. Endophasia: Utilizing Acoustic-Based Imaging for Issuing Contact-Free Silent Speech Commands. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 1 (2020), 1--26.Google Scholar
Digital Library
- Yichi Zhang, Meng Yu, Na Li, Chengzhu Yu, Jia Cui, and Dong Yu. 2019. Seq2seq attentional siamese neural networks for text-dependent speaker verification. In ICASSP 2019--2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 6131--6135.Google Scholar
Cross Ref
- Bing Zhou, Jay Lohokare, Ruipeng Gao, and Fan Ye. 2018. EchoPrint: Two-factor authentication using acoustics and vision on smartphones. In Proceedings of the 24th Annual International Conference on Mobile Computing and Networking. 321--336.Google Scholar
Digital Library
- Yongpan Zou, Meng Zhao, Zimu Zhou, Jiawei Lin, Mo Li, and Kaishun Wu. 2018. BiLock: User authentication via dental occlusion biometrics. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 1--20.Google Scholar
Digital Library
Index Terms
Smartphone-based Handwritten Signature Verification using Acoustic Signals
Recommendations
DEMO: Gradiant asymmetric encryption and verification systems based on handwritten signature
CCS '13: Proceedings of the 2013 ACM SIGSAC conference on Computer & communications securityA successful deployment of biometric-based recognition systems in real-life applications depends on crucial issues such as data security and privacy, which have to be specifically addressed. Besides, cryptographic key protection can represent the main ...
On-line Handwritten Signature Verification Using Wavelets and Back-propagation Neural Networks
ICDAR '01: Proceedings of the Sixth International Conference on Document Analysis and RecognitionAbstract: This paper investigates dynamic handwritten signature verification (HSV) using the wavelet transform with verification by the backpropagation neural network (NN). It is yet another avenue in the approach to HSV that is found to produce ...
Wavelet-Based Off-Line Handwritten Signature Verification
In this paper, a wavelet-based off-line handwritten signature verification system is proposed. The proposed system can automatically identify useful and common features which consistently exist within different signatures of the same person and, based ...






Comments