skip to main content
research-article

Using Network Traffic to Infer Hardware State: A Kernel-Level Investigation

Published:30 April 2015Publication History
Skip Abstract Section

Abstract

In this article, we illustrate that the boundary of a general-purpose node can be extended into the network by extracting information from network traffic generated by that general-purpose node to infer the state of its hardware components. This information is represented in a delay signature latent within the network traffic. In contrast, the traditional approach to determine the internal state of a node’s resources meant that a software application with internal processes had to be resident on the node. The aforementioned delay signature is the keystone that provides a correlation between network traffic and the internal state of the source node. We characterize this delay signature by (1) identifying the different types of assembly language instructions that source this delay and (2) describing how architectural techniques, such as instruction pipelining and caching, give rise to this delay signature. In theory, highly utilized nodes (due to multiple threads) will contain excessive context switching and contention for shared resources. One important shared resource is main memory, and excessive use of this resource by applications and internal processes eventually leads to a decrease in cache efficiency that eventually stalls the instruction pipeline. Our results support this theory; specifically, we have observed that excessive context switching in active applications increases the effective memory access time and wastes precious CPU cycles, thus adding additional delay to the execution of load, store, and other instructions. Because the operating system (OS) kernel accesses memory to send network packets, the delay signature is induced into network traffic in situations where user-level utilization is high. We demonstrate this theory in two case studies: (1) resource discovery in cluster grids and (2) network-based detection of bitcoin mining on compromised nodes.

References

  1. Aeroflex Gaisler. 2011. Homepage. Retrieved November 1, 2011, from http://www.gaisler.com.Google ScholarGoogle Scholar
  2. Bitcoin Forum. 2011. Homepage. Retrieved November 1, 2011, from https://bitcointalk.org/index.php?topic=7219.0.Google ScholarGoogle Scholar
  3. BitcoinCZ. 2011. Homepage. Retrieved November 1, 2011, from http://mining.bitcoin.cz.Google ScholarGoogle Scholar
  4. C. Benvenuti. 2005. Understanding Linux Network Internals. O’Reilly Publishers, Sebastopol, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. N. Binkert, L. Hsu, A. Saidi, R. Dreslinski, A. Schultz, and S. Reinhardt. 2005. Performance analysis of system overheads in TCP/IP workloads. In Proceedings of the 4th International Conference on Parallel Architectures and Compilation Techniques (PACT’05). Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. S. Chaisiri and P. Uthayopas. 2008. Survey of Resource Discovery in Grid Environments. Retrieved November 1, 2011, from http://javaboom.files.wordpress.com/2008/04/rs_grid_survey.pdf.Google ScholarGoogle Scholar
  7. Deterlab. 2011. Homepage. Retrieved November 1, 2011, from http://www.deterlab.net.Google ScholarGoogle Scholar
  8. A. Foong, T. Huff, H. Hum, J. Patwardhan, and G. Regnier. 2003. TCP performance re-visited. In Proceedings of the IEEE International Symposium on Performance Analysis of Systems and Software. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. A. Gopu, R. Repasky, and S. McCaulay. 2007. Survey of TeraGrid Job Distribution: Toward Specialized Serial Machines as TeraGrid Resources. TeraGrid 2007 Conference. Madison, WI.Google ScholarGoogle Scholar
  10. A. Gupta. 2008. Black Box Methods for Inferring Parallel Applications Properties in Virtual Environments. Dissertation, Northwestern University. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. M. Harchol-Balter, T. Leighton, and D. Lewin. 1999. Resource discovery in distributed networks. In Proceedings of the ACM Symposium on Principles of Distributed Computing. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. R. Holloway and R. Beyah. 2011. Covert DCF: A DCF-based covert timing channel in 802.11 networks. In Proceedings of the IEEE International Conference on Mobile Ad-hoc and Sensor Systems (MASS’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. L. Hu and X. Che. 2008. Design and implementation of bandwidth prediction based on grid service. In Proceedings of the IEEE International Conference on High Performance Computing and Communications. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. M. Jones. 2007. Anatomy of the Linux Kernel: History and Architectural Decomposition. Retrieved November 1, 2011, from http://www.ibm.com/developerworks/linux/library/l-linux-kernel/?S_TACT=105AGX59&S__CMP=GR&ca=dgr-lnxw01LKernalAnatomy##author1.Google ScholarGoogle Scholar
  15. H. Kim, V. Pai, and S. Rixner. 2002. Increasing web server throughput with network interface data caching. In Proceedings of the ACM 10th International Conference on Architectural Support for Programming Languages and Operating Systems. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. H. Kim, S. Rixner, and V. Pai. 2005. Network interface data caching. In IEEE Transactions on Computers. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. H. Kim, V. Pai, and S. Rixner. 2003. Exploiting task-level concurrency in a programmable network interface. In Proceedings of the ACM SIGPLAN Symposium on Principles and Practices of Parallel Programming (PPoPP’03). Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. M. Luckie, A. McGregor, and H. Braun. 2001. Towards improving packet probing techniques. In Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Linux Kernal Map. 2011. Homepage. Retrieved November 1, 2011, from http://www.gaisler.com/doc/LEON4_32-bit_processor_core.pdf.Google ScholarGoogle Scholar
  20. Linux Kernal Map. 2011. Homepage. Retrieved November 1, 2011, from http://www.makelinux.net/kernel_map_intro.Google ScholarGoogle Scholar
  21. Mathworks. 2011. Homepage. Retrieved November 1, 2011, from http://www.mathworks.com/matlabcentral/fileexchange/6291.Google ScholarGoogle Scholar
  22. P. Magnusson Sparc Architecture. 2011. Homepage. Retrieved November 1, 2011, from http://www.sics.se/psm/sparcstack.html.Google ScholarGoogle Scholar
  23. S. Makineni and R. Iyer. 2003. Performance characterization of TCP/IP packet processing in commercial server workloads. In Proceedings of the IEEE International Workshop on Workload Characterization (WWC’03).Google ScholarGoogle Scholar
  24. J. Mogul and A. Borg. 1991. The effect of context switches on cache performance. ACM SIGARCH Computer Architecture News Archive 19, 2, 75--84. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. J. Mudigonda, H. M. Vin, and R. Yavatkar. 2005. Overcoming the memory wall in packet processing: hammers or ladders? In Proceedings of the ACM Symposium on Architecture for Networking and Communications Systems. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. R. Newman and R. Beyah. 2009. On the performance of using covert timing channels for node authentication. Security and Communication Networks Journal 6.Google ScholarGoogle Scholar
  27. S. Nakamoto. 2008. Bitcoin: A Peer-to-Peer Electronic Cash System. Retrieved November 1, 2011, from https://bitcoin.org/bitcoin.pdf.Google ScholarGoogle Scholar
  28. Norton Antivirus 2002 Email Scanner Buffer Overflow Vulnerability. 2003. Homepage. Retrieved November 1, 2011, from http://www.securityfocus.com/bid/6886.Google ScholarGoogle Scholar
  29. S. Oh and S. Kim. 2006. An efficient Linux kernel module supporting TCP/IP offload engine on grid. In Proceedings of the the 5th IEEE International Conference on Grid and Cooperative Computing. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. D. Patterson and J. Hennessy. 1994. Computer Organization & Design: The Hardware/Software Interface. Morgan Kaufmann, San Francisco, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. M. Peck. 2012. How bitcoin brought privacy to electronic transactions. IEEE Spectrum Magazine, June.Google ScholarGoogle Scholar
  32. Improving System Cooling Part 2 - Keeping the North Bridge Cool. (August 2003). Retrieved November 1, 2011, from http://www.informit.com/articles/article.aspx?p=339028.Google ScholarGoogle Scholar
  33. S. Radhakrishnan, S. Uluagac, and R. Beyah. 2013. Realizing an 802.11-based covert timing channel using off-the-shelf wireless cards. In Proceedings of the IEEE Global Communications Conference (GLOBECOM’13).Google ScholarGoogle Scholar
  34. SecurityFocus. 2002. ISS Internet Scanner HTTP Banner Text Parsing Buffer Overflow Vulnerability. Homepage. Retrieved November 1, 2011, from http://www.securityfocus.com/bid/5738.Google ScholarGoogle Scholar
  35. Symantec Client Security and Symantec AntiVirus Elevation of Privilege. 2006. Homepage. Retrieved November 1, 2011, from http://www.symantec.com/avcenter/security/Content/2006.05.25.html.Google ScholarGoogle Scholar
  36. F. Standaert. 2010. Secure Integrated Circuits and Systems: Introduction to Side-Channel Attacks. Springer, New York, 27--42.Google ScholarGoogle Scholar
  37. R. Stevens, B. Fenner, and A. Rudoff. 2003. Unix Network Programming, Vol. 1: The Sockets Networking API (3rd. ed.). Addison-Wesley Professional. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. W. Stevens. 1994. TCP/IP Illustrated: The Protocols. Addison-Wesley Professional. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. S. Storie and M. Sosonkina. 2004. Packet probing as network load detection for scientific applications at run-time. In Proceedings of the 18th International Parallel and Distributed Processing Symposium.Google ScholarGoogle Scholar
  40. Symantec. 2011. Security Response Blog. Retrieved November 1, 2011, from http://www.symantec.com/connect/blogs/bitcoin-botnet-mining.Google ScholarGoogle Scholar
  41. A. Sharma and S. Bawa. 2006. An improved resource discovery approach using P2P model for condor: A grid middleware. In Proceedings of World Academy of Science, Engineering and Technology, Dec 17.Google ScholarGoogle Scholar
  42. D. M. Tullsen, S. J. Eggers, and H. M. Levy. 1995. Simultaneous multithreading: Maximizing on-chip parallelism. In Proceedings of the ACM International Symposium on Computer Architecture (ISCA’95). Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. TORQUE. (November 2011). Retrieved November 1, 2011, from http://www.clusterresources.com/pages/products/torque-resource-manager.php.Google ScholarGoogle Scholar
  44. TeraGrid. 2011. Homepage Retrieved November 1, 2011, from http://teragrid.org.Google ScholarGoogle Scholar
  45. L. Watkins, C. Corbet, and R. Beyah. 2008. Passive identification of under utilized CPUs in high performance cluster grid networks. In Proceedings of the IEEE International Conference on Communications (ICC’08).Google ScholarGoogle Scholar
  46. L. Watkins, W. H. Robinson, and R. Beyah. 2011. A passive solution to the CPU resource discovery problem in cluster grid networks. IEEE Transactions on Parallel and Distributed Systems 22, 12, 2000--2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. L. Watkins, W. H. Robinson, and R. Beyah. 2010. A passive solution to the memory resource discovery problem in computational clusters. IEEE Transactions on Network and Service Management 7, 4, 218--230. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. W. Wang, J. Wang, and J. Li. 2005. Study on enhanced strategies for TCP/IP offload engines. In Proceedings of the 11th IEEE International Conference on Parallel and Distributed Systems. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Q. Wu and T. Wolf. 2008. On runtime management in multi-core packet processing systems. In Proceedings of the 4th ACM Symposium on Architectures for Networking and Communications Systems. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. R. Yung and N. Wilhelm. 1995. Caching processor general registers. In Proceedings of the IEEE International Conference on Computer Design. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Using Network Traffic to Infer Hardware State: A Kernel-Level Investigation

                                    Recommendations

                                    Comments

                                    Login options

                                    Check if you have access through your login credentials or your institution to get full access on this article.

                                    Sign in

                                    Full Access

                                    PDF Format

                                    View or Download as a PDF file.

                                    PDF

                                    eReader

                                    View online with eReader.

                                    eReader
                                    About Cookies On This Site

                                    We use cookies to ensure that we give you the best experience on our website.

                                    Learn more

                                    Got it!