skip to main content
research-article

Understanding and visualizing full systems with data flow tomography

Published:01 March 2008Publication History
Skip Abstract Section

Abstract

It is not uncommon for modern systems to be composed of a variety of interacting services, running across multiple machines in such a way that most developers do not really understand the whole system. As abstraction is layered atop abstraction, developers gain the ability to compose systems of extraordinary complexity with relative ease. However, many software properties, especially those that cut across abstraction layers, become very difficult to understand in such compositions. The communication patterns involved, the privacy of critical data, and the provenance of information, can be difficult to find and understand, even with access to all of the source code. The goal of Data Flow Tomography is to use the inherent information flow of such systems to help visualize the interactions between complex and interwoven components across multiple layers of abstraction. In the same way that the injection of short-lived radioactive isotopes help doctors trace problems in the cardiovascular system, the use of "data tagging" can help developers slice through the extraneous layers of software and pin-point those portions of the system interacting with the data of interest. To demonstrate the feasibility of this approach we have developed a prototype system in which tags are tracked both through the machine and in between machines over the network, and from which novel visualizations of the whole system can be derived. We describe the system-level challenges in creating a working system tomography tool and we qualitatively evaluate our system by examining several example real world scenarios.

Skip Supplemental Material Section

Supplemental Material

Video

References

  1. M. K. Aguilera, J. C. Mogul, J. L. Wiener, P. Reynolds, and A. Muthitacharoen. Performance debugging for distributed systems of black boxes. In SOSP '03: Proceedings of the nineteenth ACM symposium on Operating systems principles, pages 74--89. ACM Press, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. P. Barham, A. Donnelly, R. Isaacs, and R. Mortier. Using magpie for request extraction and workload modelling. In OSDI'04: Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation. USENIX Association, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. F. Bellard. QEMU, A fast and portable dynamic translator. In USENIX Annual Technical Conference, April 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. M. Castro, M. Costa, and T. Harris. Securing software by enforcing data-flow integrity. In USENIX'06: Proceedings of the 7th conference on USENIX Symposium on Operating Systems Design and Implementation. USENIX Association, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. S. Chong, J. Liu, A. C. Myers, X. Qi, K. Vikram, L. Zheng, and X. Zheng. Secure web applications via automatic partitioning. In In Proceedings of the 21st ACM Symposium on Operating Systems Principles (SOSP'07), October 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. J. Chow, B. Pfaff, T. Garfinkel, K. Christopher, and M. Rosenblum. Understanding data lifetime via whole system simulation. In SSYM'04: Proceedings of the 13th conference on USENIX Security Symposium, pages 22--22. USENIX Association, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. M. Costa, J. Crowcroft, M. Castro, A. Rowstron, L. Zhou, L. Zhang, and P. Barham. Vigilante: end-to-end containment of internet worms. In SOSP '05: Proceedings of the twentieth ACM symposium on Operating systems principles, pages 133--147. ACM Press, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. J. R. Crandall and F. T. Chong. Minos: Control Data Attack Prevention Orthogonal to Memory Model. In MICRO 37: Proceedings of the 37th annual IEEE/ACM International Symposium on Microarchitecture, pages 221--232, Washington, DC, USA, 2004. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. J. R. Crandall, Z. Su, S. F. Wu, and F. T. Chong. On deriving unknown vulnerabilities from zero-day polymorphic and metamorphic worm exploits. In CCS '05: Proceedings of the 12th ACM conference on Computer and communications security, pages 235--248. ACM Press, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. M. Dalton, H. Kannan, and C. Kozyrakis. Raksha: A Flexible Information Flow Architecture for Software Security. In 34th Intl. Symposium on Computer Architecture (ISCA), 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. J. Dean, J. E. Hicks, C. A. Waldspurger, W. E. Weihl, and G. Chrysos. Profileme: Hardware support for instruction-level profiling on out-of-order processors. In Microarchitecture, 1997. Proceedings. Thirtieth Annual IEEE/ACM International Symposium on, pages 292--302. IEEE Computer Society, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. P. Efstathopoulos, M. Krohn, S. VanDeBogart, C. Frey, D. Ziegler, E. Kohler, D. Mazières, F. Kaashoek, and R. Morris. Labels and event processes in the asbestos operating system. SIGOPS Oper. Syst. Rev., 39(5):17--30, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Feng Qin, Cheng Wang, Zhenmin Li, Ho-seop Kim, Yuanyuan Zhou and Youfeng Wu. LIFT: A Low-Overhead Practical Information Flow Tracking System for Detecting General Security Attacks. In Annual IEEE/ACM International Symposium on Microarchitecture, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Guru Venkataramani, Brandyn Roemer, Yan Solihin and Milos Prvulovic. MemTracker: Efficient and Programmable Support for Memory Access Monitoring and Debugging. In 13th International Symposium on High-Performance Computer Architecture (HPCA-13), February 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. A. Haeberlen, P. Kouznetsov, and P. Druschel. Peerreview: Practical accountability for distributed systems. In In Proceedings of the 21st ACM Symposium on Operating Systems Principles (SOSP'07), October 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. M. Hauswirth, P. F. Sweeney, A. Diwan, and M. Hind. Vertical profiling: understanding the behavior of object-priented applications. In OOPSLA '04: Proceedings of the 19th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, pages 251--269. ACM Press, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. A. Ho, M. Fetterman, C. Clark, A. Warfield, and S. Hand. Practical taint-based protection using demand emulation. SIGOPS Oper. Syst. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. J. Newsome and D. Song. Dynamic taint analysis for automatic detection, analysis, and signature generation of exploits on commodity software. In 12th Annual Network and Distributed System Security Symposium (NDSS '05), 2005.Google ScholarGoogle Scholar
  19. N. Joukov, A. Traeger, R. Iyer, C. P. Wright, and E. Zadok. Operating system profiling via latency analysis. In USENIX'06: Proceedings of the 7th conference on USENIX Symposium on Operating Systems Design and Implementation. USENIX Association, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. E. Kiciman and B. Livshits. Ajaxscope: A platform for remotely monitoring the client-side behavior of web 2.0 applications. In In Proceedings of the 21st ACM Symposium on Operating Systems Principles (SOSP'07), October 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. J. R. Larus. Whole program paths. In PLDI '99: Proceedings of the ACM SIGPLAN 1999 conference on Programming language design and implementation, pages 259--269. ACM Press, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. J. Levon and P. Elie. Oprofile: oprofile.sourceforge.net.Google ScholarGoogle Scholar
  23. S. Narayanasamy, G. Pokam, and B. Calder. Bugnet: Continuously recording program execution for deterministic replay debugging. In ISCA '05: Proceedings of the 32nd annual international symposium on Computer Architecture, pages 284--295. IEEE Computer Society, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. G. Portokalidis, A. Slowinska, and H. Bos. Argos: an emulator for fingerprinting zero-day attacks for advertised honeypots with automatic signature generation. SIGOPS Oper. Syst. Rev., 40(4):15--27, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Z. A. Shaw. Mongrel: mongrel.rubyforge.org.Google ScholarGoogle Scholar
  26. G. E. Suh, J. W. Lee, D. Zhang, and S. Devadas. Secure Program Execution via Dynamic Information Flow Tracking. In ASPLOS-XI: Proceedings of the 11th international conference on Architectural support for programming languages and operating systems, pages 85--96, New York, NY, USA, 2004. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. P. F. Sweeney, M. Hauswirth, B. Cahoon, P. Cheng, A. Diwan, D. Grove, and M. Hind. Using hardware performance moniters to understand the behavior of java applications. In USENIX 3rd Virtual Machine Research and Technology Symposium (VM'04). ACM Press, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Úlfar Erlingsson, S. Valley, M. Abadi, M. Vrable, M. Budiu, and G. C. Necula. Xfi: software guards for system address spaces. In USENIX'06: Proceedings of the 7th conference on USENIX Symposium on Operating Systems Design and Implementation. USENIX Association, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. N. Vachharajani, M. J. Bridges, J. Chang, R. Rangan, G. Ottoni, J. A. Blome, G. A. Reis, M. Vachharajani, and D. I. August. Rifle: An architectural framework for user-centric information-flow security. In MICRO 37: Proceedings of the 37th annual IEEE/ACM International Symposium on Microarchitecture, pages 243--254. IEEE Computer Society, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. M. Xu, R. Bodik, and M. D. Hill. A "flight data recorder" for enabling full-system multiprocessor deterministic replay. In ISCA '03: Proceedings of the 30th annual international symposium on Computer architecture, pages 122--135. ACM Press, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. W. Xu, S. Bhatkar, and R. Sekar. Taint-enhanced policy enforcement: a practical approach to defeat a wide range of attacks. In USENIX-SS'06: Proceedings of the 15th conference on USENIX Security Symposium. USENIX Association, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. N. Zeldovich, S. Boyd-Wickizer, E. Kohler, and D. Mazières. Making information flow explicit in histar. In USENIX'06: Proceedings of the 7th conference on USENIX Symposium on Operating Systems Design and Implementation. USENIX Association, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Understanding and visualizing full systems with data flow tomography

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM SIGARCH Computer Architecture News
            ACM SIGARCH Computer Architecture News  Volume 36, Issue 1
            ASPLOS '08
            March 2008
            339 pages
            ISSN:0163-5964
            DOI:10.1145/1353534
            Issue’s Table of Contents
            • cover image ACM Conferences
              ASPLOS XIII: Proceedings of the 13th international conference on Architectural support for programming languages and operating systems
              March 2008
              352 pages
              ISBN:9781595939586
              DOI:10.1145/1346281

            Copyright © 2008 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 1 March 2008

            Check for updates

            Qualifiers

            • research-article

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!