Abstract
When working with a document, users often perform context-specific repetitive edits – changes to the document that are similar but specific to the contexts at their locations. Programming by demonstration/examples (PBD/PBE) systems automate these tasks by learning programs to perform the repetitive edits from demonstration or examples. However, PBD/PBE systems are not widely adopted, mainly because they require modal UIs – users must enter a special mode to give the demonstration/examples. This paper presents Blue-Pencil, a modeless system for synthesizing edit suggestions on the fly. Blue-Pencil observes users as they make changes to the document, silently identifies repetitive changes, and automatically suggests transformations that can apply at other locations. Blue-Pencil is parameterized – it allows the ”plug-and-play” of different PBE engines to support different document types and different kinds of transformations. We demonstrate this parameterization by instantiating Blue-Pencil to several domains – C# and SQL code, markdown documents, and spreadsheets – using various existing PBE engines. Our evaluation on 37 code editing sessions shows that Blue-Pencil synthesized edit suggestions with a precision of 0.89 and a recall of 1.0, and took 199 ms to return suggestions on average. Finally, we report on several improvements based on feedback gleaned from a field study with professional programmers to investigate the use of Blue-Pencil during long code editing sessions. Blue-Pencil has been integrated with Visual Studio IntelliCode to power the IntelliCode refactorings feature.
Supplemental Material
- J. Andersen and J. L. Lawall. 2008. Generic Patch Inference. In Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering (ASE ’08). IEEE Computer Society, Washington, DC, USA, 337–346. Google Scholar
Digital Library
- Jesper Andersen, Anh Cuong Nguyen, David Lo, Julia L. Lawall, and Siau-Cheng Khoo. 2012. Semantic Patch Inference. In Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering (ASE 2012). ACM, New York, NY, USA, 382–385. Google Scholar
Digital Library
- CommonMark. 2019. CommonMark MarkDown Specification. (2019). At https://commonmark.org/ .Google Scholar
- Allen Cypher (Ed.). 1993. Watch What I Do: Programming by Demonstration. MIT Press.Google Scholar
- Eclipse Foundation. 2019. Eclipse. (2019). At https://www.eclipse.org/ .Google Scholar
- Darren Edge, Sumit Gulwani, Natasa Milic-Frayling, Mohammad Raza, Reza Adhitya Saputra, Chao Wang, and Koji Yatani. 2015. Mixed-Initiative Approaches to Global Editing in Slideware. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI ’15). ACM, New York, NY, USA, 3503–3512. Google Scholar
Digital Library
- John K. Feser, Swarat Chaudhuri, and Isil Dillig. 2015. Synthesizing Data Structure Transformations from Input-output Examples. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’15). ACM, New York, NY, USA, 229–239. Google Scholar
Digital Library
- J. Nathan Foster, Michael B. Greenwald, Jonathan T. Moore, Benjamin C. Pierce, and Alan Schmitt. 2007. Combinators for Bidirectional Tree Transformations: A Linguistic Approach to the View-update Problem. ACM Trans. Program. Lang. Syst. 29, 3, Article 17 (May 2007). Google Scholar
Digital Library
- Stephen R. Foster, William G. Griswold, and Sorin Lerner. 2012. WitchDoctor: IDE Support for Real-time Auto-completion of Refactorings. In Proceedings of the 34th International Conference on Software Engineering (ICSE ’12). IEEE Press, Piscataway, NJ, USA, 222–232. http://dl.acm.org/citation.cfm?id=2337223.2337250Google Scholar
Digital Library
- GNU Emacs. 2019. Repeat Commands. https://www.gnu.org/software/emacs/manual/html_node/efaq/Repeating-commands. html . Accessed: 04/03/2019.Google Scholar
- Sumit Gulwani. 2011. Automating String Processing in Spreadsheets Using Input-output Examples. In Proceedings of the 38th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’11). ACM, New York, NY, USA, 317–330.Google Scholar
Digital Library
- JetBrains. 2019a. IntelliJ. (2019). At https://www.jetbrains.com/idea/ .Google Scholar
- JetBrains. 2019b. ReSharper. (2019). At https://www.jetbrains.com/resharper/ .Google Scholar
- Lingxiao Jiang, Ghassan Misherghi, Zhendong Su, and Stephane Glondu. 2007. DECKARD: scalable and accurate tree-based detection of code clones. 96–105. Google Scholar
Digital Library
- Miryung Kim and David Notkin. 2009. Discovering and Representing Systematic Code Changes. In Proceedings of the 31st International Conference on Software Engineering (ICSE ’09). IEEE Computer Society, Washington, DC, USA, 309–319. Google Scholar
Digital Library
- Miryung Kim, Thomas Zimmermann, and Nachiappan Nagappan. 2012. A Field Study of Refactoring Challenges and Benefits. In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering (FSE ’12). ACM, New York, NY, USA, Article 50, 11 pages. Google Scholar
Digital Library
- Tessa Lau. 2001. Programming by Demonstration: a Machine Learning Approach.Google Scholar
- Tessa Lau. 2008. Why PBD systems fail: Lessons learned for usable AI.Google Scholar
- Tessa Lau, Steven A. Wolfman, Pedro Domingos, and Daniel S. Weld. 2001. Your Wish is My Command. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, Chapter Learning Repetitive Text-editing Procedures with SMARTedit, 209–226. http://dl.acm.org/citation.cfm?id=369505.369519Google Scholar
Digital Library
- Vu Le and Sumit Gulwani. 2014. FlashExtract: A Framework for Data Extraction by Examples. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’14). New York, NY, USA, 542–553.Google Scholar
Digital Library
- Alan Leung, John Sarracino, and Sorin Lerner. 2015. Interactive Parser Synthesis by Example. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’15). ACM, New York, NY, USA, 565–574. Google Scholar
Digital Library
- H. Lieberman. 2001. Your Wish Is My Command: Programming by Example. Morgan Kaufmann.Google Scholar
- Michael Martin, Benjamin Livshits, and Monica S. Lam. 2005. Finding Application Errors and Security Flaws Using PQL: A Program Query Language. In Proceedings of the 20th Annual ACM SIGPLAN Conference on Object-oriented Programming, Systems, Languages, and Applications (OOPSLA ’05). ACM, New York, NY, USA, 365–383. Google Scholar
Digital Library
- Na Meng, Miryung Kim, and Kathryn S. McKinley. 2011. Systematic Editing: Generating Program Transformations from an Example. In Proceedings of the 32Nd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’11). ACM, New York, NY, USA, 329–342.Google Scholar
- Na Meng, Miryung Kim, and Kathryn S. McKinley. 2013. LASE: Locating and Applying Systematic Edits by Learning from Examples. In Proceedings of the 35th International Conference on Software Engineering (ICSE ’13). IEEE Press, Piscataway, NJ, USA, 502–511.Google Scholar
- T. Mens and T. Tourwe. 2004. A survey of software refactoring. IEEE Transactions on Software Engineering 30, 2 (Feb 2004), 126–139. Google Scholar
Digital Library
- Microsoft. 2019a. Create or run a macro. https://support.office.com/en-us/article/Create-or-run-a-macro-C6B99036-905C-49A6-818A-DFB98B7C3C9C . Accessed: 04/03/2019.Google Scholar
- Microsoft. 2019b. Microsoft PowerPoint. (2019). At https://products.office.com/en-us/powerpoint .Google Scholar
- Microsoft. 2019c. Microsoft Word. (2019). At https://products.office.com/en-us/word .Google Scholar
- Microsoft. 2019d. Refactoring source code in Visual Studio Code. (2019). At https://code.visualstudio.com/docs/editor/ refactoring#_code-actions-quick-fixes-and-refactorings .Google Scholar
- Microsoft. 2019e. Visual Studio. (2019). At https://www.visualstudio.com .Google Scholar
- Robert C. Miller and Brad A. Myers. 2002. LAPIS: Smart editing with text structure. In CHI ’02. ACM, 496–497.Google Scholar
- Emerson Murphy-Hill and Andrew P. Black. 2007. High Velocity Refactorings in Eclipse. In Proceedings of the 2007 OOPSLA Workshop on Eclipse Technology eXchange (eclipse ’07). ACM, New York, NY, USA, 1–5. Google Scholar
Digital Library
- E. Murphy-Hill, C. Parnin, and A. P. Black. 2009. How we refactor, and how we know it. In 2009 IEEE 31st International Conference on Software Engineering. 287–297. Google Scholar
Digital Library
- Stas Negara, Mihai Codoban, Danny Dig, and Ralph E. Johnson. 2014. Mining Fine-grained Code Changes to Detect Unknown Change Patterns. In Proceedings of the 36th International Conference on Software Engineering (ICSE 2014). ACM, New York, NY, USA, 803–813. Google Scholar
Digital Library
- H. A. Nguyen, A. T. Nguyen, T. T. Nguyen, T. N. Nguyen, and H. Rajan. 2013. A study of repetitiveness of code changes in software evolution. In 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE). 180–190.Google Scholar
- Hoan Anh Nguyen, Tung Thanh Nguyen, Gary Wilson, Jr., Anh Tuan Nguyen, Miryung Kim, and Tien N. Nguyen. 2010. A Graph-based Approach to API Usage Adaptation. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications (OOPSLA ’10). ACM, New York, NY, USA, 302–321. Google Scholar
Digital Library
- William F. Opdyke. 1992. Refactoring Object-oriented Frameworks. Ph.D. Dissertation. Champaign, IL, USA. UMI Order No. GAX93-05645.Google Scholar
Digital Library
- Saswat Padhi, Prateek Jain, Daniel Perelman, Oleksandr Polozov, Sumit Gulwani, and Todd Millstein. 2018. FlashProfile: A Framework for Synthesizing Data Profiles. Proc. ACM Program. Lang. 2, OOPSLA, Article 150 (Oct. 2018), 28 pages. Google Scholar
Digital Library
- J. Park, M. Kim, B. Ray, and D. Bae. 2012. An empirical study of supplementary bug fixes. In 2012 9th IEEE Working Conference on Mining Software Repositories (MSR). 40–49.Google Scholar
- Oleksandr Polozov and Sumit Gulwani. 2015. FlashMeta: A Framework for Inductive Program Synthesis. In Proceedings of the ACM International Conference on Object-oriented Programming Systems, Languages, and Applications (OOPSLA ’15). ACM, New York, NY, USA, 542–553.Google Scholar
Digital Library
- Veselin Raychev, Martin Vechev, and Eran Yahav. 2014. Code Completion with Statistical Language Models. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’14). ACM, New York, NY, USA, 419–428. Google Scholar
Digital Library
- Refsnes Data. 2019. XPath Tutorial. (2019). At https://www.w3schools.com/xml/xpath_intro.asp .Google Scholar
- Reudismam Rolim, Gustavo Soares, Loris D’Antoni, Oleksandr Polozov, Sumit Gulwani, Rohit Gheyi, Ryo Suzuki, and Björn Hartmann. 2017. Learning Syntactic Program Transformations from Examples. In Proceedings of the 39th International Conference on Software Engineering (ICSE ’17). IEEE Press, Piscataway, NJ, USA, 404–415. Google Scholar
Digital Library
- Rishabh Singh and Sumit Gulwani. 2012. Synthesizing Number Transformations from Input-output Examples. In Proceedings of the 24th International Conference on Computer Aided Verification (CAV’12). Springer-Verlag, Berlin, Heidelberg, 634–651. Google Scholar
Digital Library
- Stack Exchange. 2019. How to programmatically edit all hyperlinks in a Word document? https://stackoverflow.com/q/ 3355266 . Accessed: 04/03/2019.Google Scholar
- Friedrich Steimann and Jens von Pilgrim. 2012. Refactorings Without Names. In Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering (ASE 2012). ACM, New York, NY, USA, 290–293. Google Scholar
Digital Library
- Vim. 2019. Macros. https://vim.fandom.com/wiki/Macros . Accessed: 04/03/2019.Google Scholar
- Navid Yaghmazadeh, Xinyu Wang, and Isil Dillig. 2018. Automated Migration of Hierarchical Data to Relational Tables Using Programming-by-example. Proc. VLDB Endow. 11, 5 (Jan. 2018), 580–593. Google Scholar
Digital Library
- Pengcheng Yin, Graham Neubig, Miltiadis Allamanis, Marc Brockschmidt, and Alexander L. Gaunt. 2019. Learning to Represent Edits. In International Conference on Learning Representations. https://openreview.net/forum?id=BJl6AjC5F7Google Scholar
Index Terms
On the fly synthesis of edit suggestions
Recommendations
Learning syntactic program transformations from examples
ICSE '17: Proceedings of the 39th International Conference on Software EngineeringAutomatic program transformation tools can be valuable for programmers to help them with refactoring tasks, and for Computer Science students in the form of tutoring systems that suggest repairs to programming assignments. However, manually creating ...
Can reactive synthesis and syntax-guided synthesis be friends?
PLDI 2022: Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and ImplementationWhile reactive synthesis and syntax-guided synthesis (SyGuS) have seen enormous progress in recent years, combining the two approaches has remained a challenge. In this work, we present the synthesis of reactive programs from Temporal Stream Logic ...
Optimizing synthesis with metasketches
POPL '16Many advanced programming tools---for both end-users and expert developers---rely on program synthesis to automatically generate implementations from high-level specifications. These tools often need to employ tricky, custom-built synthesis algorithms ...






Comments