Definitions and Dimensions of Liveness

Anonymized author

Introduction

Over the past 34 years, we’ve grown a community with liveness as its rallying cry, and a dream of making programming direct, visible, tangible, and alive at its heart. But it’s striking to realize that – even as we come together to discuss liveness at LIVE – we don’t really know what liveness is. Definitions of liveness in the literature are often vague, more gestures towards a direction than falsifiable descriptions. When definitions do flirt with specificity, they sometimes contradict each other in interesting but unexamined ways.

Alongside the ambiguity of liveness’s definition, we have a second problem: While it is clear that approaches to liveness range widely, varying across many axes, there has been little work to articulate the dimensions of variation which efforts towards liveness encompass. Searching out patterns and mapping out these dimensions might hint us towards unexplored realms of design space, or at least give us a clearer shared language.

This essay begins what I hope will be an ongoing, collaborative effort to figure out what liveness means to us, and to map out the space of possibility we are journeying through together.

I begin by considering three intersecting definitions of liveness. I then move to dimensions, and present a handful that I think are prominent in our work. Everything here is somewhat speculative, personal, and ad-hoc, but a conversation has to start somewhere.

Definitions of liveness

The present-day conception of liveness is a bundle of definitions. While these definitions are generally correlated to and supportive of one another, at times they can peel apart and even pull in opposite directions.

The first, perhaps most obvious definition is:

Definition 1: Live programming is editing a running program.

Some instances of this definition from the literature, with emphasis added:

Liveness in programming environments generally refers to the ability to modify a running program. .

Live programming allows programmers to edit the code of a running program and immediately see the effect of the code changes.

This definition stands in opposition to the typical, un-live norm, that editing a program and running it are distinct phases in a programming workflow, often separated by long compilation delays. In contrast, Definition 1 suggests immediacy and feedback: seeing the effects of your programming actions as fast as possible.

But the letter of Definition 1 doesn’t quite match these downstream insinuations. The literal definition is focused on a specific technical question: Is the program being edited as it is running? This is only possible for programs that run long enough for editing to occur during runtime. This nicely matches some situations:

However, if a program runs in an instant, it would seem to be inaccessible to being edited as it is running. This excludes a large fraction of situations we care about, such as fast-running scripts and test cases for larger programming systems. It also excludes many systems which are colloquially called live by their authors or the broader community, including:

These systems all create a live experience by re-running a program when it is edited. This means they do not satisfy Definition 1’s strict editing a running program criterion. But they easily satisfy a second, broader definition:

Definition 2: Live programming is programming in an environment that provides feedback on the dynamic behavior of the program.

Some instances of this definition from the literature, with emphasis added:

Visual programming systems can be classified according to the degree to which they present live feedback to the programmer.

Tanimoto coined the term liveness, which categorizes the immediacy of semantic feedback that is automatically provided during programming.

[Liveness] seems to be used when describing programming tools which provide immediate feedback on the dynamic behavior of a program even while programming. In another section, the authors define liveness as the impression of changing a program while it is running, with the term impression narrowly dodging our concerns about Definition 1.

Live programming is a coding regime in which immediate feedback is provided to the programmer each time the program is modified.

Live programming environments aim to provide programmers … with continuous feedback about a program’s dynamic behavior as it is being edited.

Live programming systems give the programmer immediate feedback on the output of a program as it is being edited.

Although Definition 1 remains an important strand in live-programming research, I believe that Definition 2 has risen to become more prominent, and it is the definition I will focus on for the rest of this essay. Definition 2 does a better job of expressing the actual end goal of live programming – giving programmers environments in which they can see what programs do as they edit them. In the specific case of long-running programs, this may well require editing a program as it runs. But this is a technical means to an experiential end in a specific case, not an end in itself.

Furthermore, there are cases in which it isn’t even clear whether a program is being edited as it runs. For instance, when a spreadsheet cell is edited, the cell and all downstream cells re-run to show the consequences of the change. If the spreadsheet is viewed as a single program, this is Definition-1 liveness. If the cells are viewed as separate programs, this is not. Similar questions arise during use of REPLs or computational notebooks. In contrast, Definition-2 liveness in all these cases is clear and present.

A last note on Definition 2: Some, but not all, of the quotes above use the word dynamic to describe feedback that qualifies as live. This refers to information that is only available at runtime, like the specific values of variables, in contrast to information available through static analysis, like types. I believe access to dynamic information is broadly considered essential to liveness. I believe all the authors above would agree that, say, IDEs presenting inferred type annotations to a programmer do not qualify as live. Consequently, I include dynamic behavior in my text of Definition 2. But really, this is another fault line which the definition of liveness may cleave along!

Finally, I want to offer a third definition, which, though it is not found explicitly in the literature, is implicitly present in some discussions of liveness and which I would like to carefully distinguish from the above definitions:

Definition 3: Live programming is programming that occurs close to direct use.

This definition makes live programming a subset of end-user programming: programming by users to fulfill their own needs []. But Definition 3 refers to a specific kind of end-user programming. Someone making an iOS application for their own use in Xcode is engaging in end-user programming. But this person is working much farther from their moment of use than, say, someone writing a one-off shell script to accomplish a task. This makes their programming less live, in Definition 3’s sense.

Like the first two definitions, this definition is about making the typically abstract activity of programming closer to something more real. The first two definitions relate programming to the reality of execution, but this may still be the sort of test-case execution done by software engineers. Definition 3 brings in a different kind of reality, identifying situations when programming is a means to direct action motivated by immediate need.

To exhibit ways that these three definitions can overlap or miss one another, here is an assortment of programming situations together with my (rough) best guesses as to how well they satisfy our three definitions of liveness:

1. Modify while running2. Feedback on dynamics3. Closeness to use
Projection Boxes--
Hot Module Replacement in webdev--
Smalltalk?
TidalCycles-
Spreadsheet??
Shell scripting--

Dimensions of liveness

Now that we have a few working definitions of liveness in hand, we can begin to map out dimensions of variation between live systems.

This list is ad-hoc. It includes dimensions which have arisen from my own thinking about liveness, dimensions that have come up in conversation with other researchers, and dimensions that have been described in the literature.I have found that one useful pump for thinking of dimensions is interrogating comparisons. Whenever you feel one system is more live than another, you can introspect more deeply and ask: How do these systems differ to make one feel more live than the other? This can sometimes reveal new dimensions.

Ready? Let’s jump in.

Structural granularity

How detailed is the feedback in a live system? Specifically, how much does it provide visibility into the internals of the program? This is the structural granularity of feedback.

To explore this dimension, from the low end to the high end, we can follow the taxonomy of Approaches to Liveness offered by .

This taxonomy begins with liveness outside of code: systems [providing] quick feedback about the final output of a program, without revealing information on program internals that led to that output. Examples here include HMR systems popular for web-app development and split-screen editors popular for generative art. These systems offer the coarsest possible structural granularity: while the overall behavior of a program is made visible and responsive, the internals of the program remain a black box.

Increasing granularity from here, we reach liveness between textual code cells. This refers to systems, like spreadsheets and computational notebooks, that let a programmer break up their code into cells. Division into cells provides a convenient sites for visibility: live values flowing between cells can be visualized alongside the cells.

However, as Horowitz & Heer argue in their work on Engraft [], the flat, static structure of cell-based systems prevent visibility from extending into nested structures like functions, loops, or conditionals. This motivates efforts to push all the way to the most fine-grained end of the spectrum, liveness within textual code / liveness within structure editors, where feedback is threaded through constructs of the programming language itself. This includes, among many other systems, the code editors surveyed in Rauch et al.’s Babylonian-style Programming [].

A finer structural grain would seem to be advantageous, by providing more feedback to the programmer and leaving fewer places where they must tediously and delicately imagine in their mind what the computer is doing. However, excessive feedback can be distracting and reduce information density, diluting code with irrelevant details. Careful visual and interaction design is needed to ensure feedback doesn’t get in the way. For this reason, effective liveness may in fact be more a challenge of information design than one of programming-language design.

Change granularity

How often are changes to a program responded to with feedback in a live system? This is the change granularity of feedback.

As an example of moving along this spectrum, consider Observable []. Observable is a reactive notebook, which re-runs a cell reactively whenever its code or an upstream dependency value changes. In this way, its liveness has a finer change grain than Jupyter [], which requires cells to be re-run manually, even if their dependencies change. However, a code change in Observable only takes effect when it is saved manually or its text editor loses focus, not on every keystroke. In this way, its liveness has a coarser change grain than Natto [], which can re-run cells on every keystroke.

A finer change grain would seem to advantageous, by providing feedback to a programmer more quickly. With a fine change grain, programming can begin to feel more like aiming a water hose than aiming a bow-and-arrow, to use a striking metaphor from . However, a too-fine change grain can be dangerous. Code usually moves through invalid states on the way from one valid state to another. If these invalid programs are indiscriminately executed, this can make the programmer’s experience sluggish, produce meaningless visual output, and possibly even trigger damaging side effects.

A few systems have taken ingenious approaches to obtaining the advantages of high change granularity without its pitfalls. Hazel [] uses a structural editor so that every editor state is meaningful and can produce helpful feedback. More prosaically, some cell-based systems (including Natto and Hex []) allow the user to select how often each cell should be re-evaluated, thereby allowing the user to customize change granularity on a cell-by-cell basis.

A last note: I originally planned to call this dimension temporal granularity to make a space/time metaphor with structural granularity. But then I fortuitously stumbled upon VisiProg [], a remarkably early (avant la lettre) live-programming project. Its paper defines change granularity as the size of the change that VisiProg must detect before re-executing. In deference to the literature (and because it’s frankly a clearer name), I’m going with change granularity.

Recomputation granularity

also named another form of granularity: recomputation granularity. This refers not to the frequency with which changes to a program should trigger recomputation, but to the amount of work that must be done on each recomputation.

At one end of this spectrum lies the easiest, crudest, always-available option: recompute everything from scratch whenever anything changes. For instance, a spreadsheet could, on every change, clear out and recompute all its output values. In practice, spreadsheets do not do this. Instead, they keep track of dependencies between cells so they can intelligently know which cells require recomputation when something changes. In this way, spreadsheets maintain a finer recomputation granularity. (Terms for this in the broader world of computing include incremental and reactive.)

In contrast to change granularity, which is a part of the programmer-facing experience (a psychological issue, in ’s phrasing), recomputation granularity is a behind-the-scenes concern. A programmer will only be aware of it through how slowly or quickly the system responds to their changes. Of course, performance concerns like this are often very much of the essence. And there are interesting connections to be made between behind-the-scenes dimensions and programmer-facing dimensions. For instance, it is striking how the infrastructure required for a fine structural granularity (), such as a splitting of code into cells, can also serve as an infrastructure for dependence-tracking and maintaining a fine recomputation granularity.

Distance between code and output

How is live feedback brought into the programmer’s editing environment, and how can the programmer navigate between code and output?

When structural granularity is high, small bits of code like individual expressions and commands have associated output to show. In this case, there is an obvious approach to connecting this output to code: just put them next to each other on the screen This is the happy path taken by most high-structural-granularity live systems, from spreadsheets to notebooks to Projection Boxes [] to Lamdu [].The simplicity of this happy path may in some cases be misleading. Bringing liveness to programmers in practice may require working with their editing environments as they currently stand, and this may require placing feedback at a greater distance from code. For instance, Projection Boxes only runs on a custom fork of VS Code, since VS Code does not provide sufficient UI hooks for it to run as an extension. If the functionality of Projection Boxes were forced to be shown in a separate interface, new questions would arise about mapping back and forth between the editing and output environments.

In a system with lower structural granularity, maintaining connections between code and output becomes more difficult. For instance, take a liveness outside of code system that generates a user interface or a computational drawing. By default, the full final interface/drawing is visible as a monolith. It might update quickly when the program changes, but it will be difficult for the programmer to move back and forth between the code and the output, to answer questions like which bit of output does a this bit of code generate? and which bit of code generates this bit of output?.

Numerous live-programming systems have explored ways to bridge these gaps. Bret Victor’s Inventing on Principle canvas demo [] includes a magnifying glass feature which highlights output shapes when a line of code is highlighted, and visa versa. TouchDevelop [] similarly provides navigable bidirectional connections between rendered elements in the live view and code that created these elements in the code view. Sketch-n-Sketch [] goes beyond both of these, making output shapes on a canvas not just links to textual code, but direct-manipulation handles for editing the code.

Logs can be considered a form of liveness, albeit one that scores poorly on distance between code and output, as output travels far away to a disassociated console. Log-it [] narrows this distance with features like showing the code that generated a log next to the log, and structuring logs in a hierarchical structure mirroring code.

Distance between execution and effects

A very common approach for live-programming systems is to re-run a program when it changes. ( actually defines live programming this way: Live programming makes programming easier by re-executing a program continuously during editing.) This spreadsheet-like approach has proven broadly useful. But the spreadsheet comparison points out an important limitation. Spreadsheets don’t, by themselves, do anything: running a spreadsheet cell just produces an output, so it’s safe to re-run a cell as much as you want to examine how its output changes. While systems like spreadsheets based on a pure functional programming paradigm are amenable to this treatment, it’s less obvious how programming systems with side-effects can be made satisfyingly live.

One good idea is to eliminate side-effects where you can. For instance, let’s compare Jupyter to Observable. Cells in a Jupyter notebooks are connected through their side-effects on a Python interpreter. One cell can modify Python variables, and another can see the results of these changes. As alluded to in , this limits the liveness of Jupyter. When a cell is changed, downstream cells must be re-run manually, since relationships between cells are implicit and not necessarily cleanly reproducible. Observable overcomes this limitation by moving to a spreadsheet model, where each cell produces a value rather than editing an underlying interpreter state. (Observable cells can in fact mutate state in a shared interpreter, but this is considered poor form.) This is a story of improving the liveness of a system by eliminating unnecessary side-effects.

However, some effects are necessary: essential rather than accidental. Sometimes programs actually need to do things like write to files or send requests across a network or move a robot arm. Live-programming systems have not done much yet to grapple with this reality. As report, Although communication with the external world poses fundamental challenges for live programming, … studies about this phenomenon are rare. But there are many opportunities here, from simply flagging that a part of a program contains side-effects and thus shouldn’t be treated as live-ly as other parts, to mocking or simulating effects in a way that allows them to be performed repeatedly.

Tanimoto’s levels

I will end my listing of dimensions by offering a quick note on the only pre-existing dimension of liveness in the literature.

In his paper coining liveness, Tanimoto proposed a four-level hierarchy of liveness []. In a follow-up paper, he extended this with two additional levels []. Quoting from , these are:

  1. Informative (e.g., flowchart as ancillary description)
  2. Informative and significant (e.g., executable flowchart)
  3. Informative, significant and responsive (e.g., edit-triggered updates)
  4. Informative, significant, responsive and live (e.g., stream-driven updates)
  5. Tactically predictive (adds programming via selection from running predicted behaviors)
  6. Strategically predictive (adds inference of gross functionality)

Unfortunately, I believe that Tanimoto’s hierarchy is no longer aligned with the contemporary definition of liveness, which (per Definition 2) is centered around giving programmers dynamic feedback. His levels 1 & 2 do not require any liveness at all, in our sense of the word, and instead focus on the representation of code itself as a visual structure, which is neither sufficient nor necessary for any of our definitions of liveness. His levels 3 & 4 match our definition of liveness more closely, with the distinction between 3 and 4 arguably coming down to a difference of change granularity (). His levels 5 & 6 once again diverge from our definition of liveness, as they refer to the programming system anticipating the programmer’s future actions and synthesizing new code automatically. While this could, with a stretch, be deemed a form of feedback, it goes well beyond the goal of passive visibility that Definition 2 expresses.

Aside from the specific ways that Tanimoto’s levels deviate from the modern uses of liveness, I would caution against any attempt to reduce liveness to a one-dimensional scale. As I hope this essay has demonstrated, the design space of liveness is complex and multidimensional. Naming dimensions and mapping out their intersections is likely to be more fruitful than setting up a single ranked order.

Conclusion

There are a few things I know are missing from this discussion. For one, I would like to relate these definitions and dimensions to existing taxonomies, principally Jakubovic et al.’s Technical Dimensions of Programming Systems [].

But I hope this is a good start. I hope that with further contributions from the community, we can extend and refine these lists. I hope that it is useful to fellow researchers to think through the lenses of these definitions and dimensions.

References

  1. Abramov, D. (2015). Live React: Hot Reloading with Time Travel.
  2. Burckhardt, S., Fahndrich, M., de Halleux, P., McDirmid, S., Moskal, M., Tillmann, N., & Kato, J. (2013). It’s alive! continuous feedback in UI programming. ACM SIGPLAN Notices, 48(6), 95–104. https://doi.org/10.1145/2499370.2462170
  3. Burnett, M. M., Atwood, J. W., & Welch, Z. T. (n.d.). Implementing level 4 liveness in declarative visual programming languages. Proceedings. 1998 IEEE Symposium on Visual Languages (Cat. No.98TB100254). 1998 IEEE Symposium on Visual Languages. https://doi.org/10.1109/vl.1998.706155
  4. Edwards, J. (2004). Example centric programming. ACM SIGPLAN Notices, 39(12), 84–91. https://doi.org/10.1145/1052883.1052894
  5. Goldberg, A., & Robson, D. (1983). Smalltalk-80: the language and its implementation. Addison-Wesley Longman Publishing Co., Inc.
  6. Granger, C. (2022). Light Table. http://lighttable.com/
  7. Hancock, C. M. (2003). Real-Time Programming and the Big Ideas of Computational Literacy [Phdthesis]. Massachusetts Institute of Technology.
  8. Hempel, B., Lubin, J., & Chugh, R. (2019, October 17). Sketch-n-Sketch. Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. UIST ’19: The 32nd Annual ACM Symposium on User Interface Software and Technology. https://doi.org/10.1145/3332165.3347925
  9. Henderson, P., & Weiser, M. (1985). Continuous execution: the VisiProg environment. Proceedings of the 8th International Conference on Software Engineering, 68–74.
  10. Hex Technologies, Inc. (n.d.). Hex - Do More with Data, Together. https://hex.tech. https://hex.tech
  11. Horowitz, J., & Heer, J. (2023, October 29). Engraft: An API for Live, Rich, and Composable Programming. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. UIST ’23: The 36th Annual ACM Symposium on User Interface Software and Technology. https://doi.org/10.1145/3586183.3606733
  12. Horowitz, J., & Heer, J. (2023). Live, Rich, and Composable: Qualities for Programming Beyond Static Text. Plateau Workshop. https://doi.org/10.1184/R1/22277338.V1
  13. Imai, T., Masuhara, H., & Aotani, T. (2015, October 25). Shiranui: a live programming with support for unit testing. Companion Proceedings of the 2015 ACM SIGPLAN International Conference on Systems, Programming, Languages and Applications: Software for Humanity. SPLASH ’15: Conference on Systems, Programming, Languages, and Applications: Software for Humanity. https://doi.org/10.1145/2814189.2817268
  14. Jakubovic, J., Edwards, J., & Petricek, T. (2023). Technical Dimensions of Programming Systems. The Art, Science, and Engineering of Programming, 7(3). https://doi.org/10.22152/programming-journal.org/2023/7/13
  15. Jiang, P., Sun, F., & Xia, H. (2023, April 19). Log-it: Supporting Programming with Interactive, Contextual, Structured, and Visual Logs. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. CHI ’23: CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3581403
  16. Kasibatla, S., & Warth, A. (2017). Seymour: Live Programming for the Classroom. https://harc.github.io/seymour-live2017/
  17. Kluyver, T., Ragan-Kelley, B., Pérez, F., Granger, B. E., Bussonnier, M., Frederic, J., Kelley, K., Hamrick, J. B., Grout, J., Corlay, S., Ivanov, P., Avila, D., Abdalla, S., Willing, C., & Team, J. D. (2016). Jupyter Notebooks - a publishing format for reproducible computational workflows. International Conference on Electronic Publishing.
  18. Lerner, S. (2020, April 21). Projection Boxes: On-the-fly Reconfigurable Visualization for Live Programming. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. CHI ’20: CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3313831.3376494
  19. Lotem, E., & Chuchem, Y. (2022). Lamdu. https://www.lamdu.org/
  20. McDirmid, S. (2013, October 29). Usable live programming. Proceedings of the 2013 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming & Software. SPLASH ’13: Conference on Systems, Programming, and Applications: Software for Humanity. https://doi.org/10.1145/2509578.2509585
  21. McLean, A. (2014, September 3). Making programming languages to dance to. Proceedings of the 2nd ACM SIGPLAN International Workshop on Functional Art, Music, Modeling & Design. ICFP’14: ACM SIGPLAN International Conference on Functional Programming. https://doi.org/10.1145/2633638.2633647
  22. Nardi, B. A. (1993). A small matter of programming. MIT Press.
  23. Observable Inc. (2022). Observable - Explore, analyze, and explain data. As a team. https://observablehq.com/
  24. Omar, C., Voysey, I., Chugh, R., & Hammer, M. A. (2019). Live functional programming with typed holes. Proceedings of the ACM on Programming Languages, 3(POPL), 1–32. https://doi.org/10.1145/3290327
  25. Rauch, D., Rein, P., Ramson, S., Lincke, J., & Hirschfeld, R. (2019). Babylonian-style Programming: Design and Implementation of an Integration of Live Examples into General-purpose Source Code. The Art, Science, and Engineering of Programming, 3(3). https://doi.org/10.22152/programming-journal.org/2019/3/9
  26. Rein, P., Ramson, S., Lincke, J., Hirschfeld, R., & Pape, T. (2018). Exploratory and Live, Programming and Coding. The Art, Science, and Engineering of Programming, 3(1). https://doi.org/10.22152/programming-journal.org/2019/3/1
  27. Shen, P. (2022). welcome! – natto. https://natto.dev/
  28. van der Storm, T., & Hermans, F. (2016). Live Literals. https://homepages.cwi.nl/~storm/livelit/livelit.html
  29. Sulír, M., Chodarev, S., & Nosáľ, M. (2023, June 14). Outside the Sandbox: A Study of Input/Output Methods in Java. Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering. EASE ’23: The International Conference on Evaluation and Assessment in Software Engineering. https://doi.org/10.1145/3593434.3593501
  30. Tanimoto, S. L. (1990). VIVA: A visual language for image processing. Journal of Visual Languages and Computing, 1(2), 127–139.
  31. Tanimoto, S. L. (2013). A Perspective on the Evolution of Live Programming. Proceedings of the 1st International Workshop on Live Programming, 31–34. https://doi.org/10.1109/LIVE.2013.6617346
  32. Victor, B. (2012). Inventing on Principle. https://vimeo.com/36579366
  33. Victor, B. (2012). Learnable Programming. http://worrydream.com/LearnableProgramming/
  34. Workshop on Live Programming. (2024). Live 2024 | Workshop on Live Programming. https://liveprog.org/
  35. diSessa, A. A., & Abelson, H. (1986). Boxer: a reconstructible computational medium. Communications of the ACM, 29(9), 859–868. https://doi.org/10.1145/6592.6595