The fast-moving, dynamic world of artificial intelligence (AI) stands in stark contrast to the slow-moving, conservative world of academia.¹¹ This is particularly clear in the world of AI ethics, where in addition to the industry-academia contrast we also have the meeting of very different academic disciplines, including computer science, philosophy, ethics, and social sciences. The traditions, norms, and values of these disciplines are often at odds with one another, making interdisciplinarity challenging. Take, for example, preprinting, the practice of quickly disseminating research before potentially—but not necessarily—seeking publication in traditional academic journals.^a Interdisciplinary conflicts appear when, for example, researchers from a computer science background, where rapid publication of preprints on servers such as arXiv is the norm,² meet researchers from the social sciences and humanities, where this is less common.^1,30

AI ethics emerged as an active research field over the past decade, but it has a long history and a strong overlap with other fields of applied ethics, such as computer ethics.²⁵ AI as a technology is not new either,²³ nor are most of the issues with which it grapples, such as privacy, bias, and programmers’ responsibilities. The general scientific community has been portrayed as a “republic of science” by Michael Polanyi;²¹ some, though, have noted that this republic consists of partly overlapping domains—“fiefdoms of expertise.”¹⁰ The norms of disciplines vary greatly,^1,30 and a struggle to identify the community norms around preprinting in AI ethics is currently developing, as different cultures clash in this border fiefdom. 

In this article I focus on the practice of preprinting, including the role and status of preprinting compared with traditional journal publication and other forms of dissemination, such as blogging and mainstream media contributions. Preprinting has many advantages, such as rapid dissemination of and feedback on ideas, openness, the bypassing of quarrelsome gatekeepers in the traditional publishing system, and the opportunity to publish novel ideas in new formats.^4,7,27,35 In particular, it is often hailed as a beneficial practice for early-career researchers in need of quickly establishing a track record.^2,19

But before we accept the idea that preprinting has many benefits and should be adopted by all, certain challenges should be noted. These include the dangers of bad science, misdirection of policy and practice, disinformation, and information overload.^12,14,15,36 One important issue concerns how the growing prominence and legitimacy of preprints could more easily allow actors to use preprints in strategic efforts to influence and shape various ideals, norms, or systems. This could, for example, happen through the spread of fake science and manufactured scientific support for conspiracy theories. In addition, the seeming “democratization” of authority through preprinting can be deceptive,¹² as I will argue that departing from traditional norms of scientific publishing with peer review might in fact be most beneficial for those who already wield power.

I begin with a presentation of the benefits and challenges of preprinting based on a review of the literature, including some thoughts on how these apply to AI ethics specifically. Then I suggest some broad best practices around preprinting with the aim of sparking discussion within the community.

Why Preprint?

There are many good reasons to preprint, and a quick review of the literature on preprinting in different disciplines shows clear similarities. In the following, I highlight some of the main benefits highlighted in the literature, followed by a brief discussion of how AI ethics researchers specifically might benefit from preprinting. The benefits are listed according to how often and how strongly they are reflected in the literature, with the most strongly emphasized ones discussed first. Here, I largely exclude the potential negative effects and costs related to preprinting, as these will be discussed in detail in the next section.

The Challenges of Preprinting in AI Ethics

Preprinting provides a wide range of potential benefits, but what does the flip side of the preprinting coin look like? Here, I summarize some of the challenges emphasized in the literature, while adding a couple of potential objections not particularly well covered in extant research.

Toward Best-Practice Preprinting in AI Ethics

The preceding sections have detailed a range of benefits and challenges related to preprinting, summarized in Table 1.

At a time when disinformation,³⁷ polarization,³¹ and a lack of trust in the scientific community in large groups¹⁸ are prevalent, it is important that the scientific community considers its role and responsibilities and actively engages in the shaping of our academic culture and practices. This is no less relevant for the field of AI ethics, where we deal with technologies that are rapidly implemented, with major implications for individuals, businesses, and governments. As mentioned, there is much hype concerning AI, and a high demand for AI ethics expertise. There is a great temptation to jump into the field, and to exploit opportunities that bring recognition from the community.²⁴ When things move fast, the traditional world of scientific publishing feels exceedingly slow. As a journal editor myself, I see firsthand how researchers might struggle as their studies of brand-new technologies are stuck in review and revision processes that mean their results will be outdated by the time they are finally published. However, I would argue that any manuscript that would actually be outdated because, for example, a software solution has been updated, is not appropriate for publication anyway, as it should contain analyses of the solution and considerations related to what could be learned and developed based on the results of the research. These should be valid and useful regardless of how rapidly the technology develops. Manuscripts that do not have these more timeless elements are perhaps better suited for other forms of dissemination, such as Medium or Substack posts. Or preprints?

One key difference between preprints and these other forms of publishing, however, is that preprints are more likely to stay available, accessible, and usable by other researchers than alternative formats. Preprints get a persistent identifier such as a DOI, and are indexed in scientific search engines. Another user benefit is that preprints cannot be changed without providing a record of changes. All of this makes the preprint a more solid form of dissemination likely to be used and referenced by other researchers, potentially helping promote the broader review and extensive discussion that I’ve discussed here.

In the following, I propose a tentative set of guidelines for preprinting, intended as a basis for community discussion and debate.

Conclusion

A diverse field such as AI ethics is bound to have a number of fault lines separating various groups with different backgrounds. This, though, does not necessarily mean that the various groups are in conflict, or that they cannot learn from each other and cooperate.¹⁰ Bridging the fault lines, rather than shouting across the chasm without any real attempt to understand or constructively learn from and engage with each other, should be preferable. Fostering such cooperation, however, entails actively engaging in and developing community guidelines, coordination, and a joint understanding of these different backgrounds and perspectives.³⁸ This discussion of preprinting practices is an attempt to chip away at one small part of this challenge.

Preprinting can be highly beneficial—especially in AI ethics—but would benefit from adherence to certain guidelines to avoid some of the pitfalls. Even when these—or other, better—guidelines are followed, additional questions remain.

One important issue is how we can help each other navigate and leverage a rising tide of research of increasingly unknown merit. There is no universally accepted definition of what constitutes scientific knowledge,²² but it is commonly identified through its recognition and appropriation by the scientific community.^8,11 Developing venues for community debate and deliberation will be important to effectively make use of and curate preprints. It might also make sense to find, shape, and develop more interdisciplinary preprint servers for AI-ethics-related work, as existing servers are poorly suited for this purpose. Furthermore, these debates must include other stakeholders, including the media and citizens. Navigating research in the field is difficult enough for those in it, so making sure we help others understand what research can be trusted—for what purposes and how much—will be crucial as preprinting practices develop.

If you regularly work with open-source code or produce software for a large organization, you are already familiar with many of the challenges posed by collaborative programming at scale. Some of the most vexing of these tend to surface as a consequence of the many independent alterations inevitably made to code, which, unsurprisingly, can lead to updates that do not synchronize.

Difficult merges are nothing new, of course, but the scale of the problem has gotten much worse. This is what led a group of researchers at Microsoft Research (MSR) to take on the task of complicated merges as a grand program-repair challenge—one they believed might be addressed at least in part by machine learning (ML).

To understand the thinking that led to this effort and then follow where that led, ACM Queue asked Erik Meijer and Terry Coatta to speak with three of the leading figures in the MSR research effort, called DeepMerge.^a Meijer was long a member of MSR, but at the time of this discussion was director of engineering at Meta. Coatta is the chief technology officer of Marine Learning Systems. Shuvendu Lahiri and Christian Bird, two of the researchers who helped drive this effort, represent MSR, as does Alexey Svyatkovskiy, who was with Microsoft DevDiv (Development Division) at the time.

Terry Coatta: What inspired you to focus on merge conflicts in the first place? And what made you think you’d be able to gain some advantage by applying AI techniques?

Christian Bird: Back in the winter of 2020, some of us started talking about ways in which we might be able to use machine learning to improve the state of software engineering. We certainly thought the time was right to jump into an effort along these lines in hopes of gaining enough competency to launch into a related research program.

We tried to identify problems other researchers weren’t already addressing, meaning that something like code completion—which people had been working on for quite some time—was soon dismissed. Instead, we turned to problems where developers didn’t already have much help.

Shuvendu [Lahiri] has a long history of looking at program merge from a symbolic perspective, whereas my own focus has had more to do with understanding the changes that occur in the course of program merges. As we were talking about this, it dawned on us that almost no one seemed to be working on program merge. And yet, that’s a problem where we, as developers, still have little to rely upon. For the most part, we just look at the diffs between different generations of code to see if we can figure out exactly what’s going on. But there just isn’t much current tooling to help beyond that, which can prove to be problematic whenever there’s a merge conflict to resolve.

So, we figured, “OK, let’s look at how some deep-learning models might be applied to this problem. As we go along, we’ll probably also identify some other things we can do to build on that.”

Shuvendu Lahiri: Yes, as Chris suggests, I’ve been thinking about the issues here for quite some time. Moreover, we found program merge to be appealing, since it’s a collaboration problem. That is, even if two skilled developers make correct changes, the merge itself may introduce a bug.

We were also keenly aware of the sort of pain program-merge problems can cause, having known about it through studies within Microsoft.^b I thought maybe there was something we could do to provide relief. It also turns out that AI was just coming along at that point, and Alexey [Svyatkovskiy] had already developed a couple of powerful models that looked quite promising for code completion. What’s more, information about merge conflicts was just starting to become more readily available from the Git commit history, so that too looked like it might serve as a good up-front source of clean data.

Erik Meijer: I like the fact that you focused on merge conflict, since, when it comes to this, I don’t think source control solves any of the real problems. Maybe I’m being a little extreme here, but even if source control lets you know where you have a merge conflict, it won’t help you when it comes to actually resolving the conflict. In fact, I’m baffled as to why this problem wasn’t solved in an intelligent manner a long time ago. Are people just not listening to complaints from actual users?

Lahiri: Basically, I think it comes down to academicians consistently resorting to symbolic methods to solve this problem. Whereas people in the real world have looked at this as just another aspect of programming, practitioners have been more inclined to approach it as a social process—that is, as a problem best addressed by encouraging co-workers to figure out solutions together. Personally, I’ve always seen merge conflicts as more of a tooling challenge.

Alexey Svyatkovskiy: For me, this just looked like an exciting software engineering problem to address with machine learning. I’ve spent years working on code completion, but this effort looked like something that would take that up to the next level of complexity, since it necessarily would involve aligning multiple sequences somehow and then complementing that with an understanding of where things ought to be inserted, deleted, or swapped. And, of course, there were also those special cases where the developer would be able to add new tokens during the merge.

This took us on a journey where we ended up addressing program merge down at the line-and-token level. I found this fascinating, since a lot of people don’t have any idea about how merge actually works and so, by extension, don’t have a clear understanding about what leads to merge conflicts. Taking on this problem also seemed important in that, while merge conflicts are far less common than software bugs, they require considerably more time to resolve and can end up causing more pain.

Coatta: How did you initially attack the problem?

Lahiri: We realized that repositories (both open-source ones on GitHub and internal ones at Microsoft) contain data on merge conflicts and their resolution across several different programming languages. What’s more, Alexey had recently created a neural model that had been pre-trained on a large subset of the code in those repositories. Our initial thought was to fine-tune that model with information about merge conflicts and their resolution. And we figured that should be simple enough to be treated as an intern project. So, that’s how we scoped it initially. We figured: Just get the data, train a model, and deploy. What we failed to grasp was that, while there was an ample amount of program merge data to be mined, coming to an understanding of what the intent behind all those merges had been was at least as important as the data itself. In fact, it proved to be absolutely critical.

A considerable amount of time and effort was required to understand and interpret the data. This included some significant technical challenges—for example, how best to align these programs. And how can you communicate to a neural model that these are not independent programs but instead represent some number of changes to an underlying program? The notion of how to go from program text to a program edit became quite crucial and, in fact, required considerable research. Ultimately, we concluded that if you manage to combine your existing merge tools correctly, add just the right amount of granularity—which for us proved to be tokens—and then employ neural modeling, you can often succeed in reducing the complexity. But it took us quite a bit of time to work that out.

Of course, we also underestimated the importance of user experience. How exactly would a user end up employing such a tool—one that’s AI-based, that is? And what would be the right time to surface that aspect of the tool?

Coatta: I find it fascinating that it proved to be so difficult to scope this project correctly. Can you dig a bit deeper into that?

Svyatkovskiy: To me, at least, as we were analyzing the different types of merges, it soon became clear that there are varying levels of complexity. Sometimes we’d find ourselves looking at two simple merge resolution strategies, where it essentially came down to “Take ours or take theirs.” Such cases are trivial to analyze, of course, and developers don’t require much AI assistance when it comes to resolving these conflicts.

But then there’s another class of merge, where a new interleaving line is introduced that involves more than just concatenation. There could also be token-level interleaving, where lines in the code have been broken and new tokens introduced in between. This leads to the notoriously complex case where a switch to token-level granularity proves to be crucial. Beyond that, there’s a whole other class of merges where you find somebody has introduced some new tokens.

Meijer: How do you go about defining what you consider to be a correct merge? Doesn’t that require you to make your own value judgments in some sense?

Lahiri: Well, I’ll just say we had a very semantic way of looking at merges. Essentially: “Forget about the syntax; instead, what does it mean for the merge to be correct?” In effect, this amounts to: If something was changed in one program, then that ought to be reflected in the merge. And, if that also changes a behavior, then that too ought to be included in the merge. But no other changes or altered behaviors should be introduced.

We then found, however, that we could get tangled up whenever we ran into one of these “take my changes or take yours” merges. We also found that one set of changes would often just be dropped—like a branch being deprecated, as Alexey once pointed out. This is how we discovered that our initial notion of correctness didn’t always hold. That’s also how we came to realize we shouldn’t adhere to overly semantic notions.

So, we decided just to do our best to curate the training data by removing any indications of “take your changes or take mine” wherever possible. Then we looked at those places where both changes had been incorporated to some extent and said, “OK, so this now is our ground truth—our notion of what’s correct.” But notice that this is empirical correctness as opposed to semantic correctness—which is to say, we had to scale back from our original high ambitions for semantic correctness.

Svyatkovskiy: We now treat user resolutions retrieved from the GitHub commit histories as our ground truth. But yes, naturally, there are all kinds of ways to define a “correct” merge. For example, it’s possible to reorder the statements in a structured merge and yet still end up with a functionally equivalent resolution. Yet, that would be deemed as “incorrect,” so, there’s clearly room for retooling our definition of correctness. In this instance, however, we chose to take a data-driven approach that treats user resolutions from the GitHub commit histories as our ground truth.

Bird: Right. And let me also say that, from the beginning of this project, we decided to approach it as something that might yield a product. With that in mind, we realized it needed to be, perhaps not language-agnostic, but at least something that could be readily adapted to multiple languages—and definitely not something that would require some bespoke analysis framework for each language. That essentially guided our choice not to employ richer or more complex code representations.

Meijer: I’ve also run into situations like this where it looked really tempting to use an AST [abstract syntax tree] or something of the sort, since that would provide all the structure that was required. But then, as you go deeper into that sort of project, you find yourself wondering whether it’s actually a good idea to feed semantically rich programs into models and start thinking it might be better just to send strings instead.

Coatta: To dive a bit deeper into that, you had a practical motivation to work with a token-based approach. But what does your intuition tell you about how models behave when you do that? If you feed a model a rich, structured information set, is that model then actually more likely to make better decisions? Or is that perhaps a false assumption?

Svyatkovskiy: The model ought to be able to make better decisions, I think.

Meijer: All right, but can I challenge that a bit? The model can handle the syntax and semantic analysis internally, which suggests this work may not need to be done ahead of time, since machines don’t look at code the same way humans do. I don’t know why the model couldn’t just build its own internal representation and then let a type-checker come along at the end of the process.

Bird: I think it’s hazardous to speculate about what models may or may not be capable of. I mean, that’s a super-nuanced question in that it depends on how the model has been trained and what the architecture is like—along with any number of other things. I’m constantly surprised to learn what models are now capable of doing. And, in this case, we’re talking about the state of the world back in 2020—even as I now find it hard to remember what the state of the world looked like six months prior to the GPT models becoming widespread.

Lahiri: For one thing, we were using pretrained models to handle classification and generation, which then left us with quite a bit of work to do in terms of representing the resulting edits at the AST level before tuning for performance. That certainly proved to be a complex problem—and one that came along with some added computational costs. Also, as I remember it, the models we were using at the time had been trained as text representations of code—meaning we then needed to train them on a lot more AST-level representations to achieve better performance. I’m sure it would be fascinating to go back to revisit some of the decisions we made back in 2020.

Meijer: What model are you using now?

Svyatkovskiy: For this iteration, we’re employing a token-level merge along with a transformer-based classifier. We’ve also been looking at using a prompt-driven approach based on GPT-4.

Meijer: I love that this is now something where you can take advantage of demonstrated preferences to resolve merge conflicts instead of being left to rely solely on your own opinions.

Lahiri: Another way of looking at this that came up during our user studies was that, even after a merge has been produced, someone might want to know why the merge was accomplished in that particular way and may even want to see some evidence of what the reasoning was there. Well, that’s a thorny issue.

But one of the nice things about these large foundational models is that they’re able to produce textual descriptions of what they’ve done. Still, we haven’t explored this capability in depth yet, since we don’t actually have the means available to us now to evaluate the veracity of these descriptions. That will have to wait until some user studies supply us with more data. Still, I think there are some fascinating possibilities here that ultimately should enable us to reduce some of the friction that seems to surface whenever these sorts of AI power tools are used to accomplish certain critical tasks.

Coatta: You’ve mentioned that you had access to a vast amount of training data, but you’ve also suggested some of that data contained surprises—which is to say it proved to be both a blessing and a curse. Can you go into that a bit more?

Lahiri: Yes, we were surprised to find that a large percentage of the merges—perhaps 70%—had the attribute of choosing just one side of the edit and then dropping the other. In some of those cases, it seemed one edit was superseding the others, but it can be hard to be sure whenever the syntax changes a little. In many instances, there were genuine edits that had been dropped on the floor. It was unclear whether that was due to a tooling problem or a social issue—that is, in some cases, perhaps some senior developer’s changes had superseded those that had been made by a junior developer. Another hypothesis was that, instead of a single merge, some people may have chosen to merge in multiple commits.

This sort of thing was so common that it accounted for a significant portion of the data, leaving us uncertain at first as to whether we should throw out these instances, ignore them, or somehow make an effort to account for them. That certainly proved to be one of the bigger surprises we encountered.

Another surprise was that we discovered instances where some new tokens had been introduced that were irrelevant to the merge. It was unclear at first whether those were due to a genuine conflict in the merge or just because somebody had decided to add a pretty print statement while doing the refactoring. That proved to be another thorny issue for us.

Coatta: How did you resolve that? It sounds like you had some datasets you didn’t quite know how to interpret. So, how did you decide what should be classified as correct merges or treated as incorrect ones?

Lahiri: We curated a dataset that did not include the “trivial” merge resolutions, with the goal of assisting users with the more complex cases first. As Alexey mentioned, users may not need tooling support for those resolutions that only require dropping one of the two edits.

Svyatkovskiy: And then, from user studies, we learned that some users still wanted to be able to use the approach that had been dismissed. We solved that problem by providing a “B option” that people could get to by using a drop-down menu.

Lahiri: Which is to say we addressed the problem by way of user experience rather than by changing the model.

The other data problem we encountered had to do with new tokens that would occasionally appear. Upon closer examination, we found these tokens were typically related to existing changes. By going down to token-level merges, we were able to make many of these aspects go away. Ultimately, we built a model that excluded that part of the dataset where new tokens were introduced.

Meijer: In terms of how you went about your work, I understand one of the tools you particularly relied on was Tree-sitter [a parser-generator tool used to build syntax trees]. Can you tell us a bit about the role it played in your overall development process?

Bird: We were immediately attracted to Tree-sitter because it lets you parse just about anything you can imagine right off the shelf. And it provides a consistent API, unlike most other parsers out there that each come with their own API and work only with one language or another.

For all that, I was surprised to learn that Tree-sitter doesn’t provide a tokenizing API. As an example of why that proved to be an issue for us, we wanted to try Python, which basically lets everyone handle their own tokenizing. But, of course, Tree-sitter didn’t help there. We resorted to a Python tokenizing library.

Beyond that relatively small complaint, Tree-sitter is great in terms of letting you apply an algorithm to one language and then quickly scale that up for many other languages. In fact, between that capability and the Python tokenizing library, which made it possible for us to handle multiple languages, we were able to try out things with other languages without needing to invest a lot of upfront effort. Of course, there’s still the matter of obtaining all the data required to train the model, and that’s always a challenge. At least we didn’t need to write our own parsers, and the consistent interfaces have proved to be incredibly beneficial.

Meijer: Once you finally managed to get all this deployed, what turned out to be your biggest surprise?

Bird: There were so many surprises. One I particularly remember came up when we were trying to figure out how people would even want to view merge conflicts and diffs. At first, some of us thought they’d want to focus only on the conflict itself—that is, with a view that let them see both their side and the other side. It turns out you also need to be able to see the base to understand the different implications between an existing branch in the base and your branch.

So, we ran a Twitter survey to get a sense of how much of that people thought we should show. How much of that did they even want to see? For example, as I recall, most people couldn’t even handle the idea of a three-way diff, or at least weren’t expecting to see anything quite like that. That really blew my mind, since I don’t know how anyone could possibly expect to deterministically resolve a conflict if they don’t know exactly what they’re facing.

Some other issues also came up that UI people probably would expect, but I nevertheless was incredibly surprised. That proved to be a big challenge, since we’d been thinking throughout this whole process that we’d just get around to the UI whenever we got around to it. And yes, as this suggests, our tendency initially was just to focus on making sure the underlying algorithm worked. But then we found to our surprise just how tough it could be to find the right UI to associate with that.

Coatta: From what you say, it seems you weren’t surprised about the need for a good user experience, but it did surprise you to learn what’s considered to be a good experience. What are your thoughts now on what constitutes a good user experience for merge?

Bird: I’m not entirely clear on that even now, but I’ll be happy to share some of the things we learned about this early on. As we’ve already discussed, people definitely want to see both sides of a merge. Beyond that, we discovered that they want the ability to study the provenance of each part of the merge because they want to know where each token came from.

So, we wrote some code to track each token all the way back to whichever side it came from.

There also were tokens that had come in from both sides. To make it clear where a token had originated, we wrestled with whether we should add colors as an indicator of that. How might that also be used to indicate whether a token happens to come from both sides or simply is new?

In addition, we knew it was important that the interface didn’t just ask you to click “yes” or “no” in response to a suggested change, since it’s rare to find any merge that’s going to be 100% correct. Which is to say developers are going to want to be able to modify the code and will only end up being frustrated by any interface that denies them that opportunity.

The real challenge is that there are lots of moving pieces in any given merge. Accordingly, there are many possible views, and yet you still want to keep things simple enough to avoid overwhelming the user. That’s a real challenge. For example, we know that if we offer three suggestions for a merge rather than just one, the chance of the best one being selected is much higher. But that also adds complexity, so we ultimately decided to go with suggesting the most likely option, even though that might sometimes lead to less-optimal results.

There are some other user-experience considerations worth noting. For example, if you are working on some particular Visual Studio feature, you’re going to want to produce something that feels intuitive to someone who has been using that same tool. Suffice it to say, there’s plenty to think about in this respect. Basically, once you finally get your model to work, you might not even be halfway home, since that’s just how critical—and time-consuming—the user-experience aspect of this work can be.

Coatta: We all know that creating a tool for internal purposes is one thing, while turning that into a product is something else altogether. It seems you took that journey here, so what were some of the bigger surprises you encountered along the way?

Lahiri: Actually, we don’t have a product yet that implements the DeepMerge algorithm and aren’t at liberty to talk about how that might be used in future products. Still, as we’ve just discussed, I can say most of the unusual challenges we encountered were related to various aspects of the user experience. So, we got much deeper into that here than we normally would.

One of the biggest challenges had to do with determining how much information needed to be surfaced to convince the user that what was just done was even possible—never mind appropriate. Suddenly, you’ve just introduced some new tokens over here, along with a new parsable tree over there. I think that can really throw some users off.

Bird: What did all this look like from the DevDiv perspective, Alexey? You deal with customers all the time. What proved to be the biggest challenges there?

Svyatkovskiy: Some of the most crucial design decisions came down to choosing between client-side or server-side implementation. Our chief concern had to do with the new merge algorithm we were talking about earlier. Customer feedback obtained from user studies and early adopters proved to be particularly crucial in terms of finding ways to smooth things out. Certainly, that helped in terms of identifying areas where improvements were called for, such as achieving better symmetries between what happens when you merge A to B versus when you merge B to A.

Lahiri: I’d like to add a couple of points. One is that some developers would prefer to handle these merges themselves. They just don’t see the value of tooling when it’s used to deal with something they could do themselves. But that just resulted in some inertia, which is always hard to overcome without a lot of usage. Still, from our empirical study we learned that, even when merges were not identical to the ground truth, users would accept them if they proved to be semantically equivalent. Ultimately, that proved to be a pleasant surprise, since it revealed we had previously been undercounting our wins according to our success metrics.

Coatta: Did anything else interesting surface along the way?

Bird: At one point, one of our interns did a user study that pulled merge conflicts and their resolutions out of Microsoft’s historical repositories so they could then be compared with the resolutions our tool would have applied. As you might imagine, quite a few differences surfaced. To understand where our tool may have gone wrong, we went back to consult with those people who had been involved in the original merges and showed them a comparison between what they had done and how the tool had addressed the same merge conflicts.

We specifically focused on those conflicts that had been resolved over the preceding three months on the premise that people might still recall the reasoning behind those decisions. We learned a ton by going through that particular exercise. One of the lessons was that we had probably undercounted how often we were getting things right, since some of these developers would say things like, “Well, this may not exactly match the merge I did, but I would have accepted it anyway.”

The other major benefit of that study was the insight it provided into what the user experience for our tool should be. This all proved to be a major revelation for me, since it was the first time I’d been involved in a user study that was approached in quite this way—where developers were pulled in and presented with code they’d actually worked on.

Which is just to say this wasn’t at all like one of those lab studies where people are presented with a toy problem. In this case, we were pulling in real-world merge conflicts and then talking with the developers who had worked to resolve them. We learned so much from taking this approach that I’d recommend other researchers consider doing their own studies in much the same way.

Svyatkovskiy: Another thing that came out of these user studies was the importance of explainability. With large language models, for example, we can drill into specific three-way diffs and their proposed resolutions and then ask for summaries of certain decisions, which can be helpful when it comes to building confidence in some of these AI suggestions.

Also, as Chris indicated, even when users chose not to go with the solution offered by DeepMerge, the reasoning behind the suggestion still seemed to inform their own thinking and often led to an improved merge resolution.

Coatta: What’s next?

Lahiri: There’s room for more prompt engineering in terms of determining what goes into the model’s input. We also need to address correlated conflicts. So far, we’ve addressed each conflict as if it was independent, but you can have multiple conflicts in a file that all relate to a certain dependency. Some users have told us that, once a resolution has been worked out for one of those conflicts, they’d like to see something similar applied to each of the other conflicts that exhibit a similar pattern, which certainly seems quite reasonable.

Also, while the types of conflicts we’ve addressed so far are highly syntactic in nature, there is, in fact, a whole spectrum of merge conflicts. There’s still much to address, including silent merges that include semantic conflicts, which are much harder to deal with than anything we’ve handled so far. Still, I’d say it feels like we’re off to a reasonably good start.

From

We claim two things about our profession. Computer science studies information processes, natural and artificial. Computer science is a master of abstraction. To reconcile the two, we say that abstraction is the key that unlocks the complexity of designing and managing information processes.

Where did we get the idea that our field is a master of abstractions? This idea is cosmic background radiation left over from the beginning bangs of our field. For its first four decades, computer science struggled under often blistering criticisms by scientists that the new field was not a science. Science is, they said, a quest to understand nature—and computers are not natural objects. Our field’s pioneers maintained that computers were a major new phenomenon that called for a science to understand them. The most obvious benefit of our field was software that controls and manages very complex systems. This benefit accrued from organizing software into hierarchies of modules, each responsible for a small set of operations on a particular type of digital object. Abstraction became a shorthand for that design principle.

Approximately 25 years ago, the weight of opinion suddenly flipped. Computer science was welcomed at the table of science. The tipping point came because many scientists were collaborating with computer scientists to understand information processes in their fields. Many computer scientists claimed we had earned our seat because of our expertise with abstractions. In fact, we won it because computation became a new way of doing science, and many fields of science discovered they were studying naturally occurring information processes.

In what follows, I argue that abstraction is a by-product of our central purpose, understanding information processes. Our core abstraction is information process, not “abstraction.” Every field of science has a core abstraction—the focus of their concerns. In all but a few cases, the core abstractions in science defy precise definitions and scientists in the same field disagree on their meanings. Two lessons follow. First, computing is not unique in believing it is a master of abstraction. Indeed, this claim never sat well with practitioners in other fields. Math, physics, chemistry, astronomy, biology, linguistics, economics, psychology—they all claim to be masters of abstractions. The second lesson is that all fields have made remarkable advances in technology without clear definition of their core abstraction. They all designed simulations and models to harness the concrete forces behind their abstractions. The profound importance of these lessons was recognized with two 2024 Nobel prizes awarded to computer scientists for protein folding and machine learning.

What Is Abstraction?

Abstraction is a verb: to abstract is to identify the basic principles and laws of a process so that it can be studied without regard to physical implementation; the abstraction can then guide many implementations. Abstraction is also a noun: An abstraction is a mental construct that unifies a set of objects. Objects of an abstraction have their own logic of relations with each other that does not require knowledge of lower-level details. (In computer science, we call this information hiding.)

Abstractions are a power of language. In his book Sapiens, Yuval Noel Harari discusses how, over the millennia, human beings created stories (“fictions”) that united them into communities and gave them causes they were willing to fight for.⁴ These fictions were abstractions that often endured well beyond their invention. The U.S. constitution, for instance, applies to all its states and has guided billions of people for more than 200 years.

The ability of language to let us create new ideas and coordinate around them also empowers language constructs to refer to themselves. After all, we have numerous ideas about our ideas. We build endlessly complex structures of ideas. We can imagine things that do not exist, such as unicorns, or unrealized futures that we can pursue. Self-reference also generates paradoxes. A famous paradox asks: “Does the set of all those sets that do not contain themselves contain itself?” Self-reference is both a blessing and a curse.

Ways to avoid paradoxes are to stack up abstractions in hierarchies or connect them in networks. An abstraction can be composed of lower-level abstractions but cannot refer to higher level abstractions. In chemistry, for example, amino acids are composed of atoms, but do not depend on proteins arranging the acids in particular sequences. In computing, operating systems are considered layers of software that manage different abstractions such as processes, virtual memories, and files; each depends on lower levels, but not higher levels. Consider three examples illustrating how different fields use their abstractions.

Differing Interpretations of the Same Abstractions

It is no surprise that different people have different interpretations about abstractions and thus get into arguments over them. After all, abstractions are mental constructs learned individually. Few abstractions have clear logical definitions as in mathematics or in object-oriented languages. Here are some additional examples showing how different fields approach differences of interpretation of their core abstractions.

Conclusion

The accompanying table summarizes the examples above. The “criteria” column indicates whether a field has a consensus on criteria for their core abstraction. The “explanatory” column indicates whether a field’s existing definitions adequately explain all the observable instances of their core abstraction. The “utility” column indicates whether they are concerned with finding applications of technologies enabled by their core abstraction.

Thus, it seems that the core abstractions of many fields are imprecise and, with only a few exceptions, the fields have no consensus on criteria to determine if an observation fits their abstraction. How do they manage a successful science without a clear definition of their core abstraction? The answer is that in practice they design systems and processes based on validated hypotheses. The varying interpretations are a problem only to the extent that disagreements generate misunderstanding and confusion.

A good way to bring out the differences of interpretation is to ask people how they assess if a phenomenon before them is an instance of their core abstraction. Thus you could say “Life is an assessment,” “intelligence is an assessment,” and so on. When you put it this way, you invite a conversation about the grounding that supports the assessment. For example, a biologist would ground an assessment that a new organism is alive by showing that enough of the seven criteria are satisfied. In other fields the request for assessment quickly brings out differences of interpretation. In business, for example, where there is no consensus on the indicators of innovation, a person’s assessments reveal which of the competing core abstractions they accept. That, in turn, opens the door for conversations about the value of each abstraction.

There is a big controversy over whether technology is dragging us into abstract worlds with fewer close relationships, fear of intimacy, and interaction limited to exchanges across computer screens. This is a particular problem for young people.³ Smartphones are intended to improve communication and yet users feel more isolated, unseen, unappreciated. Something is clearly missing in our understanding of communication, but we have not yet put our collective finger on it.

Two books may help sort this out. In Power and Influence, Nobel Prize economists Daron Acemoglu and Simon Johnson present a massive trove of data to claim that increasing automation often increases organizational productivity without increasing overall economic progress for everyone. They argue that the abstractions behind automation focus on displacing workers rather than augmenting workers by enabling them to take on more meaningful tasks.¹ In How to Know a Person, David Brooks presents communication practices that help you see and appreciate the everyday concrete concerns of others.²

Maybe we need to occasionally descend from the high clouds of our abstractions to the concrete earthy concerns of everyday life.

Sustainability is undeniably a grand challenge. As the world population and the average affluence per person continues to grow, we are eagerly consuming Earth’s natural resources. The Earth Overshoot Day marks the date when the demand for ecological resources by humankind in a given year exceeds what the Earth can regenerate in a year’s time. While the world’s Earth Overshoot Day fell at the end of December in the early 1970s, it has progressively antedated since then and was computed to fall on Aug. 1 in 2024. The overshoot day is (much) earlier for many countries, as early as Feb. 26 for Singapore, March 13 for the U.S., March 26 for Canada, and April-May for most European countries as well as South Korea, Israel, Japan, and China.^a

The continuously growing consumption of Earth’s resources, including materials and energy sources, (inevitably) induces climate change. Greenhouse gas (GHG) emissions result in detrimental global warming, and a recent study reports that the contribution of information and communication technology (ICT) to the world’s global GHG emissions, currently between 2.1% and 3.9%,¹¹ is growing at rapid pace. While this percentage may seem small, it is not. In fact, ICT’s contribution to global warming is on par with (or even larger than) the aviation industry, which is estimated to be around 2.5%.^b

To combat global warming, the Paris Agreement under the auspices of the United Nations (UN) aims to limit global warming to well below 2 degrees Celsius, and preferably to 1.5 degrees Celsius, compared to pre-industrial levels. In 2019, the UN stated that  global emissions must be cut by 7.6% each year over the next decade to meet the Paris Agreement.^c More recently in November 2023, the UN stated that insufficient progress has been made so far to combat climate change.^d

Given the pressing need to act, along with the significant and growing contribution of computer systems to global warming, it is imperative that we, computer system engineers, ask ourselves what we can do to reduce computing’s environmental footprint within the socio-economic context. To do so, this article reformulates the well-known and widely used IPAT model,⁶ such that we can reason about the three contributing factors: population growth, increased affluence or number of computing devices per person, and carbon footprint per device over its entire lifetime, which includes the so-called embodied footprint for manufacturing, assembly, transportation, and end-of-life processing, and the operational footprint due to device usage during its lifetime.¹³

The growth in population and affluence leads to a growing sustainability gap as illustrated in Figure 1. If we were to keep the carbon footprint per device constant relative to the present time, the total carbon footprint due to ICT would still increase by 9.4% per year, leading to a 2.45× increase in GHG emissions over a decade. In contrast, meeting the Paris Agreement requires that we reduce GHG emissions by a factor 2.2×. Bridging this widening sustainability gap between the per-device status quo and the Paris Agreement requires that we reduce the carbon footprint per device by 15.5% per year or by a cumulative factor of 5.4× over a decade.

Analyzing the carbon footprint for a select number of computing devices (smartwatches, smartphones, laptops, desktops, and servers) reveals that vendors do pay attention to sustainability. Indeed, the carbon footprint per computing device tends to reduce in recent years, at least for some devices by some vendors. However, the reduction in per-device carbon footprint achieved in recent years appears to be insufficient to close the sustainability gap. The overall conclusion is that a concerted effort is needed to significantly reduce the demand for computing devices while reducing the carbon footprint per device at a sustained rate for the foreseeable future.

The IPAT Model

IPAT is the acronym of the well-known and widely used equation⁶ which quantifies the impact I of human activity on the environment:

$$I=P\times A\times T$$ (1)

P stands for population (that is, the number of people on earth), A accounts for the affluence per person or the average consumption per person, and T quantifies the impact of the technology on the environment per unit of consumption. The impact on the environment can be measured along a number of dimensions, including the natural resources and materials used (some of which may be critical and scarce); GHG emissions during the production, use, and transportation of products; pollution of ecosystems and its impact on biodiversity; and so on. The IPAT equation is used as a basis by the UN’s Intergovernmental Panel for Climate Change (IPCC) in their annual reports.

The IPAT equation has been criticized as being too simplistic, assuming that the different variables in the equation are independent of each other. Indeed, in contrast to what the above formula may suggest, improving one of the variables does not necessarily lead to a corresponding reduction in overall impact. For example, reducing T in the IPAT model by 50% through technology innovations to reduce the environmental impact per product does not necessarily reduce the overall environmental impact I by 50%. The fundamental reason is that a technological efficiency improvement may lead to an increase in demand and/or use, which in turn may lead to an increase, rather than a reduction, in overall impact. This is the well-known rebound effect or Jevons’ paradox, named after the English economist Williams Stanley Jevons, who was the first to describe this rebound effect. He observed that improving the coal efficiency of the steam engine led to an overall increase in coal consumption.² Although there is no substantial carbon tax as of today, Jevons’ paradox still (indirectly) applies to computer systems. Efficiency gains increase a computing device’s compute capabilities, which stimulates its deployment (that is, more devices are deployed due to increase in demand) and its usage (that is, the device is used more intensively). The result may be a net increase in total carbon footprint across all devices despite the per-device efficiency gains.

The rebound effect can be (partly) accounted for in the IPAT model by expressing each of the variables as a compound annual growth rate (CAGR), defined as follows:

$$CAGR={\left(\begin{array}{c}{V}_t\\ V_0\end{array}\right)}^{1/t}-1$$ (2)

with V₀ the variable’s value at year 0 and V_t its value at year t. The IPAT model can be expressed using CAGRs for the respective variables:

$${\mathrm{CAGR}}_{\mathrm{overall}}={\prod }_{i=1}^N(CAG{R}_i+1)–1$$ (3)

This formulation allows for computing the annual growth rate in overall environmental impact or GHG emissions as a function of the growth rates of the individual contributing factors. If the growth rates incorporate the rebound effect, that is, higher consumption rate as a result of higher technological efficiency, the model can make an educated guess about the expected growth rate in environmental impact.³

The Environmental Impact of Computing

We now reformulate the IPAT equation such that it provides insight for computer system engineers to reason about the environmental impact of computing within its socio-economic context. We do so while focusing on GHG emissions encompassing the whole lifecycle of computing devices. The total GHG emissions C incurred by all computing devices on earth can be expressed as follows:

$$C=P\times \frac{D}{P}\times \frac{C}{D}$$ (4)

P is the world’s global population. D/P is a measure for affluence and quantifies the number of computing devices per capita on earth. C/D is a measure for technology and corresponds to the total carbon footprint per device. Note that C/D includes the whole lifecycle of a computing device, from raw material extraction to manufacturing, assembly, transportation, usage, and end-of-life processing. We now discuss how the different factors P, D/P, and C/D in the above equation scale over time.

Technology.  The carbon footprint per device C/D and its scaling trend CAGR_C/D is much harder to quantify because of inherent data uncertainty and the myriad computing devices. The carbon footprint of a device depends on many factors, including the materials used, how those materials are extracted, how the various components of a device are manufactured and assembled, how energy efficient the device is, where these devices are used, the lifetime of the device, how much transportation is involved, how end-of-life processing is handled, and so on. Despite the large degree of uncertainty, it is instructive and useful to analyze lifecycle assessment (LCA) or product carbon footprint (PCF) reports that quantify the environmental footprint of a device. All LCA and PCF reports acknowledge the degree of data uncertainty; nevertheless, they provide invaluable information for consumers to assess the environmental footprint of devices.

To understand per-device carbon footprint scaling trends, we now consider a number of computing devices from different vendors. We leverage the carbon footprint numbers published in the products’ respective LCA or PCF reports. In particular, we use the resources available from Apple,^g Google,^h and Dell.ⁱ We now discuss carbon footprint scaling trends for smartwatches, smartphones, laptops, desktops, and servers.

The bottom line is that per-device carbon footprint has not increased dramatically over the past years and has even significantly decreased in some cases. Several interesting conclusions can be reached upon closer inspection across devices and vendors.

Smartwatches.  Figure 2 quantifies the carbon footprint for different generations of Apple Watches with similar capabilities (GPS versus GPS plus cellular) and sport band. All watches feature either an aluminum case (42mm in Series 1 to 3, 44mm in Series 4 to 6, and 45mm in Series 7 and 8) or a stainless case (Series 9). It is surprising perhaps to note that a smartwatch’s carbon footprint was on a rising trend until 2019 before declining. Indeed, the carbon footprint of a GPS watch has increased with a CAGR_C/D = +23.9% from 2016 (Series 1) until 2019 (Series 5), while the carbon footprint of a GPS-plus-cellular watch has decreased with a CAGR_C/D = −7.7% from 2019 (Series 5) until 2023 (Series 9).

Smartphones.  Figure 3 illustrates the carbon footprint for Apple iPhones starting with iPhone 7 (release date in 2016) until iPhone 15 Pro Max (release date in 2023) with different SSD capacity. We note a similar trend for the Apple smartphones as for the smartwatches: Per-device carbon footprint increased until 2019, when it began declining. Indeed, from iPhone 8 (2017) to iPhone 11 Pro Max (2019) with 256GB SDD, the carbon footprint has increased from 71kg to 102kg CO₂eq (CAGR_C/D = +19.8%). From 2019 onward, we note a decrease in carbon footprint per device: From iPhone 11 Pro Max (2019) to iPhone 15 Pro Max (2023) with 512GB SDD, the carbon footprint decreased from 117kg to 87kg CO₂eq (CAGR_C/D = −7.1%). While Apple has been steadily decreasing the smartphone carbon footprint since 2019, it is worth noting that the declining trend is slowing down in recent years. For example, from iPhone 13 Pro Max (2021) to iPhone 15 Pro Max (2023) with 512GB SSD, the carbon footprint has decreased from 93kg to 87kg CO₂eq (CAGR_C/D = −3.3%).

To analyze these trends across vendors, Figure 4 reports results for the Google Pixel phones; the plot reports carbon footprints for the nominal series (Pixel 2, 3, 4, 5, 6, 7, 8), the ‘a’ series (Pixel 3a, 4a, 5a, 7a, 8a), the XL series (Pixel 2XL, 3XL, 4XL), and the Pro series (Pixel 6Pro, 7Pro, 8Pro). As for Apple, we note a declining trend in recent years for Google smartphones: The per-device carbon footprint increased until mid 2021, after which it started trending downward. This is noticeable for the nominal series as well as for the high-end phone series (XL and Pro series). The decrease in carbon footprint since 2021 is substantial for the nominal series (CAGR_C/D = −10.5%) and the Pro series (CAGR_C/D = −8.8%), while remaining invariant for the ‘a’ series since mid 2021.

Laptops.  Figure 5 reports the carbon footprint for Apple MacBook Pro and MacBook Air laptops with different configurations (screen size, see legend) as a function of their respective release dates; multiple laptops are reported per release date with different storage capacity, core count, and frequency. Several observations are worth noting. First, and perhaps not surprisingly, MacBook Air laptops incur a smaller carbon footprint than the more powerful MacBook Pro laptops. Second, for a given screen size, we note a steady decrease in carbon footprint, for example, MacBook Pro 16-in. (CAGR_C/D = −6.9% from 2019 to 2023) and MacBook Air 13-in. (CAGR_C/D = −5.1% from 2018 to 2024). Third, while this continuous decrease in per-device carbon footprint is encouraging, there is a caveat: Discontinuing a particular laptop configuration and replacing it with a more powerful device comes with a substantial carbon footprint increase. In particular, replacing the MacBook Pro 15-in. with a 16-in. configuration mid-2019 increases the carbon footprint by at least 11.3%; likewise, the transition from 13-in. to 14-in. in the second half of 2022 led to an increase of at least 33.5% for entry-level MacBook Pro laptops.

Looking at Dell, we note a slightly different outcome. The data in Figure 6 reports the carbon footprint for the 3000, 5000, and 7000 Dell Precision laptops. The per-device carbon footprint increased from 2018 until 2023 for the 5000 (CAGR_C/D = +4.1%) and 7000 (CAGR_C/D = +3.8%) laptops, while being invariant for the 3000 laptops. Note that the carbon footprint drastically drops for the most recent laptops released in February and March 2024, but this is due to a change in carbon accounting from MIT’s PAIA tool to Dell’s own ISO14040-certified LCA tool.

Desktops and workstations.  The outlook is mixed for desktops and workstations, with some trends increasing and others decreasing. Figure 7 shows carbon footprint for the Dell OptiPlex 700 Series Tower desktop machines (left) decreasing at a rate of CAGR_C/D = −8.1% but increasing at a rate of CAGR_C/D = +4.0% for Dell Workstations 5000 and 7000 Series (right).

Servers.  Figure 8 reports the carbon footprint for Dell PowerEdge rackmount ‘R’ servers across four generations (13^th, 14^th, 15^th, and 16^th); this includes Intel- and AMD-based systems. Server carbon footprint numbers are subject to its specific configuration and deployment, more so than (handheld) consumer devices for at least two reasons. First, because the operational footprint tends to dominate for servers—unlike handheld devices which are mostly dominated by their embodied footprint¹³—the location of use (and its power-grid mix) has a substantial impact. Second, hard-drive capacity, memory capacity, and processor configuration heavily impact the overall carbon footprint. Overall, server carbon footprint seems to be relatively constant over the past decade, although we note a small increase in average carbon footprint from the 13^th to the 16^th generation (CAGR_C/D = +1.8%). The carbon footprint of a typical high-end server tends to range between 8,000kg and 15,000kg CO₂eq over the past decade. Entry-level servers tend to have a lower carbon footprint below 6,000kg CO₂eq with a downward trend in recent years. (See the couple of data points in the bottom right corner in Figure 8.)

Quantifying the Sustainability Gap

Recall that population and affluence increase, (CAGR_P = +0.9%) and (CAGR_D/P = +8.4%), respectively. Technology, on the other hand, seems to decrease for many devices, ranging from CAGR_C/D = −3.3% to −10.5%, while increasing for others from +1.8% to +4.0%, as summarized in Table 2. The question now is whether these trends lead to an overall increase or decrease in the environmental footprint of computing.

Figure 9 predicts the overall carbon footprint for the next decade normalized to present time for a variety of typical per-device carbon footprint scaling trends, that is, CAGR_C/D = +4%, −5%, −10%. In addition, we consider the following three scenarios:

The Socio-Economical Context

The above analysis assumes that the world population and the number of devices per person will continue to grow at current pace for the foreseeable future. The task of decreasing the carbon footprint per device by 15.5% per year to meet the Paris agreement can be loosened to some extent by embracing a certain level of sobriety in affluence, that is, limiting the number of devices per person. This requires a perspective on the socio-economical context of computing, which includes economic business models, regulation, and legislation.

The computing industry today is mostly a linear economy where devices are manufactured, used for a while, and then discarded. The lifetime of a computing device can be relatively short, for example, two to four or five years, leading to increased e-waste. Reusing, repairing, refurbishing, repurposing, and remanufacturing devices could contribute to a circular economy in which the lifetime of a computing device is prolonged, thereby reducing e-waste and tempering the demand for more devices.¹⁰ For example, Switzer et al.²³ repurpose discarded smartphones in cloudlets to run microservice-based applications. Reducing the demand for devices could possibly relax the need for reducing per-device carbon footprints.

There is a moral aspect associated with reducing the demand for devices, which is worth highlighting. As mentioned previously, affluence is higher in the western world (North America and Western Europe) compared to other parts of the world and moreover it is increasing faster. From an ethical perspective, this suggests that the western world should make an even greater effort to reduce the environmental footprint of computing—in other words, we should not necessarily expect other parts of the world to make an equally big effort to solve a problem the western world is mostly responsible for. This implies that the western world should step up its effort in embracing sobriety (that is, consume fewer devices per person) and making individual computing devices even more sustainable.

In addition to transitioning toward a circular economy, other business models can also be embraced. Today, most cloud services are free to use, see for example social media, mail, Web search, and so on, while relying on massive data collection. Maintaining, storing, processing, and searching Internet-scale datasets requires massive compute, memory, and storage resources. According to a recent study by the International Energy Agency,¹⁵ datacenters are estimated to account for about 2% of the global electricity usage in 2022; and by 2026, datacenters are expected to consume 6% of the nation’s electricity usage in the U.S. and 32% in Ireland. Moreover, data storage incurs a substantial embodied footprint.²⁴ The environmental footprint of free Internet services is hence substantial. Allowing low-priority files to degrade in quality over time could possibly temper the environmental cost for storage devices.²⁶ But we could go even further by changing the business and usage models of Internet-scale services. Imposing a time restriction for uploaded content could possibly temper the demand for more processing power and storage capacity. We may want to limit how long we keep data around depending on its usefulness and criticality. To make it concrete: Do we really need to keep (silly) cat videos on the Web forever? Limiting to a day or a week may serve the need. Alternatively, or complementarily, one could demand a financial compensation from the customer for using online services. In particular, one could ask customers if they are willing to pay to keep their content online. For example, do you want to pay for your cat videos to remain online for the next month or year?

Transitioning to renewable energy sources (solar, wind, hydropower) is an effective method to reduce the carbon footprint of computing—as with any other industry. Renewables during chip manufacturing have the potential to drastically reduce a device’s embodied carbon footprint. Conversely, renewables at the location of device use drastically reduce a device’s operational emissions. This is happening today as renewables take up an increasingly larger share in the electricity mix.²⁰ However, there are several caveats. First, total electricity demand increases faster than what renewables can generate, increasing the reliance on brown electricity sources (that is, coal and gas) in absolute terms.²⁰ In other words, the transition rate to renewables is not fast enough to compensate for the increase in population and affluence. Second, the amount of green energy is too limited to satisfy all stakeholders. For example, Ireland has decided to limit datacenter construction until 2028 because allowing more datacenters to be deployed would compromise the country’s commitment that 80% of the nation’s electricity grid should come from renewables by 2030.¹⁶ Third, while renewables reduce total carbon footprint, it does not affect other environmental concerns, including raw material extraction, chemical and gas emissions during chip manufacturing, water consumption, impact on biodiversity, and so on.

The analysis performed in this paper considered computing as a standalone industry. But computing may enable other industries to become more sustainable, thereby (partially) offsetting its own footprint. This could potentially lead to an overall reduction in environmental footprint.¹⁹ For example, computer vision enables more efficient agriculture using less water resources and pesticides; artificial intelligence and machine learning could make transportation more environmentally friendly; smart grids that use digital technologies could increase the portion of renewables in the electricity mix in real time; or Internet-of-Things (IoT) devices could help reduce emissions in residential housing. While the anticipated sustainability gains in other industries may be substantial, one must be careful when analyzing such reports, that is, one has to carefully understand the assumptions and the associated limitations to fully grasp the validity of such analyses.²¹ Moreover, one must be wary of Jevons’ paradox as mentioned before: Making a product or service more carbon-friendly may lead to an increase in overall carbon footprint if the efficiency gain leads to increased deployment and/or usage. In other words, one should be aware of the bigger picture—unfortunately, holistic big-picture assessments are extremely complicated to make and anticipate.

Finally, regulation and legislation may be needed to temper the environmental footprint. The previously cited IEA report¹⁵ states that “regulation will be crucial in restraining data center energy consumption” while referring to the European Commission’s revised energy-efficiency directive. The latter entails that datacenter operators have to report datacenter energy usage and carbon emissions as of 2024, and they have to be climate-neutral by 2030. Further, the European Parliament adopted the so-called ‘right to repair’ directive, which requires manufacturers to repair goods with the goal of extending a product’s lifetime—thereby reducing e-waste and the continuous demand for new devices. Overall, innovation in regulation, legislation, and/or business models will be needed to incentivize (or even force) manufacturers, operators, and customers to temper the demand for more devices, while making sure that our computing industry can still thrive and generate welfare. This is a call for action for our community to reach out to psychologists, sociologists, law and policy makers, entrepreneurs, business people, and so on to holistically tackle the growing environmental footprint of computing.

Related Work

Our computer systems community recently started considering sustainability as a design goal, and prior work focused mostly on characterizing,^8,12,13,24 quantifying,^9,14,17 and reducing^1,4,5,22,25 the carbon footprint per device. However, as argued in this paper, to comprehensively and fully understand and temper the environmental footprint of computing, one needs to consider the socio-economic context within which we operate. Population growth and increased affluence is a current reality we should not be blind to and which impacts what we must do to reduce the overall environmental impact of computing.

Conclusion

This paper described the sustainability gap and how it is impacted by population growth, the increase in affluence (increasing number of devices per person), and the carbon intensity of computing devices. Considering current population and affluence growth, the carbon intensity of computing devices needs to reduce by 9.4% per year to keep the total carbon footprint of computing constant relative to present time, and by 15.5% per year to meet the Paris Agreement. Several case studies illustrate that while (some) vendors successfully reduce the carbon footprint of devices, it appears that more needs to be done. A concerted effort in which both the demand for electronic devices and the carbon footprint per device is significantly reduced on a continuous basis for the foreseeable future, appears to be inevitable to keep the rising carbon footprint of computing under control and, if possible, drastically reduce it.

Concerns that artificial intelligence (AI) systems pose serious risks for public safety have caused legislators and other policymakers around the world to propose legislation and other policy initiatives to address those risks. One bold initiative in this vein was the California legislature’s enactment of SB 1047, the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act, in late August 2024.

Lobbying for and against SB 1047 was so intense that California’s governor Gavin Newsom observed that the bill “had created its own weather system.” In the end, the governor vetoed the bill for reasons I explain here. After a brief review of the main features of SB 1047, this column points out key differences between SB 1047 and the EU’s AI Act, identifies key supporters and opponents of SB 1047, and discusses arguments for and against this bill. It also explains Governor Newsom’s reasons for vetoing that legislation and considers whether national or state governments should decide what AI regulations are necessary to ensure safe development and deployment of AI technologies.

Key Features of SB 1047

Under SB 1047, developers of very large frontier models (defined as models trained on computing power greater than 10²⁶ integer or floating point operations or costing more than $100 million at the start of training) and those who fine-tune large frontier models (also measured by compute requirements and/or training costs) would be responsible to ensure that these models will not cause “critical harms.”

The bill identifies four categories of critical harms:

• Creation or use of chemical, biological, radiological, or nuclear weapons causing mass casualties;

• Mass casualties or more than $500 million damage because of cyberattacks on critical infrastructure;

• Mass casualties or more than $500 million in damages resulting in bodily injury or damage to property that would be a crime if humans did it; and

• Other comparably grave harms to public safety and security.

Under this bill, developers of large frontier models would be required to take numerous steps at three phases of development: some before training, some before use of such a model or making it available, and some during uses of covered models. Among the required steps would be installing a “kill switch” at the pre-training stage, taking reasonable measures to prevent models from posing unreasonable risks, and publishing redacted copies of the developers’ safety and security protocols. (A “kill switch” would enable humans to stop an AI system from becoming an autonomous actor capable of inflicting critical harms.)

Developers would also be required to hire independent third-party auditors to ensure compliance with the law’s requirements. They would further be obliged to submit these audits and a statement of compliance annually with a state agency. Developers would further be responsible for reporting any safety incident of which they become aware to that agency within 72 hours of learning about it.

The legislation authorized the California Attorney General to file lawsuits against frontier model developers who violated that law’s requirements seeking penalties for up to 10% of the initial cost of model development for a first violation and up to 30% of development costs for subsequent violations. Whistleblowers who called attention to unreasonable risks that frontier models pose for causing critical harms would be protected against retaliation.

In addition, SB 1047 would authorize establishment of a new California agency to publish implementing guidelines for compliance with the Act. This agency would have received the required audit and compliance reports, overseen model development, and proposed amendments as needed (including updates to the compute thresholds).

Comparing SB 1047 to EU’S AI Act

SB 1047 and the European Union’s AI Act both focus on safety issues posed by advanced AI systems and risks that AI systems could cause substantial societal harms. Both require the development of safety protocols, pre-deployment testing to ensure systems are safe and secure, and reporting requirements, including auditing by independent third parties and compliance reports. Both would impose substantial fines for developers’ failure to comply with the acts’ safety requirements.

There are, however, significant differences between SB 1047 and the EU AI Act. For one thing, SB 1047 focused its safety requirements mainly on the developers of large frontier models rather than on deployers. Second, the California bill focused secondarily on those who fine-tune large frontier models, not just on initial developers. The AI Act does not address fine-tuning.

Third, SB 1047 would require developers to install a “kill switch” so that the models can be turned off if the risks of critical harms are too great. The EU’s AI Act does not require this. Fourth, the California bill assumed that the largest models are those that pose the most risks for society, whereas the AI Act does not focus on model size. Fifth, SB 1047 was intended to guard against those four specific types of critical harms, whereas the EU’s AI Act has a broader conception of harms and risks that AI developers and deployers should design to avoid.

Proponents of SB 1047

Anthropic was the most prominent of the AI model developers to have endorsed SB 1047. (Its support came after the bill was amended to drop a criminal penalty provision and to substitute a “reasonable care” instead of a “reasonable reassurance” standard for the duty of care expected of large frontier model developers). Thirty-seven employees of leading AI developers expressed support for SB 1047 as well.

Yoshua Bengio, Geoff Hinton, Stuart Russell, Bin Yu, and Larry Lessig are among the prominent proponents of SB 1047 as a “bare minimum effective regulation.” They believe that making developers of advanced frontier models responsible for averting critical harms is sound because these developers are in the best position to prevent such harms.

Proponents consider SB 1047 to be “light touch” regulation because it does not try to control design decisions or impose specific protocols on developers. They believe that the public will not be adequately protected if malicious actors are the only persons or entities that society can hold responsible for grave harms.

The AI Policy Institute reported that 65% of Californians support SB 1047 and more than 80% agree that advanced AI system developers should have to embed safety measures in the systems and should be accountable for catastrophic harms. Proponents further believe that SB 1047 will spur significant research and advance the state of the art in safety and security of AI models.

Without this new regulatory regime, moreover, proponents believe developers who are willing to invest in safety and security will be at a competitive disadvantage to firms that cut corners on safety and security design and testing to get to market faster.

Opponents of SB 1047

Google, Meta, and OpenAI, along with associations of technology companies, as well as Marc Andreesen and Ben Horowitz, opposed SB 1047 in part because it focused on the development of models instead of on harmful uses of such models. These opponents are concerned this law will impede innovation and American competitiveness in AI industries.

OpenAI argued that because SB 1047 heavily emphasizes national security harms and risks, it should be for the U.S. Congress, not the California legislature, to regulate AI systems to address these kinds of harms.

Among SB 1047’s opponents are many AI researchers, including notably Professors Fei Fei Li of Stanford and Jennifer Chayes of UC Berkeley. These researchers are concerned about the bill’s impacts on the availability of advanced open models and weights to which researchers want access and on which they want to build.

San Francisco Mayor London Breed and Congresswomen Nancy Pelosi and Zoe Lofgren were among the other prominent critics of SB 1047. Lofgren, who serves on a House subcommittee focused on science and technology issues, wrote an especially powerful letter to Governor Newsom expressing her reasons for opposing that bill. Among other things, Lofgren said that AI regulations should be based on demonstrated harms (such as deep fakes, misinformation, and discrimination), not hypothetical ones (such as those for which kill switches might be needed).

The science of AI safety, noted Lofgren, is very early stages. The technical requirements that SB 1047 would impose on developers of large frontier models are thus premature. While the National Institute of Science and Technology aims to develop needed safety protocols and testing procedures, these measures are not yet in place. Nor are voluntary industry guides yet fully developed.

Lofgren also questioned SB 1047’s “kill switch” requirement. Although this might sound reasonable in theory, such a requirement would undermine the development of ecosystems around open models. She agreed with a report of the National Telecommunications and Information Administration that there is insufficient evidence of heightened risks from open models to justify banning them.

Lofgren also expressed concern about innovation arbitrage. If California regulates AI industries too heavily or in inappropriate ways, it might lose its early leadership in this nascent industry sector. And U.S. competitiveness would be undermined.

Governor Newsom’s Reactions

Governor Gavin Newsom issued a statement explaining his reasons for vetoing SB 1047. He pointed out that California is home to 32 of the world’s leading AI companies. He worried that this law would harm innovation in California’s AI industries. Regulation should, he believes, be based on empirical evidence and science.

Newsom questioned whether the cost and amount of computing power needed for AI model training is the right regulatory threshold. He suggested it might be better to evaluate risks based on ecosystems in which AI systems were deployed or on uses of sensitive data. He warned that the bill’s focus on very large models could give the public a false sense of security because smaller models may be equally or more dangerous as the ones SB 1047 would regulate. While recognizing the need for AI regulations to protect the public, Newsom observed that the AI technology industry is still in early stages and regulations need to be balanced and able to be adapted as the industry matures.

The governor agreed with SB 1047’s sponsors that it would be unwise to wait for a catastrophe to protect the public from AI risks and that AI firms should be held accountable for harms to which they have contributed. But SB 1047, in his view, was just not the right law at the right time for California.

To demonstrate his commitment to ensuring proper attention to public safety, Governor Newsom appointed an expert committee of thought leaders to advise him further about how California can achieve the delicate policy balance between promoting the growth of AI industries and research communities and protecting the public against unreasonable risks of harm. Joining Fei Fei Li and Jennifer Chayes on this committee is Tino Cuellar, a former Stanford Law professor, a former California Supreme Court Justice, and now Executive Director of the Carnegie Institute for Peace.

Despite vetoing SB 1047, the governor signed into law 19 other AI-related bills passed by the California legislature this year. Two of them regulate deep fakes, one obliges developers to make disclosures about AI training data, and one requires provenance data for AI-generated outputs.

Conclusion

The sponsors of SB 1047 seem to have carefully listened to and heeded warnings of some prominent computer scientists who are deeply and sincerely worried about AI systems causing critically serious harms to humankind. However, there is no consensus among scientists about AI public safety risks.

Concerns that advanced AI systems, such as HAL in 2001: A Space Odyssey, will take over and humans will not be able to stop them because their developers failed to install kill switches seem implausible. Legislation to regulate AI technologies should be based on empirical evidence of actual or imminent harms, not conjecture.

In any event, regulation of AI systems that pose risks of national security harms would optimally be done at the national, not state, level. But the Trump Administration is less likely than the Biden Administration to focus on systemic risks of AI, so maybe the state of California should lead the way in formulating a regulatory regime to address these risks.