Mapping the individual, social, and biospheric impacts of Foundation Models

Responding to the rapid roll-out and large-scale commercialization of foundation models, large language models, and generative AI, an emerging body of work is shedding light on the myriad impacts these technologies are having across society. Such research is expansive, ranging from the production of discriminatory, fake and toxic outputs


INTRODUCTION
Since the release of ChatGPT at the end of 2022, there has been considerable and sustained interest across policy, academia, and public discourses in the risks posed by artificial intelligence (AI), and particularly generative AI.Though ChatGPT may have generated the most column inches, its underpinning model-GPT 3.5 and, subsequently, GPT 4-was just one example of wider developments in the area of so-called "foundation" or "frontier" models.
Foundation models are AI technologies trained on very large, "broad" datasets that can be applied to a wide range of tasks and purposes [25].Their general, unspecified purpose and widespread deployment leads to concerns about potential unanticipated impacts when used in novel areas.These models are considered to form the "foundation" or base architecture of other systems.For example, a number of new applications such as ChatGPT and the conversational features of the Bing search engine, have been built on top of successive versions of OpenAI's GPT foundation model, which has been designed for natural language processing tasks.
While the techniques involved in developing and deploying foundation models are not new, the scale of the data used for training, the coordination of global networks of labor to process these massive amounts of data, the increased architectural complexity, and the development of increasingly powerful processors and other computational resources have together made possible new levels of predictive and generative sophistication.Due to the resources required to produce foundation models, the most prominent ones known to exist are the products of large and well-financed technology companies, such as Anthropic, Cohere, Hugging Face, Meta, OpenAI, Microsoft, and Alphabet (the latter two being substantial funders of other firms).
Responding to the rapid rollout and large-scale commercialization of foundation models, large language models (LLMs), and generative AI, an emerging body of work is exploring the myriad impacts these technologies are having across society.Such research is expansive.Prominent topics range from the production of discriminatory, fake, and toxic outputs and privacy and copyright violations to the unjust extraction of labor and natural resources.Foundation models present complex issues for law, policy, and practice, which arise concretely from the intertwined array of socio-technical systems in which their design, development, and use are embedded.These technologies pose risks and hazards that emerge from real-world contexts.Such issues arise from the global supply chains and labor sources that support their production, as well as the market forces and regulatory environments that influence their funding and financing.Risks likewise arise from the sociohistorical realities and legacies of inequity and exclusion that shape the data on which they are trained and from the patterns of privilege and socioeconomic stratification that influence the composition of the project teams that build them.All these as real-world contexts lead to the real-world consequences that have been the subject of much critical and responsible AI research on the impact of foundation models and generative AI over the past year.Yet, despite the importance of centering this empirically oriented work, much of the agendasetting policy and public discourse emerging primarily from the main geographical centres of innovation on foundation models (and relatedly LLMs and generative AI) has focused on sensational, catastrophic, and largely speculative risks relating to extreme and hypothetical scenarios.
The narrative of catastrophic risks was amplified by much mainstream, English language, media coverage of foundation models throughout 2023, fueled by high profile statements declaring potential existential threats from AI [141,142].Most notably a statement published in May 2023, and signed by many prominent figures in the field of AI, claimed that "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war" [38].While the research behind these claims is scant and broadly disputed, this narrative has reached far into predominant policy and regulatory discussions, including high profile international initiatives in the global north like the UK's AI Safety Summit [83] and the G7's Hiroshima AI Process [155].Such processes, while presenting valuable opportunities to bring together the international community in discussing the extant risks that AI presents to people, society, and the planet, and corresponding approaches to needed regulation, have been dominated by technical discourses with narrow framings of "safety" and have prioritized discussions of speculative global catastrophic or existential risks of AI.
Concerns about this narrowing of AI policy and regulatory discourse have been echoed by various academic communities who have questioned the lack of "ideological and demographic diversity" of the AI safety field [116]; and the privileging of speculative narratives as effective distractions from existing and well-documented risks and harms of AI systems today [78,92,226].A wealth of evidence demonstrates in particular the disproportionate negative impacts that AI systems have on marginalized communitiescommunities who are also underrepresented in decision-making processes regarding the design, development, deployment, or governance of AI.These issues, which are linked to historical and entrenched power asymmetries, are now being exacerbated by AI technologies, and risk becoming increasingly more urgent.
Therefore, through a review of the literature on the harms and risks of foundation models and generative AI, in this paper, we aim to address this visibility gap between real-world consequences and speculative risks.We propose a critical framework capable of adequately illuminating socio-technical risk contexts and attending to the social, political, and environmental dimensions of foundation models and generative AI.Through reviewing the existing evidence base our research maps a diverse set of risks and harms impacting individuals, communities, society, and the biosphere.
The paper starts with a discussion on the theoretical framework employed to investigate risks and harms in the context of foundation models.This framework abductively informs the methodological approach, which is detailed in section 3. Section 4 present our findings by delineating three distinct levels of algorithmic impactsindividual, social and biospheric.We conclude discussing how the integrative perspective we offer can aid in understanding the negative impacts of foundation models, and can help shape responsible AI futures.

THEORETICAL LENS: EXPANDING VIEWS ON ALGORITHMIC RISKS AND HARMS
There is a growing and multidisciplinary body of literature which explores the potential risks and harms posed by the relatively recent widespread popularization of foundation models.Although these risks and harms share many similarities with those presented by other types of AI, and indeed other data-intensive technologies more generally, it is crucial to examine how the proliferation and widespread adoption of foundation models may introduce new challenges, as well as further complicate existing ones.
The concept of risk is pervasive in modern societies and has become a central aspect of the AI governance discourse.Sociological studies have argued that theories of risk management and securitization emerged as a means to grapple with the uncertainties of modernity [18] and "making the future secure and certain" [186].The understanding of risk and harm, as well as how they interrelate, varies greatly across domains.For legal, policy, organizational, or actuarial purposes, risk is typically approached through quantifiable measures and anticipatory models of potential adverse or harmful events such pandemics, financial losses, health and safety incidents, or operational disruptions [63,147,151,170].Risk is therefore not inherently an issue of morality and can be understood as a contextdependent and relational concept-that is, expressed in relation to how factors like technical developments, organizational structures, or innovation may adversely impact a desired future and be variously interpreted by different actors [234].For example, consider a company which uses a job applicant screening software and has discovered that it makes racially biased screening recommendations.Where policy makers and civil society may be concerned about the various ways in which the technology undermines individual rights and social equity, a company might instead be more likely to focus on the risk of reputational damage and how this could in turn result in public distrust, low technology adoption, and commercial impacts.
Critical social science perspectives underline the multiple ramifications of identifying, analyzing, and managing risk associated with the impacts of technology in society [200,244].In this, some areas of research critically examine the various ways in which the risks and harms resulting from data and algorithmic systems are patterned across society at different levels [35,133].For instance, the field of science and technology studies (STS) has a well-established body of work exploring the interdependencies between technology and society, challenging the widespread deterministic view of technology as operationally autonomous and inherently value-neutral, and of innovation as following a predictable linear trajectory [136].When it comes to examining the impacts of AI, scholars in the fields of STS, critical data studies, and socio-legal studies have proposed to conceptualize the harmful effects of data-intensive technologies as issues relevant to human rights, social justice, cultural institutions, the rule of law, the public sphere, economic systems, and the environment [51,64,137,138,196,204,243].It is also from a critical perspective that some scholars increasingly highlight the need to approach algorithmic governance not only in terms of speculative notions of risk, but also in response to actual and observed harms to humans and the environment [73].
To build an understanding of risks and harms of technology from a moral and ethical standpoint, it is key to consider the questions of how power is distributed across the entire ecosystem of actors, where and how power manifests, and how asymmetries of power and information relate to differential impacts in society.Here it is helpful to draw insights from feminist and data justice scholarship, socio-legal studies, and the environmental justice literature.In particular, the concept of intersectionality makes explicit how social hierarchies manifest in different outcomes and experiences depending not only on the identity of the impacted individual or group, but also on the multiple compounding intersections of social identity and standing.These intersections go beyond crude constructions and classifications of individuals in society, to recognize the ways in which nuances of context, identity, and circumstance distinctly shape people's understanding and experiences of harm [52].
To capture these differential impacts as they relate to the proliferation of foundation models, in this paper we propose, following Leslie and Rossi [119], to assess risks and harms according to their individual, social, and biospheric levels of impact, contributing to an emerging dialogue within the FAccT and AI ethics communities [194,239].This perspective allows for a nuanced understanding of the observed and anticipated impacts of foundation models by recognizing the inherent socio-technical and globally entangled nature of these technologies.We use this framework to map the risks and harms found in the literature to date and offer an integrative perspective to address the most urgent negative impacts of foundation models and their downstream applications.

METHODS: SNOWBALL AND STRUCTURED SEARCH
We conducted a review of literature using two complementary and reinforcing qualitative literature review approaches which have been shown to be useful in studies aimed at surfacing patterns and trends in corpora of academic research [246].
We first applied a snowball approach [245], crowd-sourcing among all the co-authors, a core set of recent papers focused broadly on algorithmic harms and risks [19,22,25,58,105,110,192,239].These papers were selected based on our inclusion criteria of covering a broad range of multidisciplinary literature addressing risks and harms associated with foundation models.Populating a Zotero database, we followed snowball sampling approaches starting from the set of core articles and then carrying out a manual search using the references in those articles.Forward snowballing was used to identify new papers that focus on algorithmic risks and harms citing those in the core list using Google Scholar and Internet-based searches to access the abstract and full-text.We identified a total of 64 articles including journal articles, conference papers, and preprints drawing broadly on technical and socio-technical literature in which key risks and issues of foundation models, LLMs, and generative AI are described historically and contemporaneously.The search inclusion criteria covered English language publications and the most recent publication was of October 2023.
Complementing the snowball approach, we conducted a structured database search to identify primarily academic articles with a focus on the risks and harms associated with foundation models, LLMs, and generative AI.
Inclusion criteria covered English language journal articles, including conference papers and pre-prints, but excluding grey literature, monographs, commentaries, correspondences, and opinion pieces.The following electronic databases were searched: arXiv, ACM Digital Library, IEEE, Scopus, and Web of Science. 1 After the removal of duplicates and non-relevant papers, there were 114 papers in total for analysis, of which 11 were also papers in the snowball search results.The resulting set of papers were published in the period January 2018 to July 2023.The start date aligns with the date of appearance in the literature of transformer architectures which underpin LLMs, such as the BERT large language representation model [229].
We applied an abductive approach [213,233] to analyze the combined corpus of literature 167 papers.The abductive approach was used to code the results and subsequently map key relationships between the risks and harms found in the literature guided by our theoretical framework described in Section 2. Firstly, we extracted applicable keyword information from the abstract of each primary study as part of an initial coding process.The resulting keywords served largely as summative and process attributes of a risk or type of harm relevant to foundation models or LLMs.Three researchers subsequently clustered these attributes and assigned them into parent categories.The codes were subsequently refined by all co-authors (see Appendix for search strategy).In this stage, we considered whether at least one of the risks and harms listed in the paper implied or directly referred to individual, social, or biospheric impacts to map the papers onto our three-level framework.These codified findings are further analyzed in Section 4.

Limitations
We note that our structured search showed a prevalence of preprint papers (non-peer review) in the result set, some of which had been cited extensively in both preprint and peer-review articles. 2We used citations only as a broad indication of the preprints' impact.While some preprints were highly cited, these may have not undergone a rigorous peer-review process, and their findings should be interpreted with caution.In our sample, we analyze articles solely as an indicator of the range of concerns raised in the existing literature.The selected papers were also limited to the English language, and date range of January 2018 to July 2023.This range by default provided only a snapshot of a growing and multidisciplinary body of work which is arising in response to the fast development and roll-out of foundations models and LLMs, especially generative AI related studies.In addition, our structured literature search, which is not intended as systematic, only centred academic articles as a relevant account of the discourse on risks and harms of foundation models and LLMs to inform governance.We also note that this body of research is not focused exclusively on risks and harms, but often addresses, or points to, mitigations and governance of foundation models and AI at large.An analysis of other non-academic and informal sources was not part of this study and it warrants additional research.Notwithstanding, the codified mappings provide a substantive view of the research landscape and the range of concerns that are being raised around foundation models and LLMs, their development, implementation, and applications.

MAPPING INDIVIDUAL, SOCIAL, AND BIOSPHERIC IMPACTS OF FOUNDATION MODELS
The result set of 14 risks and harms coding (Table 1) is telling of the wide-ranging and multifaceted challenges posed by foundation models and AI systems at large.Papers typically engage with three or more aspects, exemplifying the broad spectrum of concerns in the literature to date.For instance, Zhuo et al. [258] delve into potential biases, toxicity, and issues related to the reliability and robustness of ChatGPT.Notably, a substantial set of search results focuses on bias and discrimination.These are most typically related to LLMs, and include works such as van der Wal et al. [227] and Huang et al. [99], which address multiple biases and harmful stereotypes, and others, like Abid et al. [1], Felkner et al. [72], Ovalle et al. [159], and Gadiraju et al. [76] that concentrate on specific societal biases.The result set also reveals that papers that focus on specific or a limited number of risks and harms often scrutinize issues of unreliable performance of foundation models, or misinformation and propaganda.While other issues such as lock-in and opacity, or overdependence in human-computer interaction, are less prevalent in the result set, it is important to note that their absence does not imply that they are not identified as relevant within the existing literature.Instead, they are simply not the primary focus of most papers in our dataset.
In the following, we elucidate three distinct levels of algorithmic impact (individual, social, and biospheric) with examples stemming from the literature and categorize the risks and harms emerging from our search using this typology (see Table 1).This lens on algorithmic harm aims to account for the intersectional dimensions and socio-technical embeddedness of foundation models with the aim to better inform potential technical and normative interventions to advance responsible AI.

Individual risks and harms
It is now undeniable that AI systems can have adverse impacts to individuals which may be physical, psychological, or financial, but may also negatively affect a person's dignity, reputation, or fundamental rights and freedoms.Indeed, individual-level risks and harms arising from the implementation of data-intensive technologies have been largely documented in the literature, with individual impacts having immediate as well as long-lasting implications for those at the receiving end of such harms [176].Equally so, a focus on individual rights has been at the core of conceptions of data protection, privacy, and security within digital regulation such as the European Union's General Data Protection Regulation (GDPR), where protection and redress mechanisms privilege notions of personal data or data subjects [196].
One of the biggest ethical concerns associated with AI systems has been their role in creating discriminatory, biased, and unfair outcomes which impact an individual's safety, health, wellbeing, and integrity.The proliferation of applications based on foundation models has not only augmented these concerns but has given rise to a host of new ethical, privacy, and safety issues.Biases and social prejudices present in training datasets and encoded in foundation models trickle down to applications like chatbots or image generators, thereby reproducing existing patterns of harm, exclusion, and Table 1: A mapping of foundation models risks and harms according to their level of impact (individual, social, biospheric).The table shows the number of papers across categories of harm and levels of impact, and the percentage (in brackets) of the total number of papers in the set in which the risk/harm appears.Papers appearing in more than one category were counted more than once.See Appendix for an explanation of each risk and harm category.discrimination [1,76,203].For instance, chatbots-such as Ope-nAI's ChatGPT or Google's Bard-are capable of producing natural language responses to prompts through complex statistical associations [25,254].These responses are enabled by the vast amounts of data that the foundation models have been trained on, comprising mostly of internet-scraped datasets that have been shown to encode biased and harmful associations across protected categories of religion, disability, gender, race, and ethnicity [1,61,148], and also at the intersections of these characteristics [85,211].
When it comes to issues of privacy and online safety, LLMs have been found to be effective tools for fraudulent activities such as scams, impersonation, and phishing, given their capability to generate personalized and highly convincing messages at scale [59,89,150].Research has also shown that LLMs are prone to leaking private or sensitive information contained within their training data corpus [36,106].Because these models are trained on data scraped from the Internet, privacy violations might take place unbeknownst to impacted individuals thereby limiting their ability to consent or opt out [158].Furthermore, models can associate different pieces of data to an individual, which poses privacy and security risks wherein sensitive information or even wrongful associations are revealed without consent [98].Indeed, in some instances, chatbots have been used with the intention of extracting protected information and bypassing restrictions through the use of malicious methods like prompt hacking, jailbreaking [174,236], and prompt injection [84,164].
Lastly, the proliferation of foundation models complicates the already significant negative impacts digital technologies have on people's psychological wellbeing.In the last decade, researchers have raised concerns over how hyperconnectivity and unhealthy relationships with technology can have serious mental health implications and influence how people form individual identities and construct self-hood [5,31,134].These issues are now exacerbated with the emergence of personalized AI chatbots capable of producing human-like responses.New research into the use of these conversational agents for mental health support warns about the potential for people's overdependence on these agents and the risk of social isolation [135].Similarly, documented cases of users self-harming as a result of exposure to toxic interactions with AI agents evidence the devastating impacts these technologies can have on vulnerable individuals [117,235,249].For example, prior research has demonstrated that individuals who are isolated may be more vulnerable to misleading information on social media platforms [26].When individuals are subject to algorithmic nudging and personalized information campaigns, they are at risk of loss of autonomy, dignity, and integrity [180].

Social risks and harms
While the rise of foundation models has generated huge expectations about their positive transformational potential for society in many domains, there are also concerns about how their adoption can contribute to worsening existing social inequalities, biases, and injustices.Beyond individual risks and harms, there are different ways in which technology can have wider collective and societal impacts.One way to conceptualize collective risks and harms is to look at the compounded or aggregate impacts to a particular group, or to the functioning of a society as a whole [137].
Risks and harms to communities can be characterized by how particular demographics and group formations within society are distinctively impacted by technology, particularly when it comes to vulnerable groups with protected characteristics, e.g., migrants, refugees, LGBTQI+ communities, Muslims, the elderly. 3As we have shown, not only can these technologies produce responses that are discriminatory and psychologically triggering for individual users, but the reproduction of hegemonic views, harmful stereotypes, and biases can have detrimental consequences at the collective level, affecting marginalized groups and communities including those who are not direct users, or are far removed from the technology.For instance, Abid et al. [1] demonstrate the persistent religious bias present in foundation models, noting that prompts containing the word "Muslim" produced responses with violent language.These negative associations, in turn, can spur targeted hate and disinformation campaigns centered around anti-Muslim or Islamophobic sentiments.
Collective forms of harm can also be understood from a relational lens.The relational lens captures the adverse impacts to social relationships of interdependence, trust, and solidarity, thereby negatively impacting people's experiences of belonging, or their capacity to flourish through their relationship to others.A rich and interdisciplinary body of research has shed light on the range of interpersonal effects arising from people's interactions with algorithms, including the affective impacts of the algorithmic sorting of content on social media feeds [33,67,202] and the proliferation of misinformation as detrimental to the quality of the public sphere and democratic deliberation [20,220,225].Foundation models in particular can increase the scale and speed at which disinformation campaigns can be disseminated across the information ecosystem [161,198].As generative AI applications powered by foundation models flood the public sphere with fake information, there is a risk of eroding public trust in the information that circulates online, further fueling social polarization and the creation of echo chambers [113,195].
Lastly, algorithmic systems are inextricably embedded in wider social, political, and institutional structures, and as such they also have a bearing on those structures [243].To grapple with the impacts that foundation models can have at a structural level, one must consider geopolitical, socioeconomic, legal, and organizational manifestations of power [119].For instance, some scholars have warned about the tight concentration of power arising from the limited pool of actors controlling technological capabilities globally [50,57,157]; the influence of these power asymmetries on the development of international policies, standards, and regulations for technological practices [15,43,47]; and the overwhelmingly invasive data practices carried out by governments and companies [68,74,219].
The collective and deeper structural effects of these technologies are difficult to anticipate and measure as they manifest over time as AI becomes more widespread and embedded in society.Nonetheless, it is helpful to analyze and situate foundation models in the context of well-known social implications linked with digital technologies.For instance, going back to the issues related to the public sphere, one could argue that the effortless production and spread of targeted misinformation and propaganda enabled by foundation models, can have lasting effects on wider cultural, social, and political structures, namely established democratic processes, representation, participation in public life, and institutional trust.Given the evidence of public and private actors deploying technologies to generate manipulative content to sway public opinion [48,75], scholars have warned about an increasing risk to social cohesion given the mounting erosion of trust in democratic structures [115].

Biospheric risks and harms
Lastly, biospheric risks and harms refer to the potential adverse impacts to ecological systems and environments, as well as wider impacts to the biosphere at the planetary level [169].Importantly, this category also includes harms to the individuals and communities negatively impacted by environmental effects and ecological disruptions resulting from AI development and use.In other words, biospheric, social, and individual risks and harms are fundamentally intertwined.A good entry point to understand biospheric impacts is to look at the largely hidden socio-material entanglements of data-intensive systems.That is, the complex supply chains, labor, and underpinning materials crucial to AI infrastructures-including the Rare Earth Elements (REEs) that are necessary to manufacture the semiconductors essential to computer hardware.The global nature of the supply chains of foundation models pose individual, social, and environmental risks by entangling processes of extraction, knowledge production, and ultimately decision-making [177].
The development of supercomputers which utilize REEs, and enable foundation models to be trained, has been found to have significant material footprints globally [12].The mining of silicon and other rare metals not only cause environmental harms, but can rupture local communities [4,51] and cause individual harm to the workers mining these elements [241].Furthermore, the negative consequences are usually borne disproportionately on vulnerable or marginalized communities largely located in the global south [34].Thus, when put in the context of global infrastructures, the interrelated nature of these risks and harms becomes clearer.Robbins and van Wynsberghe argue that the "interconnectedness of AI with ecological, social, and economic systems" [179] pose very real risks regarding societal lock-in to AI infrastructures.That is to say that beyond a certain point, the nature of the supply chains and materials powering certain AI infrastructures cannot be changed, whilst ecological harms would continue to multiply because of resultant carbon emissions, REE mining, and other processes.Given these potential path dependencies, we are currently at a crucial juncture for developing ethical governance mechanisms that address the interlocking risks of the biospheric and human facets of AI ecosystems.
The development and implementation of data-intensive systems like foundation models is also known to have high local environmental costs in the form of energy consumption required to train increasingly large and complex machine learning systems [113,130], and heavy water consumption needed for data center cooling [121].The amount of compute needed to train large-scale models has doubled every 3.4 months since 2012 [9,88], which translates to higher energy expenditures, and the use of costly resources even if the improvements in model accuracy are modest (i.e., diminishing returns in terms of model accuracy come at increased computational cost, see [187]).For example, the carbon emissions of training Google's BERT were roughly those of a transatlantic flight [207].All of this at a time when curbing our global emissions is crucial to slowing climate change down and effectively mitigating its effects [222].Efforts to develop responsible technologies that minimize their cascading impacts on the environment are hindered by the difficulty in comprehensively assessing carbon and energy footprints of training large models.Tools have emerged enabling practitioners and developers of these energy intensive technologies to monitor and track their models' emissions [11].
Although the environmental costs of AI are borne by everyone on the planet in terms of negative externalities contributing to climate change and ecological deterioration, they are not uniformly distributed among the world's populations or regions.The allocations of benefits and risks replicate existing patterns of environmental injustice, coloniality, and "slow violence" [152], in which a disproportionate exposure to risks and harms is borne by marginalized communities; be these in terms of pollution, destruction of local ecosystems, or involuntary displacement.By contrast, the majority of the economic benefits reaped by the use of the model mostly go to the model's proprietary owner(s), even when these technologies adopt an open source ethos [126].
Finally, the impact of pollution and ecosystem degradation on interdependent and connected biological systems (including sentient non-human animals) cannot be ignored [45,207].AI has the potential to put non-humans at further risk (e.g., animals and the ecological systems in which they live) through surveillance, monitoring, and testing for which the benefits are largely directed to humans and society.Potential harms arise from data collection on animals in intensive factory farming to increase productivity, or data tracking of protected wildlife, which hunters and poachers can hack to illegally hunt or trade animals [29,45].Beyond these physical harms to animals, the functional use of AI in image recognition and as recommender systems demonstrate biases in differentiating species borne out from underlying social factors [86].There is an urgent need to understand and respond in a systemic way to the interconnected and shared vulnerabilities of humans, animals, and the ecosystems in which we live and share [45].

Summary of findings
Our review shows that a large proportion of papers (40%) raise concerns about the potential for foundation models to perpetuate and amplify hegemonic views, harmful stereotypes, and societal and behavioral biases on an unprecedented scale.A similarly large set of papers (20%) also point to issues related to the creation and spread of misinformation and propaganda as well as the potential exploitation of these models for fraudulent services and cybersecurity attacks by malicious actors.Furthermore, the individual and societal impacts stemming from inconsistent or undesirable performance constitute a significant part of the mapped academic literature, making the case for cautious approaches to the deployment and use of such models.
Although less prevalent, a subset of the literature mapped point to concerns in connection to a range of technical and supply chain challenges of foundation models, and the potential compounding effects of those challenges as foundation models become more widespread.These range from the reliance on proprietary software and lack of transparency that enable lock-in and opacity risks to issues like outcome homogenization, narrowing of the market or monopolization, and the perpetuation of inequalities, which affect societal and economic structures.Our mapping also reveals that approximately 10% of the mapped papers address the environmental risks and harms associated with foundation models.While the primary focus of these latter works is on the biospheric-level impacts, some authors explore the interconnections between the adverse impacts of foundation models on ecology and animals and their consequential impacts on individuals and societies.This underscores the need for expanded exploration and attention to holistically understand the complexities surrounding the environmental adverse impacts of foundation models.Lastly, despite the extensive emphasis in much of the governance discourse, less attention is given in the academic literature to the extreme scenarios of existential, catastrophic, and other speculative risks of foundation models.These findings highlight the current visibility gap between real-world consequences and speculative risks, and sheds light on the areas requiring urgent and greater attention and efforts.

DISCUSSION: GRAPPLING WITH THE SCALE AND INTERCONNECTEDNESS OF FOUNDATION MODELS
Foundation models are highly consequential technologies which have sparked discussions about a spectrum of transformational impacts from the most promising to the most concerning.They pose unprecedented governance challenges but also offer an opportunity to draw lessons from, as well as re-examine, the landscape of sociotechnical impacts of data-intensive technologies.While emergent and novel, foundation models do not arrive de novo; they are "built on an installed base" [201] and inherit the tools and methods of prior generations of machine learning and neural network technologies.
As such, many of the foundation-model-related risks and harms that we highlight are likely to appear familiar because they occur, to some degree, in adjacent and precursor technologies.Even so, two key differentiating characteristics of foundation models are their massive scale and widespread embeddedness.Foundation models comprise hundreds of billions of parameters, trained on mountains of data, that consume enormous resources for both training and deployment.In particular, the scale of foundation models means that the risks and harms they present are not only likely to be magnified and amplified, but that this will happen in ways which transcend national and political boundaries, requiring a multi-pronged and transnational response.Harms that may have been minimal or just minimally attended to in prior generations of technology are now made visible and urgent.For example, data-intensive systems have been criticized for some time for their carbon footprint [205], but the high visibility and widely publicized demands of foundation models have brought this issue to the fore to the point that it is unsurprising to see this aspect discussed in the technical literature [25].
Another differentiating characteristic of foundation models is their embeddedness.Foundation models are conceptualized and architected as the base models for many and diverse types of downstream applications.The embeddedness of foundation models renders them invisible yet pervasive.As a result of their platformized architecture, foundation models form the basis of many thousands of extensions, and as such, the negative impacts and harms stemming from foundation models may be obfuscated and rendered relatively intractable.These two characteristics-scale and embeddednessposition foundation models to be both highly adaptive, highly elusive, and highly dangerous.We argue that any assessment of risks and harms should account for these socio-technical interdependencies, and any design of mitigations and policy responses should be commensurate with the level of impact be it individual, societal, or biospheric.The value of our proposed framework for conceptualizing risks and harms is that by decomposing these into individual, social, and biospheric impacts, we provide a conceptual tool with which to challenge attempts to narrow down salient risks and harms in a way that is meaningful only to some discrete set of affected actors and thereby limited in effect.The prevailing international policy discourse focusing on the technical safety of "frontier AI" systems, which has led to a flight from confrontation of the full spectrum of hazards presented here, is a signal example of such an erroneous narrowing [92].In this paper, we underline the necessity of the opposite approach.When identified at their empirical sources, the risks and harms of foundational models become visible from all angles.They cut across individual, collective, and environmental levels, spreading over and affecting entire populations, including entire socio-material and biophysical ecologies of humans and nonhumans.In this way, the impact and importance of the full range of risks and harms cannot be invisibilized or waved away in pursuit of economic, geopolitical, or other short-term goals.

Visibility gaps in the current assessment of algorithmic impact
As we have shown in this paper, the literature on risks and harms of foundation models is expansive and is likely to continue to grow.
Grappling with such a vast and heterogenous landscape is a challenging task both for those who are attempting to build a nuanced understanding of the technology as well as for those who are seeking to inform the debate on mitigations and governance.We recognize notable attempts in the literature to taxonomize these risks [239], as well as significant progress being made internationally to reckon with the scale and embeddedness of foundation models by framing AI governance as a human rights and transnational issue [101,221].However, some of the most prominent governance initiatives, particularly in Europe and the US, has thus far fallen short of contending with the most problematic harms stemming from unequal patterns of data, labor, and resource extraction and instead has focused largely on risks to adoption, technical safety issues, and catastrophic risks.One of the biggest challenges in assessing and anticipating algorithmic harm has to do with limited evidence and difficulties related to observing the indirect manifestations of harm, as well as foreseeing its effects over time.Grasping these complexities involves nuanced and context-dependent understanding.For instance, there are direct measurable impacts from the energy used to train or operate an LLM which are immediate and visible, but indirect harms arising from AI applications may only come to light over time or when enough evidence and research is made available.An AI system enabling the unsustainable extraction of mineral sites or unfair labor practices in the data labelling supply chain, are just a couple of illustrations of how negative indirect harms are likely to be largely hidden from view [154].Direct harms to individuals may translate to indirect harms to society and vice versa.For instance, the potential harm that can ensue with data misuse or privacy breach (e.g., personal data) is not just limited to the individual who is directly manipulated, but indirectly affects the interests of society at large [196].Where anthropocentric and species biases exist in AI, there are further examples of direct harm which remain understudied, largely because wider biospherical needs are not part of the conversation [45].Algorithmic systems that ignore animals or privileges a particular view of animal welfare while ignoring others can exacerbate these consequential impacts, and not least can lead to indirect and direct harms to interconnected ecosystems shared by both humans and animals [29,45].

Building a socio-technical scaffold to technical interventions
The current strategies for the mitigation of foundation model generated risks and harms that have most traction among policymakers and governments predominantly focus on technical interventions [156].There is thus an urgent necessity to delve into the social context within which these interventions are situated.This is crucial, especially since the advent of generative AI represents not only a significant milestone in technical advancement but also simultaneously transforms the very fabric of far-reaching socio-digital infrastructure like the Internet and its social experience [90,91].Consider, for instance, the issue commonly framed in technical terms as "model collapse" or "data pollution".In this scenario, the generative output from the widespread experimental use of chatbots and image generators feeds into public data pools.This influx poses the risk of influencing future datasets that AI models will subsequently incorporate.The amalgamation of human and AI-generated content potentially undermines the quality and diversity of AI-generated outputs [139].While this cycle undoubtedly necessitates technical intervention, the risk typology outlined in this paper enables the identification of concurrent impacts within the social sphere-impacts like the undermining of the long-term integrity of the information ecosystems on which modern democratic ways of life rest.
The integrative perspective we offer emphasizes, for example, that the risks surrounding the AI-enabled generation of biased or harmful images go beyond just offending individuals; in the aggregate, these systems have the potential to change social narratives around communities and to lock in cultural prejudices at scale, replicating and augmenting patterns of structural discrimination and injustice.Likewise, given the scale of their generative abilities, these technologies have broader planetary implications that derive from the cumulative costs of mass industrialized compute.Concentration on the perceived technical complexity of foundation models often mistakenly occludes such a clear ecological view of the social impact of generative outputs.Regardless, evolving research rightly insists that the AI community must grasp the subtleties, social contexts, and boundaries of human interaction with AI as a user-oriented technology, as well as the social and longitudinal aspects of innovation, more broadly [118,173,184].The risk mapping presented in this paper facilitates this understanding.Even a singular, seemingly innocuous creation of politically incorrect content by an AI tool, a feature that many such tools still permit [87], can accumulate and result in a proliferation of societally effective bias within the community.This bias is amplified by the repetitive contributions from multiple individuals.Consequently, the framework presented here helps us understand how concerns usually detected or faced at individual levels, in fact, scaffold larger risks and contribute to higher-level concerns, thereby providing a deeper socio-technical understanding of foundation models and AI at large.

Toward an integrative perspective on risks and harms
Our framework illustrates concretely how claims that there may be gains from the implementation and use of foundation models that outweigh their potential and observed harms need to be examined and nuanced through sharpened socio-technical lenses.Such claims often rely on a strictly utilitarian calculation in which the overall potential "benefits" of foundation model application may outweigh the total harms and risks and are hence largely incapable of accounting for more systemic considerations.However, as shown by our framework, foundation model harms are frequently difficult to track and measure-and this affects the capacity to coherently weigh benefits and harms.For instance, harmful use cases that are relatively intelligible when measured individually could have impacts and consequences that are much harder to trace at social and biospheric levels.A moderate, but cumulative and difficult-to-perceive, harm to planetary health has implications for billions of people whereas a significant, but straightforward, harm to some individuals may be quite limited in scope.This makes utility calculations of benefits and harms difficult to perform with sufficient precision to capture their full range, scale, and scope.For example, Bommasani et al. [25] argue that the benefits of releasing large models-such as applications that translate text in otherwise underserved languages-outweigh the risks of misuse and abuse by malicious actors.Within their analysis, there is also a recognition that relatively few firms and organizations have sufficient resources and capacities to produce foundation models and that efforts to develop them for use by lessresourced non-elites is nascent and unlikely to produce models with similar capacities.The analysis, however, fails to perceive the deeper risks engendered by the asymmetrical power structures and dynamics that lead to these inequitable differentials in resources, capacities, and access.
Where a handful of highly self-interested and profit-driven companies control the data, compute, and skills infrastructures on which the development and use of foundation models rely, social harms arising from expanding inequality, wealth polarization, concentration of economic power, and privilege biases that lead to the escalating marginalization of minoritized groups will likely abound.Risk analyses that fail to acknowledge macro-scale issues like this will discount socially consequential adverse impacts.In this case, the very fact that the architecture of foundation models relies on high resource concentration should be a launching pad for expanding the narrow lens on technical risk to account for the ways in which foundation models play a role in reproducing harmful social hierarchies and planetary degradation.Such recognition is also crucial for contending with the potential for foundation models to lead to hard-to-reverse and long-term effects as this technology progressively becomes more embedded in society.What we call an integrative perspective is essential for overcoming the dominant utilitarian and performance-oriented approach in the AI governance discourse which has tended to frame societal challenges in terms of quantifiable trade-offs between risks and benefits.

CONCLUSION
In this paper we have confronted a blind spot in the evolving AI governance landscape that derives from its reliance, particularly among prominent policymakers in the global north, on speculative risks and selective seeing.Drawing on the rapidly growing multidisciplinary body of research on foundation-model-generated risks and harms, we have shown how discerning this through integrative and socio-technically curved lenses better discloses the full spectrum of impacts across individual, social, and biospheric levels.We have argued, in this respect, that there exists a visibility gap between the range of concerns and evidence raised in the critical, empirically anchored literature, and the abstract and mainly hypothetical issues focused on within some of the most influential international AI governance initiatives.Such a gap reflects the convergence of the power dynamics, private interests, and geopolitical priorities that have agenda-setting consequences in the AI governance ecosystem-which is a challenge to coherently grappling with the unprecedented industrialization of large scale data-driven technologies, rapidly transforming veritably every domain of life and communities around the world.While potentially beneficial and transformative, foundation models also pose numerous risks to people, society, and the planet.We have aimed to deepen understandings of this broad range of risks by bridging the technical aspects of foundation models with their socio-technical underpinnings, connecting individual concerns to collective and planetary issues, and doing justice to the multifaceted and differential impacts these models have on affected communities.Our conceptualization of risk, particularly regarding the potential transformations effected by these technologies, demonstrates that the visibility of risks and harms should not be concealed or obscured by speculative concerns about existential threats, hypothetically conceived.Instead, as under the framework presented here, an understanding of risks should be grounded in robust evidence that is observable at various levels, demonstrating the potential of these adverse impacts to escalate, over time, and to widen in their scale and scope.Ultimately, the framework we have presented enables a comprehensive assessment of algorithmic impacts for which an interdisciplinary dialogue is key.As such it can be applied as an analytical tool to inform sociotechnical mitigations and to fundamentally expand existing toolkits for algorithmic fairness, transparency, and responsible AI.

IMPACT STATEMENT
1) Description of the ethical concerns the authors mitigated while conducting the work (as part of an ethical considerations statement): Throughout the process of developing the research questions and methodology, we foregrounded our work in an acute awareness and recognition of the role power plays in shaping conversation around the design, development, and deployment of technology.We grounded the evaluation of risks and harms as they relate to foundation models within well-established scholarship and critical discussion surrounding how power is distributed, the pervasiveness of stark power asymmetries, and how the differential impacts of foundation models -and technology at large-are experienced across multiple actors and layers of the ecosystem.We allowed this grounding to inform the framing of our research questions and our methodology, ensuring that a multidisciplinary body of scholarship was drawn on to inform our investigation.This approach included drawing on critical data studies, science and technology studies, and environmental justice scholarship, amongst other fields.It is through this grounding that we aimed to challenge dominant discourses that adopt speculative perspectives on the risks and harms presented by the rapid deployment of foundation models; discourses which overwhelmingly distract from current, real-world consequences.We therefore aim to foreground our work in observed impacts.Through the proposed individual -socialbiospheric framework, we work to acknowledge the inherently entangled and interdependent nature of socio-technical systems and their impacts, rather than further dominant discourses of fragmentation and division.
2) Reflections on how their background and experiences inform or shape the work (as part of a researcher positionality statement): The final form and content of this work was shaped by a few factors.Firstly, all researchers involved in the development and writing of this work are academically trained in research, and are based at research and higher-education institutions in the Global North, where the primary language is English.As such, the research itself was limited to the English language.The researchers involved in this work, however, represent a variety of communities and come from a diverse range of backgrounds, including lived and research experience in the Global South.Throughout the process of developing the work, we continuously reflected on-and engaged withhow our own values, beliefs, perspectives, and lived experiences inevitably shaped the work presented here.The research team which developed this work is interdisciplinary, coming from a variety of research specializations including anthropology, sociology, data science, and information science, as well as diverse and global industry experience.
3) Reflection on the adverse, unintended impact the work might have once published (as part of an adverse impact statement): In presenting this work, we are acutely aware that the research is squarely grounded in a rapidly evolving field.As such, we acknowledge that the proposed arguments and recommended pathways this research presents might not be relevant in the near to distant future, and we do not prescribe any solutions that could have an impact in the future.However, we also acknowledge that -despite these intentions-our risks and harms codes, and the subsequent categorizations we develop, may have unintended impacts; either through mis-categorizing certain risks and harms, or not accounting for others.These codes and categories were developed by our team's collective conceptualization of the terms 'risk' and 'harm'; as we are each working within our individual and group positionalities, there may be other forms of risks and harms not fully captured in this research.

A APPENDIX A.1 Search strategy and objectives
The primary goal of our search strategy was to identify relevant literature on risks and harms of foundation models and ensure a comprehensive understanding of their socio-political and environmental dimensions.To achieve this objective, we conducted searches across five academic electronic databases.The selected databases include arXiv, ACM Digital Library, IEEE, Scopus, and Web of Science.These databases were chosen based on their relevance in the field, their inclusion of a broader range of sources such as conference proceedings and preprints, and access to the full text of the articles.
To develop the search strategy, we chose a handful of keywords based on their relevance to the research question and maximize the scope of the search: "foundation model", "large language model", "llm", "general purpose artificial intelligence", "GPAI", "risk", "harm", "ethic".In addition, we used Boolean operators to refine search queries (see Table 2 for example).In some cases, we refined when the result set had a high percentage of irrelevant returns: AND TI "artificial intelligence" OR TI "AI" OR AB "artificial intelligence" OR AB "AI".
Searches were conducted on title and abstract fields.A sample set was piloted, before a full search was conducted across the selected databases.Inclusion criteria covered English language journal articles, including conference papers and pre-prints, but excluding grey literature, monographs, commentaries, correspondences, and opinion pieces.
The results were added to a Zotero library, which also included our snowball sample, and the merge of these two libraries was exported into a spreadsheet for refinement and coding.We then removed duplicates and non-relevant papers, reaching 167 papers in our sample.The use of an abductive approach supported establishing clearer links with the research objectives and developing a codified mapping of key relationships found in the literature results.To code the papers, firstly, we extracted applicable keyword information from the abstract of each primary study as part of an initial coding process.The resulting keywords served largely as summative and process attributes of a risk or type of harm relevant to foundation models or LLMs.Three researchers subsequently clustered these attributes (Level 2 -attribute codes) and assigned them into Level 1 or parent categories.For example, to attribute code copyright and intellectual property violations we assigned the parent category legal and regulatory violations.For the attribute codes difficult to ensure explainability and reliance on proprietary software we assigned the parent category lock-in and opacity risks (See Table 3 for full list of parent and attribute codes).The codes were subsequently refined by all co-authors.Refining #2 TI "foundation model*" OR AB "foundation model*" AND TI "Risk*" OR TI "Harm*" OR TI "ethic*"OR AB "Risk*" OR AB "Harm*" OR AB "ethic*" TI "Large Language model*" OR TI "LLM*" OR AB "Large Language model*" OR AB "LLM*" AND TI "Risk*" OR TI "Harm*" OR TI "ethic*" OR AB "Risk*" OR AB "Harm*" OR AB "ethic*" TI "general purpose artificial intelligence" OR TI "generalpurpose artificial intelligence" OR TI "GPAI" OR AB "general purpose artificial intelligence" OR AB "general-purpose artificial intelligence" OR AB "GPAI" AND TI "Risk*" OR TI "Harm*" OR TI "ethic*" OR AB "Risk*" OR AB "Harm*" OR AB "ethic*"

Risks and Harms Codes (and Papers) Code Description Count
Environmental effects and ecological disruption [11,19,25,86,98,111,113,114,121,124,130,158,163,178,192,199,218,239] Adverse and negative impacts of foundation models on ecology and animals. 19

Misuse
Misuse: [10] Biological misuse: [46,185] Dual use: [10,81,94,107,197,238,239] Illegitime surveillance: [10,25,113,238,239] Creation of violent or harmful content: [10,19,25,113,192,199] The misuse of foundation models is used here to describe specific contexts in which foundation models or technologies based on foundation models may be inappropriately used and lead to negative consequences.This includes uses that can have both civilian and military applications, or contribute to the handling, manipulation, or application of biological materials, organisms, or technologies. 19 Lock-in and opacity risks Challenges to benchmarking: [14] Difficult to ensure explainability: [166,216,218,237] Lack of replicability and transparency: [70,96,108,111,124,129,158,166,250,255] Low technological readiness: [250] Reliance on proprietary software: [124,126] Risks and harms associated with foundation models that are difficult to understand, replicate, or modify.These risks and harms arise from issues including reliance on proprietary software, challenges to benchmarking, lack of transparency, difficulty to ensure explainaibility, and low technological readiness.

15
Overdependency in human-computer interaction [49,113,124,135,143,165,166,199,216,238,239] Overreliance of humans interacting with computer systems, interfaces, and technologies built on foundation models for various aspects of their lives and potentially negatively impacting their well-being, safety, decision-making abilities, or interpersonal relationships. 11 Data risks and harms Data extractivism: [97] Data quality: [97,106] Datasets containing toxic data: [22,69,158,166,236] Degradation of the digital commons: [39,97] A range of risks and harms associated with the collection, storage, processing, and use of data within the data pipeline.These risks extend beyond the cybersecurity risks and harms described above and includes broader issues of ethical, social, and economic implications of data stewardship and management.They include issues of poor data quality, datasets containing toxic data, data extractivism, and the degradation of the digital commons.
Value misalignment [97,110] Misalignment of the output content of conversational agents with the norms and values of the human interacting with such agent.

2
Extreme or catastrophic risks and harms Extreme risks: [193] Catastrophic risks: [17] Speculative far-reaching or irreversible adverse impacts of foundation models at societal scale that extend beyond immediate impacts.

Table 2 :
Example of broad search (with refining keywords):