Development in times of hype: How freelancers explore Generative AI?

The rise of generative AI has led many companies to hire freelancers to harness its potential. However, this technology presents unique challenges to developers who have not previously engaged with it. Freelancers may find these challenges daunting due to the absence of organizational support and their reliance on positive client feedback. In a study involving 52 freelance developers, we identified multiple challenges associated with developing solutions based on generative AI. Freelancers often struggle with aspects they perceive as unique to generative AI such as unpredictability of its output, the occurrence of hallucinations, and the inconsistent effort required due to trial-and-error prompting cycles. Further, the limitations of specific frameworks, such as token limits and long response times, add to the complexity. Hype-related issues, such as inflated client expectations and a rapidly evolving technological ecosystem, further exacerbate the difficulties. To address these issues, we propose Software Engineering for Generative AI (SE4GenAI) and Hype-Induced Software Engineering (HypeSE) as areas where the software engineering community can provide effective guidance. This support is essential for freelancers working with generative AI and other emerging technologies.


INTRODUCTION
Public interest in generative AI (GenAI) has recently surged.Numerous organizations are exploring the utility of large text-to-text and text-to-image foundation models, integrating them into their existing infrastructure or creating new applications.Small to midsized enterprises and startups, known for their agility, are prone to investigate new trends.Due to limited in-house development resources, they oen depend on external freelancers who, contracted to deliver specific solutions, find themselves at the cuing edge of technological developments.Such freelancers take the role of explorers who open up new possibilities for their clients.
GenAI has limitations that may hinder independent developers working on applications.ey urgently require guidance.e first step to providing effective support is to identify and understand the challenges they face when developing GenAI-based applications.Given GenAI's rapid uptake and the significant role freelance developers play in its adoption, we ask:

RQ: What challenges do freelancers experience when developing solutions based on generative AI?
We explore the perspectives of freelance developers on GenAIrelated projects through the analysis of 52 interviews and survey responses.Approximately 60% of these freelancers work on smallscale projects as the sole developer, 20% participate in mediumsized projects with up to five team members, and the remaining 20% are either contractors in larger projects or operate a freelancing agency.ese freelancers identify 99 challenges across 11 Soware Engineering Body of Knowledge (SWEBOK) areas and 54 subcategories.ese soware engineering (SE) challenges arise from the inherent characteristics of GenAI as a technology and the associated hype.We use the term 'GenAI technology' to refer to the inherent features of GenAI as a computing paradigm, differentiating it from features of products that provide GenAI capabilities, such as large language models (LLMs) or foundation models and their APIs.e 'hype' refers to the novelty and fashionableness of GenAI among businesses.e characteristics of the paradigm, its products, its novelty, and its trendiness magnify known challenges and create new ones, as perceived by the freelancers.
ose findings are relevant to the SE community.Some of the issues identified by freelancers will likely impact the SE of GenAIbased applications for an extended period and, possibly, also beyond the context of freelancing.Further, future technologies will keep emerging as hype, requiring practitioners and researchers to manage hype-related phenomena.SE research must address these aspects.us, this paper suggests two research areas: (1) Soware Engineering for Generative AI (SE4GenAI) as an extension of SE for AI (SE4AI) discourse already present in SE, and (2) Hype-Induced Soware Engineering (HypeSE) to handle challenges associated with novel and fashionable technologies.
is manuscript contributes in three ways.First, it outlines challenges identified by freelancers who create GenAI-based solutions, adding a new, freelance-oriented perspective to SE4AI literature [15,16,37,38,42,45,57].Second, it classifies these challenges highlighting the pivotal role of hype as a source of practical SE challenges.is complements earlier discussions of hype in SE, particularly relating to blockchain [14,41,43].ird, it suggests new directions for SE research.

BACKGROUND
is study draws from practical background on GenAI, an emerging discourse on IT freelancers, SE4AI literature, and studies on the influence of technological hypes on development.

Current state of GenAI
In most general terms, GenAI embraces AI systems capable of generating new content, e.g., realistic images of people who never existed, based on examples [24].Due to a stochastic layer, GenAI is produces nondeterministic output1 , so one cannot predict what image will emerge out of a specific set of examples which creates the illusion of creativity [5,30].e broad uptake of GenAI is driven by the availability of foundation models, i.e., general-purpose models trained on extensive data sets that cover a wide range of general tasks and can be adapted for more specific tasks [8].Progress in natural language processing (NLP) yielded foundation models which accept natural text as input -LLMs.Following the November 2022 launch of OpenAI's ChatGPT, offering a chat interface for a powerful LLM, interest in GenAI skyrocketed.e initial hype around GPT models extended to other products relying on pretrained, externally hosted foundation models accessible through APIs [64].eir rapid uptake captivated the scientific community, leading to numerous papers on potential implications for various sectors [18] and evaluating the models' suitability and quality for particular uses [7].However, despite repeated calls, empirical data on GenAI's impact remains scarce.
An ecosystem of products has emerged around LLMs since late 2022.Major vendors, including OpenAI or Google offer access to hosted, closed-source models.Other providers offer open-source models, hosted on cloud infrastructure provided by HuggingFace or Replicate.Users can also train or fine-tune their own models though this requires expensive computational resources.LLMs can be accessed via dedicated APIs.Orchestration frameworks such as LangChain2 or LlamaIndex3 bundle these APIs and assist in pipelining functions.If an application depends on context data, these frameworks also support data preprocessing, storage, and management.A typical stack includes hardware or cloud resources, a LLM, its API, and optional orchestration framework and databases [30,34].Developers use this stack to create applications, typically using Python supported by major frameworks.Plugins for other languages and no-code platforms like Bubble.iomake GenAI even more accessible.is ecosystem is highly dynamic.
is brief summary highlights that the development of GenAIbased applications extends far beyond simply prompting a model.More specifically, it underscores that GenAI, as a technology, is distinguished by its ability to generate novel, unseen content from examples or instructions in a non-deterministic fashion.It also refers to a variety of products, including models, APIs, and optional frameworks that provide access to these capabilities.Systems that leverage the unique capabilities of GenAI through these products are referred to as GenAI-based applications in this manuscript.e value of these applications is derived mostly from the customization of foundational models' capabilities for a specific task.
We still lack a clear understanding of the challenges that arise during the development of such applications, particularly when the GenAI components depend on pre-existing external models.Comprehensive reviews of the challenges associated with LLMs have only recently become available as preprints [e.g., 30,34,64].ey rely on analysis of grey and white literature showing that LLMs are subjects to the following limitations: need for large training data, tokenization problems, computational requirements, resources necessary for fine-tuning, inference latency, limited context length, model updating and refinement, bias, information hallucinations, lack of explainability, reasoning errors, risk of adversarial aacks or malign use, model behavior changes over time, as well as spelling/counting errors [30].Further, results produced by LLMs cannot be reproduced even with decoding temperature set to zero [34].While these technical issues and characteristics are widely confirmed, their effect on the work of independent developers remains unclear.Furthermore, the provided summaries overlook potential challenges that may arise in various subprocesses of SE when the LLMs are required to function as a component within a more complex system.
Natural-language prompting has become a central interaction mechanism with text-to-image and text-to-text foundation models.Prompts are instructions given to LLMs to make them follow instructions or standardize the output.ey are the primary approach for users to control models' behavior.Prompt engineering has swily become a buzzword to describe the activity of craing prompts for specific tasks [11,63].Scientific literature starts to depict activities involved in prompt engineering leading to nascent guidance or development of support tools [3,54,[61][62][63].Despite an emerging body of paerns and best practices, prompt engineering involves a significant amount of trial-and-error cycles with the success depending on the intuition of the prompter rather than systematic guidance [30,63].is is further amplified by prompt brileness, i.e., unintuitive, large changes in the model output occurring due to very small changes in the prompt lake replacing words with synonyms or changing the order of requests [34].Whereas those and other challenges mentioned above are documented in public discourse and in the most recent studies, we lack empirical data on how soware engineers perceive those aspects and how the SE challenges of GenAI compare to the challenges known from non-generative machine learning (ML) technologies.
e impact of GenAI in soware (SW) industry is currently a hot topic.Yet most conversations evolve around the pros and cons of using LLMs for code generation [13,46,61,62].Discussion about the use of them in applications remains a niche discourse limited, mostly, to specific application areas [52].A proposed GenAI design process involves problem definition, model selection or model training, adaptation and alignment cycle (involving prompt engineering, fine-tuning, human alignment, and evaluation), and optimization and deployment [34,64].Although accurate, this advice is rather broad and lacks specific insights on how to tackle challenges unique to each phase.Moreover, to create value, GenAI technologies need to be integrated into either existing or new applications.is integration process encompasses standard SE steps, as well as design activities specific to GenAI.is could potentially complicate the development of GenAIbased applications, thereby introducing new SE challenges.is study aims to investigate these challenges from the viewpoint of freelance professionals.

Freelance developers
Freelancers, independent developers, and small agencies are actively engaging in the latest GenAI trends.A significant number of trending GitHub repositories related to GenAI are managed by individuals, not large tech corporations.Also, LLM orchestration frameworks such as LangChain and LlamaIndex are an outcome of independent and startup projects.Conversely, many client companies, lacking their own IT personnel and resources, are unable to delve into the potential of GenAI independently.As a result, they oen turn to freelancers, viewing them as a cost-effective alternative to professional IT firms.is scenario places freelancers in a crucial role within the GenAI ecosystem.
Freelance developers, while integral to the IT industry, face numerous specific challenges such as inconsistent income, reliance on past clients' reviews, variable competition, evolving skill requirements, and dependency on freelancing platforms [28,29].ey are oen the first to be affected by shis in trends and hypes [29], yet they play a pivotal role in the global delivery of IT development services and digital transformation [25,26].Existing research on freelancers in IT tends to take a broad approach, oen overlooking the specific activities or project types these professionals engage in [26].e subject of freelance developers is seldom addressed in SE research, and when it is, it is usually in the context of crowdsourced soware development [1,39].Nevertheless, freelancers are increasingly serving as independent developers who can be directly contracted by client companies via freelancing platforms [28], handling more complex tasks than mere sporadic assistance [39].We posit that the significance of freelancers will continue to grow with the advent of code generation models, enabling a single individual to efficiently develop an application or soware module independently.However, this potential can only be fully realized if clients understand freelancers' viewpoints on emerging trends, thereby facilitating more effective collaboration.is has motivated our focus on freelance developers.

Development of AI-based applications
SE for AI (SE4AI) and SE for ML (SE4ML) are an expanding research domain that underscores the distinctive characteristics of AI-based development in contrast to conventional SW projects.Various studies [6,15,16,31,37,38,42,57,59] have indicated that uncertainty factors such as the probabilistic aspect of ML and dependency on big data present a multitude of issues for developers.A range of meta-studies summarizes the findings [23,40,42,45].Multiple studies aended to those challenges.Meanwhile, several comprehensive meta-reviews emerged listing and sorting the challenges according to ML process steps [47], SWEBOK knowledge areas [23,42], or newly developed categories [44].
Many of the challenges listed in those studies overlap.Nevertheless, we also observed subtle differences in how they frame the challenges or how they order them.Given this observation, we combined four influential refereed meta-reviews published between 2021 and 2023 [23,42,44,47] to obtain a good coverage of various challenges reported in the literature.We identified 137 distinct challenges provided in supplementary material [17] 4 .Table 2 presents a selection of those challenges.To summarize, the main challenges stem from inadequate or absent data, necessity to engage in training as basic mode of creating AI capabilities, and the unpredictable and inherently complex nature of ML-based AI as evidenced by the changing anything changes everything (CACE) principle and models' opaqueness.e literature identifies lack of adequate guidance and process models to address those challenges, as well as problems with integrations of AI into end-user applications due to downstream compatibility problems caused by unpredictable or difficult to control output from AI components.
As explained above, GenAI contributes further technical challenges which the community just has started to understand and describe.is paper takes the perspective of freelance developers to understand the challenges perceive as particularly important and how they compare to SE4AI challenges.However, we speculate that technology is not the only source of challenges related to GenAI that freelancers need to deal with.

Technological hype and development
Despite the ongoing hype around GenAI [55] and other technologies, we found limited literature on SW development for hyped technologies and no recent comprehensive reviews.To provide background, we gathered literature from various fields, but the fragmented knowledge highlights the need for more research.
Hype occurs when a novel subject is heavily promoted across media platforms like newspapers, TV, and social networks to garner aention [10].Since 1995, Gartner Inc., a consulting firm, has measured hype as fluctuating expectations or visibility over time, marking stages such as innovation trigger, peak of inflated expectations, trough of disillusionment, slope of enlightenment, and plateau of productivity [21,53].Typically, hype is used in relation to technologies reaching the peak of inflated expectations.Such technologies become fashionable, i.e., popular and socially demanded at this specific point in time [51].Considering this discussion, we suggest defining technological hype as such: A technology experiences hype when it possesses inherent novelty (or includes novel features), thereby making it fashionable within business circles and society.
Hypes come and go.Various technologies gained aention in the past two decades.SE literature specifically aended to the implications of blockchain hype.e blockchain hype peaked in 2016-2017 [22].With many platforms offering blockchain services, the focus of SE literature was on developing reliable smart contracts.However, traditional SW development life cycle models proved inadequate due to the immutability of smart contracts and the reliance on expert reviews over code testing [14,43].is triggered development of new techniques for Blockchain-Oriented SE to accommodate for the inherent nature of this technology [14,49].e hype around blockchain, rather than being solely negative, became an opportunity to problematize and improve SE practice and theory as pointed out by some researchers [12,41].Similarly, the hype around ML also garnered SE aention yielding insights listed above [12].e concept of hype as a topic started entering SE discourse.Yet, it remains unclear how the fact that a technology is subject to hype impacts practitioners' work.
Agile development and its extensions, like MLOps or DevOps, were proposed to manage the novelty of technology and changing context.ey became a standard across many companies [2].Agile assumes that development can be divided into manageable parts, conflicting with the CACE principle.It also disregards the impact of hype, which significantly influences requirements.Despite these issues, agile remains influential in shaping SE practice.
Hype around technology oen increases demand for related solutions and prototypes.Driven by media coverage and success stories, companies seek to leverage these new technologies.is mirrors solution-based probing, where a technology is field-tested [9].However, companies aim not just to learn, but also to create business value.e increased demand significantly affects the workforce, potentially leading to new developer profiles.
Two phenomena emerge due to this dynamic.e first, resumedriven development, occurs when firms and developers include trending technologies in job listings and resumes.is practice can foster harmful dynamics, compromise team reliability, and stress both employees and employers [19,20].e second, hypedriven development, oen discussed by practitioners but not researchers [4,33], occurs when developers use hyped technology without considering its architectural implications or integration effort.is particularly affects independent and agile developers.
Despite the recognition of hype's impact on SE practice, there is still limited research and guidance for freelancers navigating the technological opportunities and risks.Given the rapid pace of technological advancements, this topic warrants more aention.Future hypes are inevitable.Particularly, the specific challenges that arise when a technology, like GenAI, becomes a major hype, remain unclear.is study contributes to this by analyzing challenges related to GenAI and discussing the role of hype in them.

METHOD
is qualitative study identifies challenges reported by freelancers in GenAI projects.It uses a survey and interviews from 52 active freelance developers.e study was approved by the University's Institutional Review Board in May 2023.We collected data in May and June 2023 and analyzed it in June and July 2023.e study compares those issues against SE4AI / SE4ML challenges reported in earlier studies.We provide data concerning the method employed and the full list of challenges identified in SE4AI meta-reviews in supplementary material.

Literature Study
To compile a comprehensive, current list of challenges reported in SE4AI and SE4ML literature, we referred to four extensive metareviews [23,42,44,47].ese were published in recognized, peerreviewed outlets from 2021 to 2023, summarizing challenges related to AI-based application development reported by empirical studies.We extracted individual challenges from each meta-review, not only from tables or graphics but also from textual descriptions for a more detailed understanding.Aer removing duplicates, we identified 276 unique challenges with slight differences.We then grouped these challenges into 11 Knowledge Areas, including a new one, Training and Testing Data, based on the reviews [23,42,44].Within these Knowledge Areas, we categorized the challenges into 54 groups, and aer merging near-duplicates, we ended up with 137 challenges (see Table 2 and supplementary material [17]).ese serve as a comparison basis with challenges mentioned by freelancers interviewed in this study.

Participants
We selected freelancers via Upwork, a freelancing platform.On May 1, we posted a job description offering $35 for a 10-minute survey and a 45-to-60-minute interview.We invited 450 freelancers (300 in May, 150 in June) who met specific criteria, including having a profile created before 2023, explicitly mentioning GenAI ('GPT', 'LLM', etc.) in their profile, and showing prior experience with the GenAI platforms and APIs.Following to the Upwork process, interested freelancers needed to submit a 'proposal' which, in our case, consisted of answers to four questions about their experience, GenAI knowledge, project history, and education.We received 96 answers.Some applicants had one or none GenAI projects while we required min.2, provided nonsensical answers (e.g., empty ones), or proposed significantly higher rates, e.g., $1000.Aer filtering, we were le with 81 potential participants.
ese individuals received a formal Upwork contract offer and survey invitation.However, 24 did not accept the offer or take the survey.Five participants withdrew from the study or became unresponsive aer accepting the offer or filling out the survey.We excluded their responses from our analysis.Ultimately, 52 freelancers participated in the study.
Our participant group was diverse, as detailed in Table 1.e mean age was 32.6 years.On average, respondents had participated in 4.8 GenAI-related projects, a reasonable figure considering GenAI's broad uptake began in late 2022.eir average experience as developers was 6.9 years, be it as freelancer or in other seings.Most held a bachelor's or master's degree, with 36 specializing in informatics-related subjects.ey reported using 14 different models/platforms, with 52 using GPT and 21 using DALL-E.ey came from 23 countries across all inhabited continents.Unfortunately, gender diversity was low, with only two females despite our efforts to aract more.

Data Collection
We employed three data collection methods.Initially, participants provided basic demographic details with their proposal.After contract acceptance and study consent, they completed a survey on 2-5 chosen projects, and the main challenges and rewards of GenAI applications development.en, they arranged an interview.e first author conducted these interviews, based on the problem-centered interview paradigm [58].e data collected in the proposal and survey provided background for the exploration of the problems.e interview commenced with introduction and project background, followed by interviewees narrating their journey to becoming GenAI freelancers and discussing one or two selected projects that they were particularly proud of, or found challenging or intriguing.e elicited narratives were enhanced by dialogues featuring semi-structured questions related to freelancers' positive and negative experiences with GenAI, comparisons earlier and GenAI-based projects, project uncertainties, challenges, enjoyable aspects of working with GenAI, and their views on freelancing.e sequence of questions was adapted to the dialogue's flow.During the interview, the interviewer was referring back to challenges and projects participants described in the survey.e interviewer took notes to revisit crucial points from the narratives later in the interview.
e interviews were conducted via Zoom, averaging 55.7 minutes each (median: 55, min: 42, max: 81).All were in English and transcribed verbatim, then anonymized.e connection between demographic and other data was only possible through a token, ensuring privacy and security standards.We achieved data saturation -last 10 interviews did not introduce any significant new insights.We provide interview guides and other material related to data collection in the supplemental material.

Data Analysis
e data analysis process was iterative and combined thematic analysis during the initial coding followed interpretive analysis step during workshops [50,56].e second author conducted qualitative coding of the interview data in a boom-up manner to avoid potential biases which could have emerged in the first author during interviews.He identified problems or issues, coding 268 segments as challenges.is process was supervised by all authors, with edge cases debated among them.Aer initial coding, two interpretation workshops were held.For a granular overview of challenges, they were distributed according to the SWEBOK Knowledge Areas [42] identified in the literature analysis of SE4AI meta-reviews (see Table 2 and [17]).We obtained multiple overlapping statements for all categories (except Training and Testing Data) confirming the saturation.
During interpretation workshops, the authors observed that freelancers differentiated between challenges that occurred because of the GenAI technology and because of the hype around it.For instance, freelancers predicted that some challenges might resolve at the end of the hype.Others were claimed to result from the aributes of GenAI as technology.Aer further exploration, authors were able to discriminate between challenges that were associated with the novelty and fashionableness of Gen AI as two dimensions of hype, and paradigm and product as two dimensions of technology.All authors agreed that this differentiation contributes to the understanding of collected insights.Supplementary material [17] provides further information about the coding schema and the freelancers quoted below.

RESULTS
Freelancers perceive multiple challenges distributed across 11 KAs and 54 categories as presented in Table 2.We count at least 99 distinct challenges related to development of applications based on GenAI of different granularity.Importantly, rather than identifying objective challenges, we focus on freelancers' perceptions.We want to understand what they consider unique, amplified, or reinforced due to the technological aspects of GenAI or the hype.is focus allows to produce guidance and identify development areas which address specific needs of this professional group.We first present the findings along the dimensions of technology and hype.In 4.5, we summarize the findings and present Table 2 which systematizes the findings.

Characteristics of GenAI Paradigm
Freelancers associate many challenges with the unique nature of GenAI.While some issues may be common to AI in general, GenAI's specific characteristics and accessibility create unique obstacles.LLMs are trained on vast datasets to produce high-quality, versatile outputs.However, these characteristics makes it impossible to trace and explain the model's behavior or identify the sources of a specific, problematic output.Freelancers indicate that this issue is exacerbated by many LLM providers' lack of transparency about training data sources and fine-tuning processes.For instance, it is unclear whether the providers used manual correction for specific use cases.is uncertainty affects both the content and form of the model's response.Freelancers say, they can control output's content by limiting the model to provided context data in retrieval augmented generation tasks which rely on processing of contextual information delivered to the model together with the prompt.Yet the response can still vary in tone, style, and structure.For instance, it might have inconsistent output format (bullet list vs. flow text).Measures to limit variability, such as GPT's 'temperature' parameter, are probabilistic, so freelancers report to observe different output for the same prompt even if temperature is set to minimum.e opaque relationship between a prompt and the output requires freelancers to invest significant time in trial-and-error process for craing prompts:

I guess one of the most time-consuming things is to make it work. AI is a bit random and prompt engineering quite takes some time. And at that moment when you're like, 'okay, it works', some unstudied questions or something, you didn't try, [comes up] and you suddenly figure out that it doesn't produce results you wanted. (va-19-3) 5
A significant issue discussed by many freelancers are hallucinations.When prompted, GenAI models will aempt to generate an answer, even without relevant knowledge about the topic.is can result in incorrect or nonsensical text or images.Some freelancers feel uncomfortable delivering products without a guarantee of consistent, correct behavior.A seasoned freelancer explains that his dedication to high-quality products makes him the slowest part of the development process:

Well, the hallucinations, if the AI starts going off in creating information because one asked for it (...). Sometimes it's amusing, but other times it's like, 'Oh my God!' (...) It's lies. It's things that it's made up. There's no truthful basis to [check it] it, especially involving people. (...) So, I'm the slowest part in the process because I don't trust the AI. (...) So, I have to read everything and validate what it's doing. (...) Because, if you just take the output of the AI and trust it, it's going to bite you. (cl-55-15)
Output hallucinations necessitate the establishment of guardrails, which may include filters or temperature reduction.However, this can limit GenAI's creativity which is desired for some use cases.A freelancer suggests that safety measures may conflict with client expectations, especially if the laer are unaware of GenAI's potential for hallucination: <experience>).For a detailed information about the participants, including the authors of those statements, see supplemental material (2.1.4-2.1.5).

[There is] the idea of safety and kind of guardrails. (...) It's kind of a push-and-pull with certain clients. Because they say, 'oh, the text looks boring'. And, in my mind, I'm thinking, 'yeah, but it's better than the text looking crazy'. (...) Just because everything's looking good right now, doesn't mean things can't go off the edge. And, maybe, clients don't un-
derstand that, because, if you're doing a good job, they're never seeing it go off the edge.So, they don't know that the edge is there.(ev-38-13) Freelancers mention further challenges related to GenAI appearing during SW construction.Retrieval augmented generation applications rely on clients' data which provide context for LLMs to generate the answer.e intricate nature of LLMs, coupled with complex architecture and data, makes it exceedingly difficult for developers to identify the cause of incorrect or imprecise responses.A developer describes this issue as follows: For me, it's always about the data.(…) You get the data, you pump it in Pinecone and all that, and then start querying that.You realize at some point whatever the bot is giving you back is not as accurate as you wanted it to be.(...) Now you start asking yourself: is it the problem with the system, the workflow, the Pinecone, and querying setup problem?Is it a little problem?Or now you have to start thinking what could be the problem?So, for me, the data is the first step.(do-  One of the consequences of these challenges is the emergence of downstream compatibility issues, which many freelancers find novel.e lack of control over the output, including its format, complicates soware design.Freelancers must accommodate for variability through architectural decisions and focus on the interdependencies between components.is situation exacerbates reproducibility issues, rendering soware testing less reliable and quality assurance more difficult.Furthermore, freelancers observe that large language models (LLMs) provide a set of interconnected advanced capabilities (e.g., evaluating and responding to a leer), making modularization unachievable.is complexity is a departure from earlier AI models, which were trained for individual capabilities.e opacity and scale of LLMs and other general AI technologies, combined with the absence of explanatory mechanisms, heighten the necessity for a trial-and-error approach.
Some interviewees consider training or fine-tuning as potential solution to some of the problems.Yet, they link the GenAI paradigm with enhanced expenses, high computational demands, and need for large volume of data beyond what was needed for earlier ML models.Only one interviewee mentioned having trained a model and claimed its cost higher than 10'000 USD for a single training round.For most freelancers such expenses are beyond the project budgets.is is a new aspect to those who worked with ML before and were used to train and re-train smaller models.
However, freelancers also indicate that available LLMs paradigm lessened the data and domain dependency.ey speak of LLMs being able to deal with various data formats and structures for retrieval augmented generation.Also, they emphasize that progress they achieved in one project (e.g., prompt paerns) can be transferred easily to a project in a different domain, thanks to the generality of LLMs.is reduces pressure on SW construction and is ideal for freelancers who frequently move between projects.

Characteristics of GenAI Products
Many providers offer GenAI as a product or service, supplemented with APIs.is is the primary delivery method for LLMs.Despite offering easy access and a range of tools, freelancers report these models can also be unpredictively expensive.Providers adopt a pay-per-use strategy, where the cost per request is determined by the length of the prompt or response.is makes it challenging to estimate running costs for applications, particularly when user inputs are factored into the original prompt designed by the developer.is uncertainty also complicates budgeting for developers, as they struggle to gauge the economic resources required for the trial-and-error process and subsequent application testing.is complicates the SW configuration management and SE management.One freelancer elaborates on these budgeting challenges: Further, freelancers identify unique technical constraints in LLMs which impact SW construction and maintenance.A commonly cited issue is the prompt or context window's size limits.ere are strict restrictions on the number of tokens a single prompt can deliver (for example, 16,000 for GPT4), which might be insufficient.A freelancer who designed a search and summary engine for a large dataset explains this as follows: It is very important to find knowledge in the huge amount of data and you can take, for example, 1010 pages of a form with needed data and summarization for these ten pages.But for now, GPT can utilize only two pages and it is the limit.(eu-  Soware maintenance is hindered by the need to frequently adapt to model changes, a factor beyond the control of freelancers who depend on proprietary, closed-source models.Providers oen update these models without transparency, leading to daily performance changes.is raises concerns about the consistency of model outputs, the predictability of application usefulness, and the need for ongoing re-validation of prompts and applications.is uncertainty also limits freelancers' project selection as they prefer to undertake projects, they can successfully deliver to garner positive client feedback and improve their completion rate.To mitigate this, freelancers engage in preselection activities like testing different prompts or a subset of client data before committing.However, their preselection decisions can be invalidated within days due to model changes, posing challenges due to increased dependency on external providers.Freelancers also worry about quality assurance, as they lack information on whether the external models are quality-assured, particularly in terms of security or privacy.e absence of this information from providers leaves freelancers' clients with doubts.
Despite obstacles, freelancers note that the emergence of robust foundational models has revolutionized their approach to delivering intelligent features.Previously, access to large, high-quality data sets, particularly in smaller projects, posed a significant hurdle in producing satisfactory products like free text chatbots.e introduction of LLMs has alleviated the strain associated with training and testing data, a key issue in previous AI generations.

GenAI Hype as Source of Challenges
Novelty of GenAI e novelty of GenAI adds further challenges.Novelty amplifies and reinforces challenges related to unrealistic expectations from the clients and their limited knowledge.Lacking experience and reference projects, clients do not have a realistic grasp of GenAI's capabilities and do not know what developers can or cannot do.
is imposes extra effort on freelancers in the project's initial phase: instead of focusing on work, they must spend time elucidating and demonstrating GenAI's limitations which demands more communication with the clients and demands new explanation skills.A freelancer notes that clients sometimes question whether the problems depend on the technology or the freelancer.
In the end, this is a source of issues for me because they [clients]  Another issue is the absence of a supportive community.While many freelancers appreciate the wealth of resources like courses, videos, and blogs, some struggle to locate the right community.As the GenAI becomes more popular and use cases solidify, this is expected to shi.is situation puts additional pressure on soware engineering practices.e absence of a shared knowledge base and reference communities can hinder soware development.A freelancer and start-up founder, who creates GenAIbased gaming applications, makes the following comparison: e unique nature of GenAI amplifies the challenge of unpredictable results, necessitating a trial-and-error method.With scarce experience, intuition, and lack of reference communities, freelancers struggle to assess a client's vision's feasibility.ey experiment, boosting their initial, unpaid workload and stress.Given that clients choose freelancers based on early interactions, freelancers might end up working extended hours for no gain.
Freelancers concur that certain challenges may diminish over time.Initially, they anticipate accumulating knowledge that will simplify choosing appropriate clients and projects, confidently communicating with them, and establishing connections with reference communities.ey believe that emerging metrics and quality assurance measures will validate their work's robustness.Secondly, they trust that potential clients will beer understand their expectations and how to articulate their projects.Lastly, they predict that new tools, processes, orchestration frameworks, and additional resources will be introduced to aid in the development of applications and systems with GenAI components.

Fashionableness of GenAI
e fashionable character of GenAI developments present significant challenges.Older GenAI approaches like GANs, while available for around 10 years [24], have remained niche, used for specific cases.However, the arrival of GenAI relying on large, pre-trained models that accepts natural input has sparked new interest.Many companies are eager to utilize its seemingly unlimited capabilities hastily, oen overlooking cheaper or simpler alternatives and not adequately examining their business needs.ey are uncertain about their desired outcomes, making it challenging for them to define the metrics for evaluations.ey also mistakenly assume that domain-dependent applications can be constructed without domain-specific data, expecting that LLMs possess access to all conceivable knowledge.An interviewee from a freelance agency emphasized that the hype exacerbates the problem of insufficient and unrealistic expectations, increasing pressure on processes associated with soware requirements engineering and customer communication: I think the biggest challenge from a non-technical perspective is understanding the use case.Many times, people would rush into [GenAI], 'Yeah, I want to chat about something.I want a generative AI solution.',but their problem can be solved with a much simpler solution.And, I think, it's really about understanding the problem before jumping into the solution instead of starting with the solution and trying to find the problem that fits for it.(ad-30-10) However, some clients limit access to end-users and other resources necessary for requirements engineering driven by the intention to obtain their solution in the shortest time possible.is has implications for subsequent phases of the SE process and might result in dissatisfied clients and, consequently, freelancers' low ranking.erefore, some freelancers try to avoid potential clients who appear too much hype driven.
e surge in interest has additional implications.e rise in requests puts freelancers in a position of choice.is demand and accessibility of GenAI capabilities also entices less-experienced developers to enter the market.On the one hand, it increases the price competition as new freelancers try to aract clients through low prices.On the other hand, new freelance developers frequently lack the necessary skills to produce high-quality SW, but the high demand assures that they will be booked for projects.Some freelancers take on projects previously developed by others.An interviewee explains how he had to navigate these issues: The work done on LangChain and OpenAI on Upwork is by developers that are not as experienced.Oftentimes clients end up with spaghetti code or some issue that they struggle with.(...) They got one file of Python, maybe a thousand lines of code, something like this.And I sat down, and I refactored it, and then I split into 6 or 7 files.(jf-  is observation pertains to anti-paerns previously identified in standard SE and non-generative ML.Yet, the hype tends to draw freelancers who perpetuate these anti-paerns.ese become problematic when there is a need to expand the application or assume responsibility for its maintenance and updates.
e ecosystem around generative models is incredibly complex and dynamic, with new solutions constantly emerging.is enhances pressure on freelancers, who oen find their newly created solutions are rendered obsolete by new off-the-shelf tools, frameworks, or APIs.Moreover, they struggle to keep pace with the frequent technological changes.While upskilling is crucial for success, the unprecedented rate of change demands daily effort to stay updated and flexibility to adopt new solutions quickly.e dynamics underscores the importance of sensemaking and staying updated.Even seasoned freelancers perceive this rate of change as something distinct to the buzz around GenAI.An experienced developer illustrates how this influences his daily routine: We're right on the edge of trying to keep up.And especially if you're looking at multiple solutions like I do, not just OpenAI, but Google and then, IBM, -it's a lot.There's so much coming out.And then, the cloud vendors and supporting vendors, they're releasing like crazy.And then, all of their AI-integrated products are hitting the market almost every day.(...).So, I'm still trying to catch up on their press release.And then boom, here comes AWS.You know, it's easily a couple of hours a day just trying to stay current with everybody.(...) In the early morning, I go through my LinkedIn connections, and then go through my emails, and my calendar.And then around noon I'll repeat that.And then just before dinner.And then, sometime between 10 and 11 at night.(cl-  e rising GenAI trend increases pressure on freelancers, leading to issues in requirements engineering, professional practice, and SE management.Freelancers must skillfully manage their clients and invest significant effort to match technology with client needs.Even experienced freelancers find this hype novel and unprecedented as it magnifies older challenges and yields new ones.

Summary
Freelancers face many struggles when developing GenAI-based applications.Many appear new to them.Other seem aggravated compared to ML or their previous experience.However, freelancers also indicate that some old challenges get less important.
Based on the opinions collected from the freelancers and the literature covering SE4AI / SE4ML challenges, we compiled Table 2 to summarize our results and compare them against earlier studies.Literature refers to general challenges of developers while our results present freelancers' standpoints.Specifically, columns three and four in the table reflect statements from our data and our interpretation thereof.We use the last column in the table to indicate factors with primary influence on the challenges subsumed under a specific category.For instance, freelancers indicated fashionableness and novelty as major factors amplifying and aggravating challenges related to customer expectations, which we indicate with ↑. e star, ★, points to categories for which freelancers identified unique or new challenges.↗ indicate which factors contribute to upholding challenges regarding GenAI in the category.For instance, novelty of GenAI results in clients' lack of knowledge about its capabilities.While freelancers mention this aspect as important, from their perspective similar problems were occurring earlier with ML. ↘ informs when a factor lessened the impact of challenges previously reported for AI/ML.is pertains especially to novel technological characteristics of LLMs which resolve problems of earlier ML-based solutions.
Table 2. Comparison between challenges and needs reported in SE4AI literature and challenges and needs expressed by freelancers in our study along with indication of sources sorted by SWEBOK KAs and identified subcategories.See supplementary material [17] for the full version of this table with further distinction between singular challenges' impact factors.

Software Design
Components Orchestration • difficult dependencies between all parts of MLbased system, • additional complexity due to distributed architecture, • hard-to-manage interactions between ML models, • orchestration was very difficult after release of some LLMs due to their novelty but lessened with new orchestration frameworks, • downstream compatibility is hard to achieve and maintain due to nondeterminism of GenAI's output format

Complexity of ML
• changing anything changes everything, • paradigm shift to pipeline-driven / system-wide view, • entanglement created by ML models, • abstraction boundary erosion

DISCUSSION
Freelancers report on a multitude of challenges they experience when developing applications including GenAI components.ey are surprised by the magnitude of some challenges, e.g., issues related to stay on top of changes, while others appear completely new to them, e.g.impracticality of separating functionalities.
Given the increasing role of freelance developers in digital transformation [25][26][27][28][29]39], their insights are invaluable to the broader community.Simultaneously, the results provide information about freelancers and their work, such as the importance of the adequate projects' selection.In the following, we interpret our findings and suggest avenues of research for SE4GenAI and for HypeSE.

Software Engineering for Generative AI
What does GenAI imply for freelancers?Freelancers suggest that GenAI equips them to develop potent applications within a constrained timeframe.Rather than engaging in expensive and lengthy processes of data collection, preprocessing, and training, they delve into prompts that can be efficiently handled by an individual.Freelancers from non-technical backgrounds express that GenAI has unlocked new avenues for them, fostering opportunities for economic and professional growth.ey leverage freelance jobs to enhance their skills and render their freelance career more sustainable.However, they require practical guidance, illustrative examples, and consistent interaction to achieve this.
e freelancers would have appreciated more structured guidance on GenAI technology, both as a concept and a product.ey reported unexpected issues stemming from hallucinations, challenges in handling context data during prompting, and complications associated with prompt engineering.eir pursuit of solutions and understanding of these phenomena across the Internet was oen messy, leading to a disorganized process and outcome.Our own research indicates that comprehensive, professional guidance on potential challenges is in its infancy and fragmented across platforms and communities.[30, 34-36, 60, 64].Freelancers' ability to improvise becomes a core asset in this situation.Short trial-and-error typical for working with GenAI turns improvisation into the actual value offering of freelancers.is, however, comes not without challenges for SE.
Freelancers are particularly affected by challenges related to characteristics of GenAI paradigm during SW design, construction, testing, and quality assurance due to hallucinations, low reproducibility, reasoning errors, inherently probabilistic output, and the CACE principle.Also, insufficient explainability leads to issues in contact with customers affecting freelancers' professional practice.Whereas the technical challenges are known [30,34], our results documents their impact on day-to-day SE practice of freelancers.
e characteristics of GenAI products influence SW construction, SW maintenance, SW configuration management, and SE management.Factors influencing this situation include the constant changes and opaque update cycles of leading providers like OpenAI, the availability and latency of models (including not just inference latency, but also latencies from workload distribution), cost structures and policies enforced by providers, and the minimal control over model behavior.Currently, freelancers do not view any significant alternatives to these major providers due to the restricted capabilities of open-source models or the prohibitive costs of fine-tuning and training.is indicates that the question of whether GPT is all one needs [64] will remain valid for the years to come.e dependency on external providers and their products was a marginal issue in earlier SE4AI literature [23,42].Yet, the fact that training and hosting own GenAI models is frequently impracticable, the issue of managing external dependencies rises.
Based on freelancers' reports, we propose a new area of research, Soware Engineering for Generative AI (SE4GenAI).We propose the following key components that SE4GenAI must incorporate: (1) systematic guidance for activities like managing the trial-and-error loop as the primary mode in making progress during the SW construction; (2) strategies and metrics for reliable testing, quality assurance, and downstream compatibility to handle GenAI's variable outputs and dynamic capability changes, including ways to make adequate guarantees concerning functionality; (3) guidance on dealing with external dependencies impacting functional and non-functional aspects of the system during maintenance as well as the cost structure of the product; (4) support for more effective explanation practices in communication with the clients about (at least currently) unexplainable and unpredictable model behavior.Equipped with that knowledge, freelancers in our study would have had a beer chance to complete their products more efficiently.

Hype-Induced Software Engineering
What does hype mean for freelancers?e escalating excitement around GenAI has significantly influenced the dynamics of the freelance market.Our data clearly indicates that the success of freelancers is heavily dependent on job platforms and their respective communities.To maintain their success, freelancers must master the art of navigating these platforms, which may sometimes involve declining projects of personal interest if the likelihood of success seems uncertain.In this era of heightened interest, the key strengths of freelancers are their autonomy, agility, and flexibility.Particularly for those who work on short, time-bound projects, they can swily adapt to new technologies and capitalize on the current buzz.However, this also means that freelancers must be able to acquire new skills quickly and independently, a process that demands both time and effort.
e integration of GenAI and the escalating demand for developers has presented new challenges that have affected freelancers, their work routines, and even their personal lives.We hypothesize that the rapid advancement of technology will likely generate similar trends in the future, and the impact of these trends on the IT development sector will extend beyond freelancing.is could potentially disrupt many established SE practices and guidelines.We argue that the strategies employed by experienced freelancers to navigate the hype provides valuable insights for SE research.Concurrently, formalizing this knowledge about navigating trends could be beneficial for new freelancers seeking guidance.
Fashionableness of GenAI is primarily reflected in freelancers' comments concerning new or amplified challenges on SW requirements, SE professional practice, and SE management.e hype incites unexperienced clients and unexperienced, low-skilled freelancers to enter the market.Clients come with exacerbated, misinformed expectations and unclear or hard-to-measure targets.Frequently, they are simply interested in solution-based probing [9] to discover new opportunities.Unexperienced freelancers, even if trained as ML specialists or soware developers, lack tools and skills to interact with such clients in a professional way.
Freelancers are now operating in a highly competitive environment that necessitates careful project selection and intense precontract solution testing.is challenge, which has not been addressed in previous SE4AI research [42]or SE hype literature [19], is substantial.We hypothesize that this will lead to a reciprocal effect on the client side, particularly among inexperienced businesses and startups.ese entities must undertake a complex search for suitable talent, prompting freelancers to engage in resume-driven development to establish credibility [19,20].e dynamics of GenAI hype differ from previous trends such as blockchain.For instance, the development of smart contracts necessitated extensive client-engaged requirements engineering [14,49], which made the freelancers' work transparent and accountable, compelling the client to contemplate their objectives.Moreover, blockchain's use cases were primarily confined to finance and value capture [12,14,41,43,49], whereas GenAI encompasses a wider range of applications.is compels freelancers to rapidly acquire the requisite domain expertise.
Novelty of GenAI further amplifies effects created by the GenAI's fashionable character concerning SW requirements, SE management, and SE professional practice.It increases planning uncertainty due to lack of heuristics, reference projects, and reference communities, which makes the overall situation harder.e novelty encourages ad-hoc efforts and quick fixes, anti-paerns in SW design, and inefficient communication in SE practice.
Considering the likelihood of future hypes, we advocate for research in Hype-Induced Soware Engineering (HypeSE).Freelance and independent developers, due to their flexibility and agility, are most impacted by hypes and require support.HypeSE should incorporate: (1) guidance on managing inflated expectations and communicating these to clients and users, extending existing literature on requirements elicitation and management; (2) processes and tools for prioritizing and managing project opportunities to support economic stability and mitigate risks associated with riding high waves; (3) support to help developers manage incoming hype information, including fostering effective community building and facilitating expert information exchange; (4) definition of new roles such as domain-specialist freelancers who provide in-depth insight and consultancy to clients based on expertise collected in similar projects assuring diffusion of knowledge between client companies and enabling them to spot opportunities for clients.To accommodate for such new roles, we claim CS and SE education should focus on equipping future developers with specific domain skills rather than producing domain-agnostic generalists.is could foster directions like SE for the financial industry, life sciences, or social media.

Generalizability and Threats to Validity
While this study highlights the need for SE4GenAI and HypeSE based on freelancer experiences, we propose that other stakeholder groups and professions may face similar, related, or complementary challenges.Further research is required to comprehend these challenges, emphasizing the unique aspects of each stakeholder group.For example, issues related to imprecise or exaggerated soware requirements may be interpreted differently by clients, contractors, and requirements engineers.Addressing these problems necessitates understanding the mechanisms that produce these challenges for each group.is viewpoint would supplement the currently prevalent approach, which aempts to aggregate soware engineering challenges and their solutions at the technology level, such as GenAI [18,30,34,63,64] or AI/ML [23,40,42,44,47].We recognize existing efforts to concentrate on developers as a stakeholder group [59] and specific studies focusing on particular areas of SE, including requirements engineering or testing [32,48].We advocate for the amplification of these efforts by incorporating the perspectives and specific contexts of project managers, client companies, ML model providers, and so on.
Interview studies risk low interpretation validity due to suggestive questioning, personal bias, and coder assumptions.We mitigated this by allowing interviewees to begin with an open narrative, capturing extensive information without a specific focus.An uninvolved researcher coded the data to minimize bias.We maintained a theory-agnostic approach to avoid guiding interviewees or analysis.However, we acknowledge the first author's prior involvement in AI-based studies may have influenced him.is study's internal validity may be influenced by context and respondent biases, as it relies on reports potentially affected by recent events or second-hand reflections.e external validity could be impaired due to the nascent nature of GenAI, which is yet to stabilize.We mitigate this by distinguishing challenges linked to GenAI's characteristics from those with different origins.Additionally, we sought to improve triangulation by interviewing a diverse group of developers.

CONCLUSION AND FUTURE WORK
is study was conducted with the objective of pinpointing the challenges freelancers face in relation to GenAI.Freelancers, who serve as ideal collaborators for small to mid-sized enterprises exploring new technologies, provided insights into the challenges they perceive as new or significantly heightened due to the hype surrounding GenAI, as well as GenAI's inherent and product-related features.Freelancers have observed a shi in workload from data management and model training to model interaction and output management, which complicates their ability to rely on previous intuition and experience.Moreover, the hype around GenAI contributes to problems arising from overblown expectations and swi changes that demand the freelancer's aention.is situation parallels notable historical events such as the gold rush or the exploration of the wild west, where independent pioneers ventured into uncharted territories.
We believe the SE community should monitor future developments in this field.e findings of this study could be reinforced through additional qualitative research or large-scale confirmatory surveys.Ethnographic studies based on in-situ observation of larger development teams could offer insights into how freelancers tackle practical challenges and uncover unexplored issues.We also recommend normative and prescriptive research to develop new standards and techniques for soware engineers and managers.
e study provides practitioners with insights into the challenges of SE for GenAI-based solutions.e provided list of SE challenges is thought to sensitize them for specific issues to emerge.e study also educates practitioners about the impact and risks of hype.Especially freelancers depending on external contracts are subject to those risks.SE researchers gain an empirical basis for deriving guidance for practitioners and become more aware of hype as an inherent element in SE.Its influence extends beyond the context of GenAI and directly affects SE practice.We suggest addressing these challenges through intensified research on SE4GenAI and HypeSE.

Extended Version of Table 2
The table below is an extended version of Table 2 from the manuscript.It includes subcategories which were identified in the literature but were either confirmed by freelancers without further deliberation (↔) or no comments were made by freelancers on the given category of challenges (◇).The star, ★, points to categories for which freelancers identified unique or new challenges.↗ indicate which factors contribute to upholding challenges regarding GenAI in the category.↘ informs when a factor lessened the impact of challenges previously reported for AI/ML.↑ indicates factors that significantly amplify challenges belonging to the given category.), but some basic information about the functionality might be necessary for me to understand the use case.
These are the selection criteria for interview participants: -software developer or software project manager with 2 years of experience or more, -participated in 2 or more development projects involving the use of generative AI platforms / APIs, -the developed software uses, processes, or interprets the output from the API (i.e., at least one of your projects/apps should be more than simply an adapted/extended version of the OpenAI's playground).

Your tasks involve:
-filling out a short survey (5-10 min) prior to the interview including an informed consent for participation, -participating in an online interview via zoom (approx.45-50 min).
Each participant will be paid 35 USD after the completion of the interview.
The dates for the interview will be arranged individually.The interviews will be audio-recorded.Later on, a research team will anonymize, transcribe, and analyse them.
If you think you're the right participant for the interview, drop me a line.

Interview Guide
After the survey, participants were provided access to a calendar in which they could book an appointment that suits their time schedule.After booking the appointment, they received access to the interview guide so they could see it before the interview (but they were not requested to do so).
Interview "Developing Apps that use Generative AI" The interview will be conducted in a semi-structured manner.It means, I will try to address most of the topics listed below.However, we might change the order of the questions depending on how the conversation develops.Some of the questions might not perfectly fit your specific situation or experience.Do not worry.We will figure out how to go about it.
For example, if you have any trouble with the Unity [a game engine platform], I'm sure you can point to the Unity Forum, and someone asked these questions.(...) I'm sure there are many little communities [around GenAI].(...) There are a lot of people [engaged in them], but I'm not sure how many people worked in Unity integration with the generative AI and how can we find each other?I don't know.(tx-30-6)

(-
DE -DALL-E, MJ -Midjourney, SD -Stable Di:usion, custom -custom self-trained models) -Two exemplary use cases for which the interviewee developed solutions ab 38 5 2 MSc Economics Serbia GPT, LLaMA, Vicuna -Classifying SaaS websites with GPT and Vicuna -Investment and trading prediction using GPT and LLaMA ad 30 4 10 MSc ML / AI USA GPT, LaMDA, Others -AI sales assistant based on fine-tuned GPT -Recommendation chatbot for retail based on fine-tuned GPT cl 55 2 15 BSc CS USA GPT, MJ, PaLM, SD -Generative shoe design with text-to-image models -Designing Kubernetes cloud environment using GPT and Bard/LaMDA do 31 5 5 MSc CS Kenya DE, GPT, LLaMA -Text summarization of demand letters in financial industry using GPT -API for Q&A bots for text-to-text and table-to-text scnearios eu 35 4 15 MSc IT Mngmt.Serbia GPT, LLaMA, Vicuna -Q&A chatbot for local knowledge base based on GPT -Collection of news on a topic and providing summary -GPT ev 38 6 13 MSc CS France GPT, MJ, Others -Customer support chatbot for home appliances based on GPT -Integrating Midjourney output in an AR application in luxury industry ia 28 3 5 PhD Physics Romania GPT, MJ -Summarization of market research reports based on GPT -Summarization of output obtained from an SQL database using GPT jf 23 5 5 BSc CS Romania GPT -Customer support assistant using dynamically scrapped content, uses GPT -Q&A chatbot as a medical assistant based on GPT ml 30 4 5 PhD ML / AI Pakistan GPT, LLaMA, SD, LaMDA -Customized medical prescription recommendation based on GPT -Modules for search and Q&A for regulatory documents based on GPT ss 30 5 5 MSc Math.Pakistan DE, GPT, SD, custom -Generation of videos with avatars reading input text, with various models -Mobile guidance app using video input and output with custom model tx Generation of Game Design Documents for mobile games based on GPT -Instant game content generation by player with voice based based on GPT va 19 3 3 Stud.CS Ukraine DE, GPT, SD, Others -Teaching assistant Q&A-bot providing sources, images, and summaries -Generating a multi-perspective, multi-opinion newsletter with GPT Page: A-10

Table 1 .
Descriptive statistics regarding study participants You have a budget limit.So, you can't spend all the client's budget by doing a lot of prompts.You have to write it and to check it, but you don't have a lot of opportunities to rewrite, especially for big data.It's also a very difficult question and difficult issue because (...) if you if you have a coin and have 50 times tails, it doesn't mean that after the next one, you will get a tail.So, it's the same with GPT results because if you a dataset with 10 million [documents] and you will check manually 100 results, it doesn't mean everything was okay.(...) It's a kind of tradeoff because there are the limits, the time, and the money.And you can spend one of them, but you always have to check on the other.(ab-38-2) Many interviewees cite misinformation in public or social media as a problem.Clients oen develop unrealistic expectations from such misleading information about what GenAI can achieve and how quickly.A developer, who is holds a PhD in computer science and has extensive experience with ML, explains:I got a man who watched a video on YouTube, came to me, and said 'We need exactly this'.I said 'The thing you are referring is like a work of dozens of people for 3 to 6 months.And you want me to build this solution within seven days.So, this is not possible.'Buthe was insisting me to: 'If they can do this, then you must do too'.(ml-30-5) Job Description Posted to Upwork I am looking for developers who create software applications involving intelligent features based on generative AI (including any of the recent large language models): it might involve web apps, mobile apps, plug-ins or packages towards bigger projects.I want to learn about problems and challengesyou encounter in your daily work with GPT, LLaMA, LaMDA, and others, their APIs, or your own generative models, how you solve those challenges, and what would you wish for in the future.I do not need any sensitive information (employer, name of the project/app, country, private addresses, etc. 2.1.2Question Asked in the Proposal Phase 1.What generative AI platforms or APIs did you use in your past projects?2. How many years of experience as software developer do you have?3. What is your highest training / education degree? 4. How many from your recent projects used a generative AI platform or API? 2.1.3Conflict of Interests Declaration The authors, unaailiated with Upwork Inc., did not receive any benefits from the company or its subsidiaries at any time.The choice was based on specific criteria: (1) the ability to post a job with a fixed rate, to ensure equal pay for all freelancers, (2) detailed search criteria, including GenAI-related topics, (3) a long-standing presence with experienced freelancers, (4) a clear, consistent contract price, to fit University's study participant reimbursement rules.We consulted multiple online resources, e.g., arc.dev/employer-blog/toptal-upwork-fiverr-arc/, www.g2.com/categories/freelanceplatforms?attributes[221]=420#grid Page: A-9 2.1.5Quoted Participants This list provides more information about the participants quoted in the manuscript.
2.2 Data Collection 2.2.1 Survey Questions Survey was hosted on a LimeSurvey server of the university.Participants received invitation to the survey after accepting the contract on Upwork.All communication with the participants was happening through Upwork.In what projects and for what did you use the tools?(open question) Provide two or more examples.A brief description is enough, e.g., "Connected to GPT to create ideas on a given topic in a brainstorming app." or "Used Stable DiBusion to generate graphical summaries of daily news for a news app."If you want, feel free to paste external links.Please, give preference to projects or usage scenarios which challenged you most.3. What is the biggest challenge when creating apps with generative AI capabilities?(open question) 4. What is the biggest reward from integrating generative AI capabilities in apps?(open question)