SOFTWARE Editor: Christof Ebert

Vector Consulting Services

TECHNOLOGY christof.eber t@vector.com

Generative AI for
Software Practitioners
Christof Ebert and Panos Louridas

From the Editor

Generative artificial intelligence (AI) tools, such as Bard, ChatGPT, and CoPilot, have
rapidly gained widespread usage. They also have the potential to boost software en-
gineering productivity. In this article, we elaborate technologies and usage of genera-
tive AI in the software industry. We address questions, such as: How does generative
AI improve software productivity? How to connect generative AI to software develop-
ment, and what are the risks? Which technologies have what sorts of benefits? Practi-
tioner guidance and case studies are shared from our industry context. I look forward
to hearing from you about this column and the technologies that matter most for your
work.— Christof Ebert

GENERATIVE ARTIFICIAL INTEL-

LIGENCE (AI) has the potential to
change the software profession more
than any other recent technology.
Bill Gates sees it as the biggest move
forward since the invention of the
Internet. It can improve software
©SHUTTERSTOCK.COM/DEEMERWHA STUDIO

productivity in several ways, such

as automating repetitive tasks (e.g.,
testing or requirements traceability),
improve software quality by creat-
ing test suites from requirements,
and automate workflows by rout-
ing work products to the next suit-
able step in a production pipeline.
At the same time, generative AI cre- IPR and cybersecurity are prominent
Digital Object Identifier 10.1109/MS.2023.3265877
ates fully new risks because it is nei- examples that limit usage in profes-
Date of current version: 14 July 2023 ther deterministic nor explainable. sional software engineering.

30 useI Elimited
Authorized licensed E E S to:
O FDr.
T WD.
A Y. | PEducational
R EPatil U B L I S H E Complex
D BY TH E I E EDownloaded
Akurdi. E COMPUT onE July
R SO  05:44:55 UTC from IEEE Xplore.
C I E T Yat
23,2024 0 7 4 0 - 7Restrictions
4 5 9 / 2 3 © 2 0apply.
23IEEE
 SOFTWARE TECHNOLOGY

Generative AI Technologies To provide meaningful answers, gen- can be used to guide the AI in gener-
Generative AI has been around for erative AI undergoes further train- ating contextually relevant and well-
many years. With no means to prove ing based on human feedback. Many informed responses.
validity, researchers hesitated to bring human trainers pose questions and Underlying all this, generative AI
such technology to the mass market provide feedback on the generated an- is powered by large language models
of rather naïve data citizens. As we swers, rewarding good answers and (LLMs). As the name again suggests,
have observed many times in recent punishing unsatisfying ones. This kind these are large neural network models
IT history, the perceived gold rush
makes people close their eyes to ob-
vious risks. Even tools designed for
good will eventually have devastating The basic idea is to use a large
consequences. When ChatGPT was
finally released to a wide public audi- language corpus to train a neural
ence in 2022, the AI arms race started network to learn the language, by
at a speed never seen before. It took
just two months for ChatGPT to reach
hiding part of the text and asking the
100 million users. Figure 1 shows this network to guess the missing parts.
fast evolution for different technolo-
gies spanning a mere 100 years of
recent human history. A technology
like the wheel even took thousands of of reinforcement learning guides the that are trained on big language cor-
years to reach 100 million users. system toward providing more accu- pora. Technically, they have a trans-
For every developer, turning to rate answers, while guarding against former architecture, which is based on
StackOverflow or Google has been a harmful responses. This has led to a mechanism called attention. The pub-
natural part of the job for years now. glimpses of a new way of working, lication of the attention mechanism, by
Condemning the “not invented here where the focus is on “prompt engi- Google researchers, must now rank
syndrome” to the dustbin, our first neering”: find the most appropriate among the most influential papers in
reaction when in doubt how to code way to frame a question or a whole computer science.1 Two early LLMs
something has been to look it up on dialogue. Generative AI does not work were the Bidirectional Encoder Repre-
the Internet. Search engines have be- with individual question and answers: sentations from Transformers (BERT),
come better at indexing code reposito- it maintains a context window, which developed by Google in 2018,2 and
ries, myriads of which exist online, and
community advice sites, such as Stack-
Time to Reach 100 Million Users
Overflow, provide reasoned solutions
and valuable commentary on user ChatGPT 2
questions. What is common in search
engines and question-and-answer web- TikTok 9
sites is that you can look up informa-
WhatsApp 40
tion that has already been stored there.
Generative AI is different. As the Internet 80
name suggests, it can synthesize—or
generate—the answers to the ques- Mobile Phone 190
tions you pose. Instead of trawling a
Car 400
prefabricated answer as classic search
engines are doing, it will create an Telephone 900
answer for you. The answer is based
on vast amounts of data on which 0 100 200 300 400 500 600 700 800 900 1,000
it has been trained, such as those ar- FIGURE 1. Time to reach 100 million users for different technologies in months after
chived and indexed by search engines. initial deployment.

J U05:44:55
Authorized licensed use limited to: Dr. D. Y. Patil Educational Complex Akurdi. Downloaded on July 23,2024 at LY/A U G UST
UTC 2 3 |Xplore.
2 0IEEE
from I E E E Restrictions
S O F T WA R E
apply. 31
 SOFTWARE TECHNOLOGY

Generative Pretrained Transformer 1 word as its embedding. Once you can through an encoder, which is a series of
(GPT-1), developed by OpenAI, who represent the words in the appropriate attention mechanisms. Attention mech-
went on to develop subsequent GPT way, you can use these representations anisms are an algorithm used in LLMs
models, getting to GPT-4 today.3 to generate new material by trans- that enables the AI to focus on specific
The basic idea is to use a large lan- forming the representations to new parts of the input text when generating
guage corpus to train a neural net- words: for instance, the answer to a an output. The output of the encoder
work to learn the language, by hiding query. When we talk about language is a vector representation of the input,
part of the text and asking the net- and words, we are not restricted to which is produced by analyzing sur-
work to guess the missing parts.4 The human language: it can be computer rounding context and attentions. You
neural network does that by paying code, having code tokens instead of can think of the encoder’s output as the
selective attention to the words com- words; the idea is the same. meaning of the input, as understood
prising the surrounding context of the You can see a simplified depiction by the neural network. The meaning is
missing parts. The words themselves of the internals of an LLM in Figure 2. a vector, corresponding to a point in a
are represented as vectors, called em- The model follows the transformer ar- multidimensional space.
beddings, in a multidimensional space. chitecture.4 The inputs, which are lan- Once we have the encoded input,
In essence, the neural network learns guage or code tokens, are represented we need to transform it to the desired
and represents the meaning of each as vector embeddings. Then they go output; that is, take it from a vector
representation and transform it back
to a language or code token. To do this
Probabilities we feed it to another series of attention
mechanisms, the decoder. The output
of the decoder are candidate tokens,
which are then assigned probabilities.
Calculate The most probable token is the final
Probabilities
output. These probabilities result from
training the entire transformer model,
including both the encoder and decoder,
Encoder Decoder with vast amounts of text. ChatGPT is
Output Output said to be trained with the “entire inter-
net”. The training process is referred to
as self-supervised learning (or masked
language modeling), which is achieved
Attention Attention
by hiding some parts of known text and
checking the quality of how it is auto-
matically completed. In this way the de-
Decoder
Encoder

coder will learn to predict the missing

Attention Attention output given the encoded input. After
training, the model receives prompts or
Attention Attention
queries. Each prompt will be encoded
as before and then fed to the decoder.
This time the decoder works only with
the encoded input as we don’t have a
Input Output known output at hand. With adequate
Embedding Embedding training, the model will predict a useful
final output. Obviously with restricted
domains, such as code fragments or test
Inputs Outputs cases, the LLM needs less training.
Figure 2 corresponds to the b ­ asic
FIGURE 2. Generative AI technology. model and training procedure. As

32 useI Elimited
Authorized licensed E E Sto:
O FDr.
TW D.AY.
RE | Educational
Patil W W W. C O Complex
M P U T E Akurdi.
R . O R GDownloaded
/ S O F T W Aon | @
R EJuly I E E E Sat
23,2024 O05:44:55
F T W A RUTC
E from IEEE Xplore. Restrictions apply.
 SOFTWARE TECHNOLOGY

mentioned above, a vital part of gen- GPT-4 can generate code from doc- can respond to particular questions and
erative AI models is that once they strings and solve coding questions in act on them.
can produce sensible outputs from a software engineering interviews on a Code completion, at a line or whole
prompt, they can be further trained par with or surpassing human perfor- function level, is offered by Tabnine,
by having humans provide feedback mance. It can program for the front- which positions itself apart from the
to their output. The feedback is used
to fine-tune the model using rein-
forcement learning techniques.

Generative AI for Developers If anything, we can expect existing

Several generative software platforms
have been available in labs, which al- tools to improve and new tools to
low us to turn simple instructions into enter the scene.
computer code. GitHub’s Copilot is
available as an extension to develop-
ment tools and editors, such as Visual
Studio Code, Visual Studio, Neovim,
and JetBrain IDEs. It offers code auto- end and interact with LaTeX. It can competition by paying special atten-
complete that is powered by OpenAI reverse engineer code, execute Python tion to licensing and privacy issues.
Codex, a generative AI system devel- code, and execute pseudo code. Ope- Tabnine has been trained only on
oped by OpenAI. A tantalizing devel- nAI, the company behind GPT-4, offers open source software with permis-
opment is the recent announcement of programmatic access to its LLMs. That sive licenses. Moreover, it assures
Copilot X, powered by GPT-4, which means that developers can use them not developers that it does not retain
apart from improved autocompletion only in conversational manner, but also any of the code in which it is used,
can aid in other development tasks, embed them into their applications. It is and it is also possible to download
such as understanding code, improv- also possible to develop plug-ins, which and use the underlying models lo-
ing pull requests, and aiding in script- are ways to connect the underlying cally, instead of only being accessi-
ing and shell tools. models with third-party services that ble as a service.

Table 1. Generative AI technologies for code development

Name URL Technology Cost Use cases

ChatGPT https://chat.openai.com/ GPT-4 USD$20 per month for the chat Code completion, code generation,
interface; pricing for application code comprehension, reverse
programming interface use engineering, pseudo code reasoning
depends on usage. and execution.

CoPilot https://github.com/ OpenAI CodeX, USD$10 per month/USD$100 Code completion for CoPilot; CoPilot
features/copilot GPT-4 per year for individuals, $19 per X uses the more advanced GPT-4
user per month for business model and can answer questions
plans. based on code documentation; aid in
pull requests, shell commands, and
scripting.

Tabnine https://www.tabnine. Proprietary ML Basic tier is free, Pro tier starts Code completion. Runs also on private
com/ engine trained with at USD$12 month/user; also desktop to protect IPR.
OSS possible to self-host

Hugging Face https://huggingface.co/ Transformer, details Free and open source Code completion and code generation,
(various different docs/transformers/index vary depending on depending on the model.
models) the model

J U05:44:55
Authorized licensed use limited to: Dr. D. Y. Patil Educational Complex Akurdi. Downloaded on July 23,2024 at LY/A U G UST
UTC 2 3 |Xplore.
2 0IEEE
from I E E E Restrictions
S O F T WA R E
apply. 33
 SOFTWARE TECHNOLOGY

Table 1 summarizes some mainstream re-establishing traceability, through application programming

technologies. Note that the landscape explaining code, refactoring of interface (API) calls, in this way
changes very fast. If anything, we can legacy code, software mainte- integrating the use of LLMs with
expect existing tools to improve and nance with augmented guidance, our development pipeline.
new tools to enter the scene. and improving existing code. • Problem solving: Generative AI
has been shown to be able to
Generative AI for Software We will focus here on ­improving exhibit mathematical and algo-
Productivity software development and software rithmic abilities, though currently
Generative AI has the potential to sig- productivity with generative AI (See still weak, because the LLMs
nificantly improve software produc- “Generative AI Industry Case Study” were not trained to interpret the
tivity by automating tasks, enhancing for an application). meaning of numeric representa-
creativity, improving accuracy and ef- Even though we have focused on tions and complex relationships
ficiency, and streamlining development code-related tasks, the capabilities of between numbers. Programmers
processes. Application domains with a generative AI tools extend well beyond can describe a problem that needs
high business impact are the following: assisting in writing code. Generative to be solved and provide some
guidance on how to solve it (e.g.,
“use dynamic programming”) and
then the tool can generate code to
Generative AI can be used to the task. For instance, you provide
a requirement like: “Make a list of
streamline software development major taxation laws in country X
processes by automating tasks, in Python language” and you will
get a usable piece of code, includ-
such as testing, debugging, and ing well-named variables and doc-
deployment. umentation. For a wide variety of
problems, generative AI will spit
out a usable solution, especially
where commonly used algorithms
• media content creation, e.g., AI can improve software productiv- are involved. Also, here we have
text, audio, video, pictures, ity in various aspects of software engi- new challenges, namely the more
­content for news feed, and neering, such as the following: detailed and not trivial your
social media request, the higher the probabil-
• media content improvement, • Enhancing creativity: Generative ity that the code contains errors,
i.e., from making enhancements, AI can assist developers in gener- which you will not find as you
references (though mostly they ating new ideas and solutions for don’t know how to test excep-
hallucinate and invent references software and UX development. tion cases.
that do not exist), and explana- For example, it can help develop- • Efficient development: Genera-
tions up to augmented content ers generate new designs, logos, tive AI can be used to streamline
generation with pay per use and user interfaces. software development processes
• generative design, such as chip- • Summarizing documentation, by automating tasks, such as
design in semiconductor busi- reviews, interviews, meeting testing, debugging, and deploy-
ness, developing novel building minutes: LLMs are particularly ment. An example could be rout-
and city architectures, material good at these tasks, freeing time ing of tasks, such as a regression
design in chemical plants, inno- for other tasks. Generative AI test. This can help to reduce
vative drugs and medication in responds to prompts, and these development time and costs and
pharma industries prompts can be customized and improve overall productivity.
• software development, such predefined for various tasks, such • Maintaining legacy: Most soft-
as code generation, test case as automatic search and enhance- ware that is produced today
generation from requirements, ments. We can give the prompts is legacy or based on legacy.

34 useI Elimited
Authorized licensed E E Sto:
O FDr.
TW D.AY.
RE | Educational
Patil W W W. C O Complex
M P U T E Akurdi.
R . O R GDownloaded
/ S O F T W Aon | @
R EJuly I E E E Sat
23,2024 O05:44:55
F T W A RUTC
E from IEEE Xplore. Restrictions apply.
 SOFTWARE TECHNOLOGY

A lot of trusted safety-critical into any other language, e.g., a the selection of appropriate test
software in domains, such as Python code into JavaScript. In cases. This can help develop-
power plants or defense, is de- many cases, the software not ers to write more accurate and
cades old. Many federal systems only points out possible quality efficient code, and to identify
are based on Cobol and other problems, it also provides you and fix bugs more quickly. New
antiquated languages. The chal- with concrete alternatives. challenges arise, such as how to
lenge is increasingly to find • Improving software quality: ensure that AI-based systems
people able to maintain such a Generative AI can analyze large would not be validated by AI
legacy. Generative AI tools in amounts of data and identify systems that are programmed
the future might explain how patterns that human develop- to overlook certain defects or
the code works and translate it ers might miss. An example is backdoors. Deep fake applies to

GENERATIVE AI INDUSTRY CASE STUDY

At Vector we are often called to improve the quality of software issues, such as ambiguous or conflicting requirements.
systems. One typical finding is insufficient requirements and It can also help to identify missing requirements or
test strategy. Some software is tested several times, while inconsistencies in the requirements documentation.
some requirements and scenarios remain untested, making •• Predictive analytics: AI can be used to analyze
increasingly complex software systems impossible to trust. historical data and predict the likelihood of certain
While software development is simple, identifying the requirements being implemented successfully. This
right requirements and test cases is a challenge for practically can help to identify potential risks and prioritize
all companies. This is a high risk, especially when automating requirements accordingly.
critical systems, such as autonomous vehicles, medical •• Testing automation: AI can be used to automate test-
devices, and finance systems. Policy makers demand trusted ing processes and ensure that each requirement is
AI, which demands a clear specification of intended functional- properly tested and verified. This can help to reduce
ity, border cases, and clear demarcation lines of what must not the time and effort required for manual testing and
happen. With today’s level of requirements engineering and improve overall testing accuracy.
test methodology, we are far away from trusted AI-systems.
In such cases, requirements traceability ensures that The return of generative AI in connecting require-
requirements are properly tracked, verified, and validated ments engineering and testing comes in different curren-
throughout the development cycle. It is perceived as highly cies, namely less effort for maintaining traceability, higher
necessary and demanded by all standards along functional product quality by test case updates even across hetero-
safety and cybersecurity. However, in practice traceability geneous tool chains, and better understanding of complex
is not maintained, and most software systems today are systems. The risks of generative AI remain as mentioned
insufficiently tested.S1 Here are some ways in which AI can in the article. Be aware of your intellectual property rights
help with requirements traceability: and never upload software to external platforms. Do not
take results of generative AI as sufficient to automate
•• Automated tagging and categorization: AI can be quality checks, because these tools are neither determin-
used to automatically tag and categorize require- istic nor explainable in their chain-of-thought.
ments based on their type, priority, and other
characteristics. This can help to ensure that require- Reference
ments are properly tracked and easily searchable. S1. C. Ebert, D. Bajaj, and M. Weyrich, “Testing of software sys-
•• Natural language processing: AI can be used to analyze tems,” IEEE Softw., vol. 39, no. 4, pp. 8–17, Jul./Aug. 2022,
natural language requirements and identify potential doi: 10.1109/MS.2022.3166755.

J U05:44:55
Authorized licensed use limited to: Dr. D. Y. Patil Educational Complex Akurdi. Downloaded on July 23,2024 at LY/A U G UST
UTC 2 3 |Xplore.
2 0IEEE
from I E E E Restrictions
S O F T WA R E
apply. 35
 SOFTWARE TECHNOLOGY

software even more than only data developed by generative AI and what is completely made up for
pictures. around corner cases and critical misinformation. The answers and so-
• Improving data quality: An im- scenarios facilitate development, lutions are generated by models based
portant feature of using generative testing, approval, and homologa- on probabilities, not necessarily found
AI for software-related tasks is tion of automatic, robotic, and from some authoritative source. That
the ability to fine-tune an existing autonomous systems in critical means that they may be wrong. AI
model on specific data. LLMs industries, such as medical, aero- tools can hallucinate, responding in
are trained on open data that is space, mobility, and industrial a wildly erroneous manner while be-
trawled from the Internet, and production.5 ing supremely confident that they are
their answers are based on what right. Things are improving (GPT-4
they can learn from that data. It Hints for the Practitioner seems to be better than its predeces-
is possible, through appropriate While generative AI can help compa- sors), but the user should always check
APIs, to give to LLMs our own nies to grow competences, there are the answers. Relying on AI tools for
data. The typical steps are to several risks that need to be considered tasks where you cannot determine the
prepare our training data, then and mitigated.6 Technology companies correct answer or how to verify it can
upload them to the model and let and especially their venture-capitalist lead to complications and pitfalls. For
it train on that data, on top of its backers tend to just look for fast money software development, it means that
existing training. Then we used and repeat past mistakes, namely pri- human supervision and intervention is
the fine-tuned model, which will oritizing growth over safety. OpenAI’s necessary, such as reviews.
be able to provide more relevant 2020 predecessor to ChatGPT, for in- With statistically driven synthesis of
answers to our prompts, either stance, was known for “creative” out- results, generative AI does not have a
typed in or through API calls. puts, which were as easy to read and real understanding of language. More
• Achieving trust: While tra- use as Wikipedia entries but were in- dangerous is that it has no knowledge
ditional software is based on humane and racist.7 Google explicitly of the real world. The language model
predefined algorithms, current announced plans to release a premature produces its “facts” with nice-to-read
software is adaptive, self-chang- Bard, accepting the high risk it is will- text or code. But it is the user’s re-
ing, and learning. Such systems ing to take when releasing tools based sponsibility to verify these statements.
do not behave according to on AI technology. The lessons from so- Creativity and the capability to detect
initial specifications and might cial media should guide us in develop- defects are the most important charac-
even “unlearn” what they were ing AI. What is labeled social networks teristics that distinguish you and your
initially developed to do. It is not had over the past 10 years eroded true code from AI. Even if it is tempting to
meaningful to discuss the valida- social connections and trust between let an algorithm do as much work for
tion of AI systems without using people. With several hours per day on you as possible, you should always be
nearly realistic systems and con- these networks, mental-health and in- aware that it makes mistakes, which
texts. Testing nondeterministic telligence is declining at an alarming it admittedly packages very credibly.
systems is difficult with deter- pace. Societies in many countries are When users started posting bug fixes
ministic tests. With traditional deeply polarized due to “fake news,” generated with tools such as ChatGPT,
software testing, release criteria which is consumed without much think- StackOverflow banned such posts.
are based on comparing reactions ing about it. Tools such as ChatGPT How do they identify the fake con-
to a given series of inputs with could further replace professional in- tent? The same way a professor today
expected outputs. Simulations dependent media and spread fake news has to verify homework assignments,
of cyberphysical systems suffer that are neither traceable nor explain- namely by means of AI. It is an arms
from the enormous space of AI able. As developers we must get hands- race, and quite good tools are around
systems if not applied purpose- on and deal with such risks. to identify AI-generated documents
fully. For instance, autonomous The name generative AI cues the with statistical analyses.
vehicles need several hundred major pitfall. If humans rely on AI Practitioners should also be aware
million kilometers to statisti- for information, it will be increas- of privacy and security implications.
cally prove that they are suit- ingly difficult to tell what is factual, When using a tool that analyzes your
able for real traffic. Synthetic what is an exaggerating advertisement, code, you should be careful about

36 useI Elimited
Authorized licensed E E Sto:
O FDr.
TW D.AY.
RE | Educational
Patil W W W. C O Complex
M P U T E Akurdi.
R . O R GDownloaded
/ S O F T W Aon | @
R EJuly I E E E Sat
23,2024 O05:44:55
F T W A RUTC
E from IEEE Xplore. Restrictions apply.
 SOFTWARE TECHNOLOGY

what happens to that code. If your code

ABOUT THE AUTHORS

is open source, then it probably does
not matter that it may leave traces in
the tool’s models, or even be used as CHRISTOF EBERT is the managing director of Vector Consulting
training material for the tool itself. But Services, 70499 Stuttgart, Germany. Contact him at christof.ebert@
in proprietary code, you may not wish vector.com.
for your code to leave the confines of
your private repositories. Different tools
give different assurances for that; you
should read the terms of use carefully. In
fact, today it is difficult to impossible to PANOS LOURIDAS is an associate professor in the Department of
identify what is original work and what Management Science and Technology, Athens University of Econom-
is generated fake, such as pictures and ics and Business, 10434 Athens, Greece, and the director of research
videos used for misinformation. De- and development at GRNET S.A., 11523 Athens, Greece. Contact him
manding a source statement will not at louridas@aueb.gr.
work, because those who want to do
evil will not follow such self-imposed
rules. Watermarks on all levels or being
embedded up to steganographic algo-
rithms can easily be removed. Forensic
AI researchers propose using end-to-end
blockchain trust mechanisms to label IT professionals—must specify up- most software companies will have an
what has been a proven original piece front what is not (!) to happen. Re- AI-augmented development and test-
of work. Yet the challenge remains to quirements engineering must start ing strategy, up from very few today.
make the initial proof on which the with negative requirements, such as Most Internet content and mobile apps
blockchain will be based. which misuse cases, confusion cases, will be fully or largely commanded
A major risk is about cybersecu- and abuse cases must be avoided, and by generative AI. Software develop-
rity of generated code. Generative AI what transformations would be ex- ers will need new competencies, such
tools and platform might be misused plicitly allowed. Generative AI will in as improving automatically generated
and trained to insert unwanted code the near future think and learn much software, feeding learning engines, and
fragments into any code they process. more efficiently than humans. With fu- exploring behaviors which are not ex-
Such snippets might look innocent but ture AI systems rapidly improving them- plainable. Generative AI will accelerate
could introduce backdoors, manipu- selves without human intervention, they software development. Yet be aware of
late data, or feed information to exter- could potentially wipe out humanity. A marketing hype on shortcuts to build-
nal targets. Though this holds for any simple thought experiment would be ing secure, resilient software based on
code reuse, cyber warfare will enter ecology. Today many politicians put cli- not deterministic technologies.
a new stage with AI-based generated mate change risks at extraordinary lev-

T
code which is hard to understand and els, forgetting about other challenges we
test. Verification and validation of AI have. A generative AI that copies such hese are exciting times, not
will grow in relevance. Software prac- single-minded behavior might conclude just for software engineers.
titioners need to enhance their compe- to just stop human life on earth to re- Some even argue that we may
tences on the right side of the “V” in duce climate change. be witnessing the first sparks of arti-
order to verify accuracy of underlying In a world of generative AI and ficial general intelligence8—or maybe
AI and resulting artifacts. low code, it’s hard to imagine a future not (yet).9 In any case, generative AI
Software engineering for and with where software engineers are as highly is here to stay, is likely to be a game
AI must start with assertions that cre- paid as today. Many traditional roles, changer, and change software engi-
ate boundaries of what is allowed and such as programmer, will change. With neering as well. As with any new tech-
what not. Like Isaac Asimov’s robotic the current evolution speed, we can ex- nology, it can be used, misused, and
rules, our society—and specifically pect that within the next three years abused. Elon Musk, who is known for

J U05:44:55
Authorized licensed use limited to: Dr. D. Y. Patil Educational Complex Akurdi. Downloaded on July 23,2024 at LY/A U G UST
UTC 2 3 |Xplore.
2 0IEEE
from I E E E Restrictions
S O F T WA R E
apply. 37
 SOFTWARE TECHNOLOGY

much, but not stopping innovations, 3. A. Radford et al., “Improving lan- no. 3, pp. 20–28, May/Jun. 2023,
demands oversight for AI, having de- guage understanding by generative doi: 10.1109/MS.2023.3242179.
scribed the technology as “potentially pre-training,” OpenAI, San Fran- 7. A. R. Chow and B. Perrigo, “The AI
more dangerous than nukes.” As those cisco, CA, USA, Jun. 2018. [Online]. arms race is changing everything,”
who develop this technology, we as Available: https://cdn.openai.com/ Time, Feb. 2023. [Online]. Avail-
leading practitioners must safeguard research-covers/language-unsuper able: https://time.com/6255952/
and control AI. vised/language_understanding ai-impact-chatgpt-microsoft-google/
_paper.pdf 8. S. Bubeck et al., “Sparks of artificial
References 4. S. Wolfram, What is ChatGPT general intelligence: Early experi-
1. A. Vaswani et al., “Attention is all Doing… and Why Does it Work? ments with GPT-4,” 2023.
you need,” in Proc. 31st Conf. Adv. (2023). [Online]. Available: [Online]. Available: https://arxiv.org/
Neural Inf. Process. Syst. (NIPS), https://wolfr.am/SW-ChatGPT abs/2303.12712
2017, vol. 30, pp. 5998–6008. [Online]. 5. C. Ebert, D. Bajaj, and 9. R. Lim, “GPT-4 is amazing but still
Available: https://papers.nips.cc/paper/ M. Weyrich, “Testing of software struggles at high school math
7181-attention-is-all-you-need.pdf systems,” IEEE Softw., vol. 39, competitions.” Cantor’s Paradise.
2. J. Devlin et al., “BERT: Pre-training no. 4, pp. 8–17, Jul/Aug. 2022, Accessed: Apr. 1, 2023. [Online].
of deep bidirectional transformers doi: 10.1109/MS.2022.3166755. Available: https://russelllim22.
for language understanding,” 2018. 6. C. Ebert and U. Hemel, “Technology medium.com/gpt-4-is-amazing
[Online]. Available: https://arxiv.org/ trends 2023: The competence chal- -but-still-struggles-at-high-school
abs/1810.04805 lenge,” IEEE Softw., vol. 40, -math-competitions-cbc2e73738e

Since 1994, the SEI and the Institute of Electrical and

2024 IEEE COMPUTER SOCIETY/
WATTS S. HUMPHREY SOFTWARE QUALITY AWARD
NOMINATIONS

SOFTWARE ENGINEERING INSTITUTE

Electronics Engineers (IEEE) Computer Society have

cosponsored the Watts S. Humphrey Software Quality
Award, which recognizes outstanding achievements
in improving an organization’s ability to create and
evolve high-quality software-dependent systems.
NOW OPEN

Humphrey Award nominees must have demonstrated

an exceptional degree of significant, measured,
sustained, and shared productivity improvement.

TO NOMINATE YOURSELF OR A COLLEAGUE, GO TO

computer.org/volunteering/awards/humphrey-
software-quality

Nominations due by September 1, 2023.

FOR MORE INFORMATION

resources.sei.cmu.edu/news-events/events/watts

Digital Object Identifier 10.1109/MS.2023.3284195

38 useI Elimited
Authorized licensed E E Sto:
O FDr.
TW D.AY.
RE | Educational
Patil W W W. C O Complex
M P U T E Akurdi.
R . O R GDownloaded
/ S O F T W Aon | @
R EJuly I E E E Sat
23,2024 O05:44:55
F T W A RUTC
E from IEEE Xplore. Restrictions apply.