Uses and Misuses of ChatGPT

Uses and Misuses of ChatGPT by Academic Community: An Overview and Guidelines

Muhammad Abbas1
FAST School of Management,
National University of Computer and Emerging Sciences,
A.K. Brohi Road, H-11/4, Islamabad, Pakistan.
Phone: +92-51-111-128-128 Ext: 383.
Email: pirthegreat@gmail.com

Abstract

The field of ethics in information technology has become increasingly important in recent

years, particularly with the rapid development of AI technology such as ChatGPT, an

artificial intelligence-based language model developed by OpenAI which processes and

generates text in response to input from users. This paper provides an overview of the

academic and research-related uses and misuses of ChatGPT. Particularly, the paper

discusses the potential benefits and risks associated with the use of ChatGPT by academic

community. The paper further identifies potential ethical concerns related to issues such as

authorship, accountability, privacy, transparency, disclosure, credibility and integrity of

scientific inquiry and provides some guidelines to authors, editors, and publishers to navigate

these challenges.

Keywords: Ethics; Artificial Intelligence; ChatGPT; Guidelines

1
Corresponding Author: Muhammad Abbas, FAST School of Management, National
University of Computer and Emerging Sciences, A.K. Brohi Road, H-11/4, Islamabad,
Pakistan. email: pirthegreat@gmail.com.
 Uses and Misuses of ChatGPT

1. Introduction

Artificial intelligence (AI) has been increasingly used in academia across various fields

within natural, applied and social sciences. Academicians and researchers have found it to be

an efficient tool that can be used for a variety of purposes ranging from generating codes to

and generating text (Aydın, Ö., & Karaarslan, 2022; Gao et al., 2022; Nakazawa et al., 2022;

Stokel-Walker, 2022; Valtonen, Mäkinen, & Kirjavainen, 2022). In medical sciences, for

example, AI can be used in medicine specifically in diagnosis of diseases and prescribing

medicine (Briganti & Le Moine, 2020). Similarly, scientists can use AI to generate text on

any topic for a variety of purposes (e.g., writing papers) or analyze text to identify patterns

and trends in public opinion, experiments, writing stance detection (e.g., extracting the

standpoint (i.e., favor or against), and help with the design and execution of simulations

among others (Cahan & Treutlein, 2023; Gao et al., 2022; Kaibel & Biemann, 2021; Hutson,

2022; Zhang, Ding, & Jing, 2022).

In the same vein, ChatGPT (Conversational Generative Pre-training Transformer) is a

large-scale, deep learning-based natural language processing model developed by OpenAI

that uses unsupervised learning to pre-train on a massive amount of text data and then fine-

tune on specific tasks (OpenAI, 2022). This model, which was trained using Reinforcement

Learning from Human Feedback (RLHF), is so far the best artificial intelligence chatbot ever

released to the general public – and it can understand and respond to natural language text,

generate human-like text, answer questions, complete sentences and work with many

languages. ChatGPT has got huge popularity, across the globe, since December 2022. Recent

reports suggest that it took only five days for ChatGPT to received more than one million

subscribers (Mollman, 2022).

Although the use of ChatGPT is in its earlier stages and it has not reached up to its

full potential, academicians can use it for a wide range of academic and research activities
 Uses and Misuses of ChatGPT

ranging from generating codes, assisting in the design and execution of simulations, surveys

or experiments, analyzing text, perform content analyses and identifying patterns (Cahan &

Treutlein, 2023; Gao et al., 2022; Kaibel & Biemann, 2021; Hutson, 2022; Zhang, Ding, &

Jing, 2022) (see Table 1 for an overview). This indicates that the potential of ChatGPT in

scientific research is vast. However, it also brings a number of challenges that all

stakeholders (e.g., researchers, editors, reviewers, sponsoring agencies, publishers, and

regulators) must navigate in near future. This article reviews the academic and research-

related uses of ChatGPT, discusses the relevant ethical challenges, and provides guidelines

for all stakeholders to navigate such challenges. Such a review can help researchers, editors,

reviewers, sponsoring agencies, publishers, and regulators to drawing ethical boundaries on

the use of AI-based text generating chatbots (e.g., ChatGPT) in scientific inquiry and utilizing

the potential of such tools to improve the quality of scientific inquiry.

Therefore, there are several objectives of the current review. First, I provide an

overview of the potential use of ChatGPT in teaching-related and research-related academic

activities. I then discuss the potential ethical challenges for numerous stakeholders and

provide relevant guidelines regarding the use of AI text generating chatbots. Since the

potential use of ChatGPT in a variety of roles (e.g., scientists, professionals, and students) is

very large, if not unlimited, the current review will focus only on the use or misuse of

ChatGPT in research-related or non-research academic activities only. Furthermore, with the

introduction of ChatGPT as an AI language model, there is a possibility that similar or even

more powerful AI based chatbots may appear in future, the current paper provides broader

guidelines for the stakeholders to decide on how to proceed with such AI-based technologies

that generate human-like text.

2. Potential use of ChatGPT

 Uses and Misuses of ChatGPT

ChatGPT can potentially be used for a wide range of teaching-related and research-related

purposes in scientific research. However, it should be noted that the text generated by

ChatGPT must be thoroughly evaluated by the user for errors as this model can generate

incorrect text and provide false information. Particularly, the suggestions (e.g., literature,

citations, methods) provided by this model may be incorrect and therefore the user needs to

careful review and cross-validate the appropriateness of these suggestions from reliable

sources. Below I discuss some of these uses.

2.1.Teaching-related Usages:

ChatGPT can be used for a variety of educational purposes. These may include the

following:

2.1.1. Generating teaching material: if some context or keywords are provided, this

chatbot can generate a variety of teaching material such as course outlines, lesson

plans, and worksheets.

2.1.2. Generating assessment material: Academicians can use this model to analyze

student essays and to generate test questions and quizzes.

2.1.3. Designing training material, simulations, or case studies: Academicians or

trainers can use this model to design simulations, training material, classroom

activities and hypothetical cases on numerous topics for learning purposes.

2.1.4. Assistance in writing proposals: This model can assist in writing proposals for

business ideas, feasibilities studies, and concept papers.

2.1.5. Generating instructions and guidelines: Academicians can use this model to

generate a variety of texts, such as assignment, project guidelines, or emails

containing instructions for students.

2.1.6. Summarization: Users can summarize large amounts of text such as text from

books or reports into short notes for learning purposes using this chatbot.
 Uses and Misuses of ChatGPT

However, academicians need to be very careful while generating such summaries

as the current version of the model may provide incorrect information.

2.1.7. Improving the readability and coherence of text: It can be used to improve the

readability and coherence of text by asking to suggest alternative phrases,

sentences, and paragraphs.

2.1.8. Assisting with grammar, punctuation, and formatting: The model can also assist

with grammar, punctuation, and formatting of the text, ensuring that the text

follows academic conventions.

2.1.9. Cheating in assignments and exams: This model also poses a potential threat to be

used for cheating in assignments and online written or oral exams (Susnjak, 2022).

The strength of the chatbot in providing a response to each prompt in seconds

makes it a potential source of cheating in assignments or during an online exam,

viva or even an interview.

2.2.Research-related Usages:

ChatGPT can also be used for a variety of research-related purposes. Below are some of

these uses.

2.2.1. Assistance in text generation: Researchers can use this model for assistance in

generating text, such as abstracts, introductions, literature reviews, discussions

and conclusions. For example, a researcher can use this model to generate an

abstract for his or her article by providing the model with a summary of the

research background, objectives, methods, and findings. However, since this

model is not able to understand the context or meaning of the information it is

processing, the text generated by this model must be carefully analyzed for

accuracy, truth, and relevance. As this model is only as good as the data it is

trained on, therefore, the text generated by the model has the potential for bias.
 Uses and Misuses of ChatGPT

2.2.2. Data analysis: This chatbot can be used to analyze qualitative data, such as

interview responses, to identify patterns, relationships, and common themes.

However, it is not specifically designed for analyzing quantitative data. That said,

future versions of the model may be able to integrate tools like Python and R

libraries to conduct quantitative data analyses.

2.2.3. Review and Summarization: Researchers can use this chatbot to review the

literature in a specific domain and summarize large amounts of text, such as

journal articles, books, and reports, to identify key findings and generate a

summary. However, researchers need to be very careful while generating such

summaries as the current version of the model is poor at extracting relevant

citations and references of prior studies. For example, if a researcher asks the

model to summarize prior literature on a topic along with relevant citations and

references, the model will generate text and provide citations and references.

However, those citations and references may be incorrect or even nonexistent

(King & ChatGPT, 2023). Therefore, more caution is needed while using the

literature generated by ChatGPT because it may include fake references.

2.2.4. Suggesting research topics and ideas: Researchers can use this model to generate

a list of topics and ideas for research in a given field if it is provided with some

keywords. For example, if a user asks it to generate some topics or ideas for

research related to human-computer interaction, it can generate as many research

topics in this domain. However, these ideas may be relatively simple. Therefore,

the user has to determine the significance or the uniqueness of those ideas.

2.2.5. Generating research questions and hypotheses: This chatbot can generate

research questions and hypotheses based on the background information and the

context provided by the user.

 Uses and Misuses of ChatGPT

2.2.6. Generating outlines and structure for research papers: Researchers can use this

model to generate outlines and structure for research papers, which can help

authors organize their thoughts and ideas in a coherent way.

2.2.7. Identifying gaps in literature: It can be used to analyze existing literature in a

research area to identify research gaps and suggest possible new areas for further

investigation. However, the gaps identified by the model need to verified as the

claims regarding gaps in existing literature can be incorrect and misleading.

2.2.8. Suggesting underpinning theories: If some details on the variables and their

proposed relationships are provided to ChatGPT, it can provide suggestions on the

overarching theories to explain these relationships.

2.2.9. Suggesting appropriate research methods: If some background details on a

study’s research questions and objectives are provided, this model can provide

suggestions on the relevant data collection or data analytical techniques. However,

these suggestions may be incorrect and therefore the user needs to careful review

and cross-validate the appropriateness of these methods from reliable sources.

2.2.10. Designing simulations: This chatbot can generate material and simulations for

research studies if the relevant details on the context, subjects, and the variables

are provided to it.

2.2.11. Generating and validating scale items: This model can generate and validate the

contents of survey items or questions, if some details on the variables including

the operation definition(s) and relevant examples are provided to it. It can also

assist in the rephrasing of items to make them more effective in capturing the right

information.

2.2.12. Developing interview guides: This model can generate interview guides, if some

details the research topic, interviewees and the context are provided.
 Uses and Misuses of ChatGPT

2.2.13. Developing experimental protocols: The model can generate experimental

protocols, such as instructions for participants, procedures, and analysis plans.

This can help researchers create more accurate and effective protocols for

experimental research.

2.2.14. Improving the, grammar and readability of text: Researchers can use this model to

improve the grammar, punctuation, and formatting of a research article. Moreover,

it also be used to improve the readability and coherence of text by suggesting

alternative phrases, sentences, and paragraphs.

2.2.15. Generating references in a given format: It can be used to generate references in a

specific style or format (e.g., APA).

2.2.16. Assisting with data visualization: This model can only generate text descriptions

of data, but it is not able to create visual representations of the data (e.g., figures

or graphs). However, by making some tricky prompts (e.g., make a tabular text,

text-based table, or table-based text), the users can get their text or information

presented in tables.

--------------------------
Table 1 about here
---------------------------

3. Guidelines for stakeholders

Currently, the guidelines provided by many research journals and the Committee for

Publication Ethics (COPE) discuss only the ethical implementation of AI in research review

process. For example, the guidelines discuss the need for fairness in the AI-aided

recommendations provided to the editors (COPE, 2021). However, these guidelines are silent

on the fair use and reporting of AI by the authors. In the meanwhile, the trend of including

ChatGPT as a co-author in scientific papers has rapidly increased (e.g., Ali & ChatGPT,

2023; King & ChatGPT, 2023; Kung et al., 2022). Recently, a few scholars have raised
 Uses and Misuses of ChatGPT

concerns over the use of AI in scientific writing and publishing (Grimaldi, & Ehrler, 2023;

Nakazawa, Udagawa, & Akabayashi, 2022). However, the question on the moral agency or

responsibility of AI is debatable (Coeckelbergh, 2020; Stahl, 2006). In near future, the

scholarly community may face a greater challenge in differentiating between the use of AI as

an assistant in completing the cumbersome job of writing scientific papers verses claiming AI

written content as one’s own. Since the scholarly community lacks clear guidelines on the use

of AI chatbots (e.g., ChatGPT) for conducting scientific research and publishing scientific

articles, I provide some preliminary guidelines to all stakeholders for the use of ChatGPT and

other AI chatbots in scientific studies. These guidelines are discussed in Table 1. The

scientific community urgently needs to look into these matters and address these ethical

concerns.

4. Conclusion

The use of ChatGPT by academic community, both for teaching and research purposes, is

rapidly increasing. Like other inventions, this also poses a number of ethical challenges that

must be addressed. The current review highlights the potential benefits and risks associated

with the use of ChatGPT by the academic community. The paper also offers some ethical

guidelines for researchers, journals, editors, reviewers, and publishers to help address these

challenges. The paper concludes that academicians and researchers should consider the use of

ChatGPT as an aid rather than a replacement for traditional research and writing approaches,

and should always verify the output generated by ChatGPT from reliable sources. Moreover,

academic community needs to develop more detailed guidelines on the use of AI-based

chatbots or language models and how to report these uses.

 Uses and Misuses of ChatGPT

References:

Ali, F., & ChatGPT, OpenAI, Inc (2023). Let the devil speak for itself: Should ChatGPT be
allowed or banned in hospitality and tourism schools?. Journal of Global Hospitality
and Tourism, 2(1), 1-6.
Aydın, Ö., Karaarslan, E. (2022). OpenAI ChatGPT Generated Literature Review: Digital
Twin in Healthcare . In Ö. Aydın (Ed.), Emerging Computer Technologies 2 (pp. 22-
31). İzmir Akademi Dernegi.
Briganti, G., & Le Moine, O. (2020). Artificial intelligence in medicine: today and
tomorrow. Frontiers in medicine, 7, 27.
Cahan, P., & Treutlein, B. (2023). A conversation with ChatGPT on the role of computational
systems biology in stem cell research. Stem Cell Reports, 18(1), 1-2.
Coeckelbergh, M. (2020). Artificial intelligence, responsibility attribution, and a relational
justification of explainability. Science and engineering ethics, 26(4), 2051-2068.
COPE. (2019). COPE Discussion Document: Authorship. Committee on Publication
Ethics. Retrieved from
https://publicationethics.org/sites/default/files/COPE_DD_A4_Authorship_SEPT19_
SCREEN_AW.pdf on 10 February, 2023.
DOI: https://doi.org/10.24318/cope.2019.3.3
COPE. (2021). Artificial intelligence (AI) in decision making. Committee on Publication
Ethics. Retrieved from https://publicationethics.org/sites/default/files/ai-in-decision-
making-discussion-doc.pdf on 01 February, 2023.
DOI: https://doi.org/10.24318/9kvAgrnJ
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A.
T. (2022). Comparing scientific abstracts generated by ChatGPT to original abstracts
using an artificial intelligence output detector, plagiarism detector, and blinded human
reviewers. bioRxiv, 2022-12.
Grimaldi, G., & Ehrler, B. (2023). AI et al.: Machines Are About to Change Scientific
Publishing Forever. ACS Energy Letters, 8, 878-880.
Hern A. AI bot ChatGPT stuns academics with essay-writing skills and usability (2022). The
guardian. Retrieved from https://www.theguardian.com/technology/2022/dec/04/ai-
bot-chatgpt-stuns-academics-with-essay-writing-skills-and-usability on December 30,
2022.
 Uses and Misuses of ChatGPT

Kaibel, C., & Biemann, T. (2021). Rethinking the gold standard with multi-armed bandits:
machine learning allocation algorithms for experiments. Organizational Research
Methods, 24(1), 78-103. https://doi.org/10.1177/1094428119854153
King, M. R., & chatGPT. (2023). A Conversation on Artificial Intelligence, Chatbots, and
Plagiarism in Higher Education. Cellular and Molecular Bioengineering, 1-2.
Kung, T.H., Cheatham, M., Medinilla, A., ChatGPT, Sillos, C., De Leon, L., Elepano, C.,
Madriaga, M., Aggabao, R., Diaz-Candido, G. and Maningo, J. (2022). Performance
of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large
Language Models. medRxiv, 2022-12.
Mollman, S. (2022, December 10). ChatGPT gained 1 million users in under a week. Here's
why the AI chatbot is primed to disrupt search as we know it. Retrieved January 25,
2023, from https://finance.yahoo.com/news/chatgpt-gained-1-million-followers-
224523258.html
Nakazawa, E., Udagawa, M., & Akabayashi, A. (2022). Does the Use of AI to Create
Academic Research Papers Undermine Researcher Originality?. AI, 3(3), 702-706.
OpenAI (2022). ChatGPT: Optimizing language models for dialogue. OpenAI. Published
November 30, 2022. Accessed December 17, 2022. https://openai.com/blog/chatgpt/
Stahl, B. C. (2006). Responsible computers? A case for ascribing quasi-responsibility to
computers independent of personhood or agency. Ethics and Information
Technology, 8, 205-213.
Stokel-Walker, C. (2022) AI bot ChatGPT writes smart essays - should professors worry?
Nature. Published online December 9, 2022. doi:10.1038/d41586-022-04397-7
Susnjak, T. (2022). ChatGPT: The End of Online Exam Integrity?. arXiv preprint
arXiv:2212.09292. https://doi.org/10.48550/arXiv.2212.09292
Valtonen, L., Mäkinen, S. J., & Kirjavainen, J. (2022). Advancing Reproducibility and
Accountability of Unsupervised Machine Learning in Text Mining: Importance of
Transparency in Reporting Preprocessing and Algorithm Selection. Organizational
Research Methods, 10944281221124947.
Zhang, B., Ding, D., & Jing, L. (2022). How would Stance Detection Techniques Evolve
after the Launch of ChatGPT?. arXiv preprint arXiv:2212.14548.
 Uses and Misuses of ChatGPT

Table 1: Guidelines on the use of AI language models (e.g., ChatGPT) for conducting and publishing research

Ethical Aspect Guidelines for Authors Guidelines for Stakeholders

Authorship The first issue relates to the authorship of All stakeholders including editors, reviewers,
ChatGPT. So far, a handful of papers have sponsors, publishers and associations related to
included ChatGPT as a coauthor. Although it the publication ethics should initiate discussions
is debatable, using an AI language model or a on numerous platforms regarding the inclusion
chatbot as a coauthor may raise several issues. of AI chatbots as coauthors to the manuscripts.
First, authorship to a scientific paper does not The editors and publishers should clearly
require contribution to that paper only. COPE specify their policies on allowing or not
guidelines suggest that an authorship to a allowing AI-based language models as
paper should be at least based on substantial coauthors. I believe the practice of including AI
contribution to the study and responsibility chatbots as coauthors should be discouraged for
and accountability for the work being the reasons discuss earlier.
published (COPE, 2019). An AI language
model cannot be held responsible for any error
or misreporting in a scientific paper. Second,
authorship requires significant intellectual
contribution to the study, which includes
conceiving an idea, designing the study,
collecting and analyzing data and writing the
report. An AI language model can assist in
some of these tasks but it is not capable of
making any intellectual contribution in the
way humans do. Third, including an AI model
as a coauthor may result in infringement on
the rights of the creator of that AI model.
Authors should be held accountable for the The stakeholders should hold the authors
results generated by AI, and should explain accountable for the results generated by AI.
and defend their use of AI in any form. Since These points should be part of the contracts or
AI has the potential to generate incorrect or agreements between authors and publishers or
false information (King & chatGPT, 2023), sponsors.
only humans can be held responsible for an
Accountability error in any scientific report. Also, authors
should be responsible to make sure that any
text generated by AI (e.g., scale items,
experiment scenarios, summaries) does not
violate, for example, the intellectual property
rights. An AI language model cannot be held
accountable because it lacks moral agency and
 Uses and Misuses of ChatGPT

moral responsibility like humans do.

Authors should clearly disclose the use of AI- It is highly difficult to determine the amount of
generated text in any part of their manuscript contribution or the degree of assistance to a
or any other type of assistance received from scientific study or a report provided by an AI
AI chatbots (e.g., assistance in code language model, editors and reviewers can only
generation, simulation development, and rely on the information provided by the authors
methodology). on the use of AI. Editors and publishers should
Transparency and
ensure that the use of AI has been
Disclosure
acknowledged in the methodology section of a
paper to ensure transparency and
reproducibility, and that the results generated by
AI have been reviewed, edited, and approved by
the author to ensure their accuracy, relevance,
and compliance with ethical standards.
Authors may make a limited use of AI The stakeholders should encourage authors for a
chatbots and refrain from using AI language limited use of AI language models in
models to the extent that the model may not conducting and writing scientific papers. For
Restricted Use of
justify authorship to a scientific paper. example, AI language models may not be used
AI-based language
beyond the contribution a human author is
models
generally expected to make. Nevertheless, it is
still unclear how much contribution would
justify authorship to a scientific paper.
Relate to above, authors should provide The stakeholders should ensure that the authors
sufficient justifications that the use of AI is have provided sufficient justification for the use
Justification for
appropriate to the research question or study of AI, in any form, in their research.
using AI-based
and that this will significantly help to improve
language models
our understanding of the phenomenon being
studies.
All stake holders need to realize that the use of AI-based language generating models at a larger
Credibility and scale would raise serious concerns about the credibility and integrity of the scientific endeavor
integrity of per se. If a major contribution to a scientific paper can be provided by an AI language model, it
scientific inquiry would suggest that the contributions of the human researchers are less important as compared to
AI.