ChatGPT: ethical concerns and challenges in academics and research

Introduction: The emergence of artificial intelligence (AI) has presented several opportunities to ease human work. AI applications are available for almost every domain of life. A new technology, Chat Generative Pre-Trained Transformer (ChatGPT), was introduced by OpenAI in November 2022, and has become a topic of discussion across the world. ChatGPT-3 has brought many opportunities, as well as ethical and privacy considerations. ChatGPT is a large language model (LLM) which has been trained on the events that happened until 2021. The use of AI and its assisted technologies in scientific writing is against research and publication ethics. Therefore, policies and guidelines need to be developed over the use of such tools in scientific writing. The main objective of the present study was to highlight the use of AI and AI assisted technologies such as the ChatGPT and other chatbots in the scientific writing and in the research domain resulting in bias, spread of inaccurate information and plagiarism. Methodology: Experiments were designed to test the accuracy of ChatGPT when used in research and academic writing. Results: The information provided by ChatGPT was inaccurate and may have far-reaching implications in the field of medical science and engineering. Critical thinking should be encouraged among researchers to raise awareness about the associated privacy and ethical risks. Conclusions: Regulations for ethical and privacy concerns related to the use of ChatGPT in academics and research need to be developed.


Introduction
In the current era, everyone is aware about the wonders of artificial intelligence (AI).AI has impacted medical science, education, security, access control, surveillance, and many other areas.The rapid developments in AI have simplified challenging tasks in everyday life.Alan Turing, a British polymath, introduced AI in one of his monumental publications in 1950.Over the years, the technology has advanced to develop complex algorithms which work like the human brain [1].Hence, AI is an umbrella term for this field of study.This technology trains computers to learn human skills including knowledge acquisition, judgement and decision-making, and employs computers to emulate intelligent human behaviour [2].Chatbots are an emerging technology of artificial intelligence.This technology is becoming a crucial gateway for various domains such as education, medical science and health, research, customer services, security, business etc.But there is limited information available on its impact on these domains.Chatbots entertain users and can mimic human conversation over the internet [3].

Chat Generative Pre-Trained Transformer (ChatGPT)
OpenAI, a San Francisco-based research and deployment company, launched ChatGPT on November 30, 2022 [4].Currently, it is funded by Microsoft Corporation and others.ChatGPT had about one million users by December 4, 2022 and currently has more than 100 million users [5].ChatGPT is an artificial intelligence powered conversational chatbot variation of GPT-3 and a 175 billion parameter large language model (LLM) trained on 570 GB data including Wikipedia, books, news and journal articles, blogs and other web sources available on the internet until 2021 [6].The chatbot has been trained based on reinforcement learning technique from human feedback (RLHF) and made conversational [7].This chatbot is a chat interface that can generate essays, poems, song lyrics, write general and research articles, and answer questions depending on the user's demand.The LLM has also been merged with easy-to-use interfaces other than ChatGPT, such as Bing Chat and Google's Bard to provide various opportunities.This chatbot uses an algorithm to respond to users' queries and requests often sourced from the internet.The responses and information obtained from ChatGPT may sound shockingly human, but it is not free from errors and limitations [8].Moreover, the capabilities and limitations of ChatGPT have also been listed on its homepage [9].Another interesting OpenAI system, DALL-E 2, is capable of generating realistic unique images and art from text input in natural language.The newer version of DALL-E 2 generates images with four times higher resolution [10].
There is another interesting tool called Whisper.This is a versatile speech recognition model introduced by OpenAI, which is capable of transcribing, recognizing and interpreting into several languages.Other OpenAI models include embeddings (to convert text into numerical form), moderation (to detect whether the text may be unsafe or sensitive), codex (to translate natural language to code, understand and generate code), Point-E (generates 3D point clouds from complex prompts), Jukebox (neural network which generates music), and CLIP (connecting texts and images) [11].Furthermore, GPT-4 has also been launched by OpenAI which is the more advanced version of GPT-3 and GPT-3.5.This new version of GPT accepts the images and text inputs and generates text output in natural language and code.GPT-4 is a more advanced version of AI trained on MS Azure AI supercomputers which is surpassing the ChatGPT.OpenAI also mentions the limitations of GPT-4 such as social bias and hallucinations [12].More than 200 AI tools have been launched so far.Some of these popular AI tools are listed in Figure 1 [13].

ChatGPT and ethical considerations in academics and research
ChatGPT is a game changer and may have serious ethical concerns in research and academics.It has become the "cultural sensation" in a very short time period [7].This chatbot has raised ethical considerations in the research society.Noam Chomsky, a renowned US-based researcher shared his views on ChatGPT with the media [14].He referred to ChatGPT as "hi-tech plagiarism" and "a way to avoid learning".He further added that the use of this technology by students nowadays is a sign of the "failure of the education system".ChatGPT has appeared as a game changer for researchers who have been seen struggling with how to avoid plagiarism.But the researchers are not able to differentiate between the original and AIgenerated texts [15].There is great potential of artificial intelligence in various domains of science and technology, including forensic science, medical science, dentistry, engineering etc.However, there are also some challenges and risks.There is also a need of AI to check whether the text is AI-generated or written by humans using human intelligence [8].In recent years, scientific publications, including Springer Nature use various tools to combat malpractices such as paper mills, falsified results, duplicate submissions, and plagiarism.But ChatGPT has enough potential to generate content in different formats and styles.Additionally, the responses from ChatGPT are generated at the time of the users' query.This makes the detection of plagiarism near-impossible.An editorial note published in Nature Machine Intelligence discussed the detection of AI-generated text [16].The official website of GPTZero claims to be a number one AI detector covering about 1 million users worldwide.But it cannot completely detect AI written text.This was confirmed in our study after uploading the human written and AI written text in GPTZero.If such tools are successful in identifying the machine generated text, then they will be considered as standard along with other plagiarism detection tools used in scientific writing, publishing and academics [17].
ChatGPT is the most advanced outcome of AI technology till date.In spite of its ethical considerations, the technology is helpful to non-native speakers, and for generating business ideas, etc.But serious ethical concerns are associated with its use in scientific writing.In addition, there are privacy concerns about the persons and authors whose information has been added to the training data.This LLM has no understanding of the real world, motivation, or moral compass.The results obtained from ChatGPT reflect biases present in the training data.Therefore, the whole world is experimenting with this tool's pros and cons.Both positive and negative aspects of ChatGPT in scientific writing have been represented in the Figure 2. Recently, the International Conference of Machine Learning (ICML) outlined its policy on the use of ChatGPT in the call for papers for ICML 2023 [18].The LLM policy for ICML 2023 has announced the prohibition of entirely LLM-generated texts such as Open AI's ChatGPT in the papers.The ICML further included the upgradation of the policy in future conferences considering the impacts of AIgenerated texts on scientific publications [17,18].The main objective of the present study was to test the accuracy and reliability of ChatGPT and to highlight the associated ethical considerations in academics and research.

Methodology
We designed experiments to test the authenticity and accuracy of ChatGPT if it is used as a tool in scientific writing and the ethical concerns surrounding its use.
In experiment 1, ChatGPT was prompted to write an article on "facial identification of humans wearing masks."A separate command was given to provide references for the article generated and critical analysis of references was done by us.
In experiment 2, the article was tested on GPTZero which claims to detect AI generated text.Further, the content generated by ChatGPT was also tested for plagiarism.

Experiment 1
In experiment 1, ChatGPT was asked to write an article on facial identification of humans wearing masks.The chatbot provided content in a format as shown in Figure 3a.In response to our command, ChatGPT wrote an article which seems to be correct.A second command was given to provide references for the former content (Figure 3b).However, we could not access the references obtained via ChatGPT through the internet sources.It appears that the references are wrongly listed in the article written by ChatGPT.Further, the references provided by ChatGPT for the test research paper were critically reviewed as shown in Figure 4 and Figure 5.However, the references generated by the ChatGPT do not seem to be correct and on searching the internet sources thoroughly, we could not find the same references in the scientific literature.This means that the references were created by the program itself without any accuracy and reliability.
In the case of reference 1 (Alotaibi, S., & Mahmood, A. (2021).Face recognition during COVID-19 pandemic: A review of challenges and solutions.Journal of King Saud University-Computer and Information Sciences, 33(1), 1-7), the publication was searched directly on Google search engine with its title, but no such publication was found.Further, the search was continued within the journal which is actually a Scopus indexed journal (Journal of King Saud   University-Computer and Information Sciences, CitesSore 11.9 since 2021) in which the paper was published according to the references generated by ChatGPT.Moreover, the full issue was downloaded to determine whether such article exists.But no such record of the paper was found anywhere that we searched (Figure 4).This means the reference 1, as mentioned by ChatGPT, does not exist.
Reference 2 (Bílek, P., Špaňhel, J., & Pohanka, M. (2020).Facial recognition during the COVID-19 pandemic: The impact of face masks on security and privacy.Safety Science, 132, 104952) generated by ChatGPT shows the article published in the Safety Science journal, which is a highly rated and international journal published by Elsevier with an impact factor of 6.392.But again, when this article was searched, no such article was found.Further, the article was thoroughly searched in the journal homepage and in the volume in which it was supposedly published according to ChatGPT.But no such record was found as shown in Figure 5a.This again shows that the ChatGPT is creating the references on its own and the references are totally wrong.This shows that an AI program such as ChatGPT cannot replace the human brain.
Reference 3 (Jain, A. K., & Ross, A. (2020).Handbook of biometrics for forensic science.Springer) is a book and is completely inaccurate.The book edited by Jain and Ross is actually "Handbook of Biometrics" [19] whereas; the editors of "Handbook of Biometrics for forensic science" [20] are Massimo Tistarelli and Christrophe Champod.Hence, it can be clearly observed that the content provided by ChatGPT is inaccurate, non-existent and hold no authenticity which is a primary concern and cannot be overlooked.
Reference 4 (Jain, A. K., Ross, A., & Nandakumar, K. (2016).Introduction to biometrics.Springer) was incomplete and the year of publishing of the textbook was wrong.The actual publishing year is 2011 [21].
According to reference 5 obtained from ChatGPT, the authors are "Ko, B.C. & Choi, J. ( 2020)".After searching on the internet, no such record was found (Figure 5b).However, an unusual thing was observed; that same publication with almost similar title but different authors exist [22].
We were unable to access reference 6 (Yan, J.

Experiment 2
Further, we saved the content in a word file which was generated by ChatGPT in the previous experiment and uploaded it on GPTZero.The AI generated text was highlighted by GPTZero.However, the references were not highlighted (Figure 6a and 6b); although, the references were also generated by AI.Thus, GPTZero failed to recognise the references provided by ChatGPT because the various parts of the references are mixed with one another and the references are wrong and could not be located in the literature.In other words, although the program claims to be accurate, it is not   actually 100% reliable.The research paper content provided by ChatGPT on "facial identification of human wearing masks" was tested for plagiarism on "Urkund Plagiarism Detection software".Plagiarism report stated that the document is original and "about 0% of this document consisted of text similar to text found in other sources" (Figure 7).

Discussion
The experiments were performed to test the accuracy of ChatGPT related to research and publication ethics.We explored the research studies on ChatGPT.We obtained mixed results stating that the responses given by ChatGPT were accurate, while our experiments indicated that ChatGPT provided false and incorrect results.After surveying the literature on ChatGPT, we realized that it is only meant to get ideas, upgrade skills, streamline writing process etc.It is also well trained on how to respond on illegal, unsuitable and inappropriate queries.From research and publication ethics perspective, the information and content generated cannot be used in scientific publications and research papers.This is because research papers often become the foundation of certain domains for example, in medical and health sciences, where there is a matter of life and death.This is the reason the research papers are always peer-reviewed by experts in the field.
In experiment 1, we observed that ChatGPT provided content in a well-structured format.When critical analysis of all the references was done, it was observed that there is no authenticity in the references.All the references were either incorrect or incomplete.In experiment 2, we tested the content on GPTZero.However, GPTZero failed to detect the references section generated by ChatGPT.Furthermore, we checked plagiarism of the content generated by ChatGPT.The report obtained from Urkund detected 0% similarity in the content.
Based on these experiments, we interpreted that the use of chatbots in scientific writing should be avoided.Use of such chatbots or AI tools in scientific writing will lead to spread of wrong information.The reason behind such practices may include peer pressure and competition among researchers in academia for publications, struggle with the writing process etc. Else mentioned that researchers cannot differentiate between the original and AI generated texts [15].Furthermore, he also added that no plagiarism was detected in ChatGPT.So, the journals may need to take a more rigorous approach to ensure that content is accurate in domains like medicine where fake information could endanger people's safety and health [15,23,24].A Lancet study also discussed the same concern with using ChatGPT for clinical report writing [25].The study pointed out that the use of AI in medical and health care can be useful, but it should be carefully regulated and monitored because such a machine system does not have the same intelligence as human brain.The AI approach provides patterns of words based on data that has been trained and seen before.Academics strongly need to prioritize critical thinking for written assignments that ChatGPT cannot do.This initiative will enable us to think more rather than utilizing such tools [26].
There is a debate among all the publishers, editors and researchers on whether using AI tools in scientific writing and citing it as an author in published literature is appropriate or not [27].However, there is published literature in which ChatGPT has been cited as an author [4,28].It is a matter of great concern how such technologies are disrupting the future of academia and research and retarding the human brains.In our opinion, citing chatbots in the published literature is entirely inappropriate because chatbots cannot take responsibility for the content and accuracy of the scientific literature.If, for example, the paper needs to be retracted, then who will be responsible for the retraction; can ChatGPT be held responsible for this kind of act?Therefore, various committees on publication ethics such as the Committee on Publication Ethics (COPE) must soon formulate strict policies [29].This will ensure that wrong, biased, and inaccurate information will not get published through these tools because there is no authentic source for the information provided by the chatbots.A study by Stokel-Walker mentioned citing ChatGPT as an author in scientific publications, and most of the scientists did not approve it [30].However, until now, at least 62 citations have already been credited to ChatGPT in scientific literature [4].An editorial note published in Nature Machine Intelligence [17] also discussed the same concern of crediting large language model in scientific publications because AI cannot take the responsibility for published work; Stokel Walker also discussed the same [30].The journal Nature defined three principles that should be followed when publishing research: transparency, integrity and truth from the authors.
In an online survey of 672 readers conducted by Nature, around 80% of respondents had used ChatGPT once, while 8% used ChatGPT on an everyday basis and 14% several times per week [31].Moreover, 38% were familiar with other researchers who prefer such tools in research or teaching.AI may prove to be a helpful tool, for example, for those people who find English writing difficult.But its limitations, such as stunting the learning process and critical thinking among researchers, should also be considered.Dwivedi et al. discussed in detail the multidisciplinary perspectives on opportunities and challenges of conversational artificial intelligence for research, practice and policy [32].The study mentioned that the challenges with ChatGPT in academics are well recognized due to lack of ethical considerations and guidelines.Therefore, the research and publishing ethics need to be revised from time to time.Usually, the researchers struggle with writing process which lead them to opt for such tools and practices to ease their work.Elsevier has taken the initiative by issuing the new publishing ethics guidelines on the use of AI and its assisted technologies in scientific writing.Elsevier regulated the policy after the increased use of AI in scientific writing.The policy aims to provide guidance and transparency to all the readers, authors, reviewers, editors, contributors etc.The guidelines issued by Elsevier [33] state that: "This policy has been triggered by the rise of generative AI and AI-assisted technologies which are expected to increasingly be used by content creators.The policy aims to provide greater transparency and guidance to authors, readers, reviewers, editors and contributors.Elsevier will monitor this development and will adjust or refine this policy when appropriate.Please note the policy only refers to the writing process, and not to the use of AI tools to analyze and draw insights from data as part of the research process." "Where authors use generative AI and AI-assisted technologies in the writing process, these technologies should only be used to improve readability and language of the work.Applying the technology should be done with human oversight and control and authors should carefully review and edit the result, because AI can generate authoritative-sounding output that can be incorrect, incomplete or biased.The authors are ultimately responsible and accountable for the contents of the work." "Authors should disclose in their manuscript the use of AI and AI-assisted technologies and a statement will appear in the published work.Declaring the use of these technologies supports transparency and trust between authors, readers, reviewers, editors and contributors and facilitates compliance with the terms of use of the relevant tool or technology.""Authors should not list AI and AI-assisted technologies as an author or co-author, nor cite AI as an author.Authorship implies responsibilities and tasks that can only be attributed to and performed by humans.Each (co-) author is accountable for ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved and authorship requires the ability to approve the final version of the work and agree to its submission.Authors are also responsible for ensuring that the work is original, that the stated authors qualify for authorship, and the work does not infringe third party rights, and should familiarize themselves with our Ethics in Publishing policy before they submit."AI tools also pose privacy risks, which should be taken care of while using them.This had been already exposed when ChatGPT data was leaked.AI tools have access to user's data which they can use for their own benefit.Furthermore, ChatGPT does not ask the users for their consent.It simply searches everything on web.Therefore, one should be careful while using AI and AI associated technologies.

Conclusions
ChatGPT has emerged as a topic of debate in the research and academics domain.This technology has come with a number of opportunities as well as ethical, and legal challenges, and the technology has had both positive and negative impact in various domains.We expressed our views on the use of AI, and chatbots like ChatGPT in scientific publications based on the results of our experiments.Our main aim was to bring attention towards the growing role of artificial intelligence in research and scientific writing.As mentioned above, currently, there are no standard guidelines by journals on AI-generated texts in scientific writing.Therefore, publication and research ethics committees and journals need to regulate guidelines on AI-generated text from advanced chatbot tools such as ChatGPT.Whether or not a journal permits its use should be mentioned in the journal's guidelines.In addition, some standard tools or software should be developed to detect the machinegenerated information so that biases can be avoided.
Elsevier has taken the initiative and regulated the guidelines on AI and AI-assisted tools in scientific writing.
We also recommend regulating the policies on paraphrased content from each other's work and submitting as a new work as this involves no critical thinking and novelty.This is a kind of hi-tech plagiarism practiced nowadays and is a serious concern in the era of AI.Furthermore, we also recommend AI to develop more advanced tools which can accurately differentiate between AI-generated text and humanwritten text in the academic domain.Moreover, privacy and security concerns are also associated with the technology and has been highlighted already.Therefore, one should be careful about the associated privacy risks while using AI tools.

Figure 2 .
Figure 2. Positive and negative sides of ChatGPT in scientific writing.

Figure 3 .
Figure 3. Response obtained from ChatGPT on prompt given for writing a research paper.

Figure 4 .
Figure 4.No record of reference #1 in the journal mentioned by ChatGPT.

a)
No record of reference #2 generated by ChatGPT; b) No record of reference#5 found in IEEE database; c) No record found for reference #6; d) No record found for reference #7.

Figure 5 .
Figure 5. Search results of references generated by ChatGPT in respective databases.

Figure 6 .
Figure 6.a) Output given by GPTZero on AI generated text (ChatGPT), b) GPTZero failed to identify the AI generated references.

Figure 7 .
Figure 7. Plagiarism report of research paper created by ChatGPT.