{"id":104985,"date":"2025-01-16T07:15:00","date_gmt":"2025-01-16T10:15:00","guid":{"rendered":"https:\/\/www.ufrgs.br\/jornal\/?p=104985"},"modified":"2025-01-16T09:23:56","modified_gmt":"2025-01-16T12:23:56","slug":"chatgpt-impacts-daily-routines-in-research-and-education-and-raises-questions-about-truthfulness-and-evaluation-methodologies","status":"publish","type":"post","link":"https:\/\/www.ufrgs.br\/jornal\/chatgpt-impacts-daily-routines-in-research-and-education-and-raises-questions-about-truthfulness-and-evaluation-methodologies\/","title":{"rendered":"ChatGPT impacts daily routines in research and education and raises questions about truthfulness and evaluation methodologies"},"content":{"rendered":"\n

Latest News <\/strong>| Teachers, students, and researchers are forced to update their practices due to the availability of artificial intelligence tools, but lack of knowledge about how new technologies operate is still significant<\/strong><\/h4>\n\n\n\n

*By\u00a0Giordano Benites Tronco
*First published April 13, 2023
*Image: Fl\u00e1vio Dutra – Dall-E\/ JU- In the first line, representations about the topic generated by Dall-E, an application that creates images using Artificial intelligence.<\/em> In the second line, the same images were put through image processing.<\/p>\n\n\n\n

<\/div>\n\n\n\n

In November 2022, the OpenAI organization provided free access to a computer program that speaks like people \u2013 and it is open to anyone. You type a question, the program answers \u2013 no groundbreaking performance apart from the fact that the answers are good, very good.<\/p>\n\n\n\n

The program \u2013 called ChatGPT \u2013 has the accumulated knowledge of nearly the entire internet and answers complex questions, takes extremely specific steps, creates unpublished stories in the style of established writers and even improves a previously written text \u2013 all of that in human natural language.<\/p>\n\n\n\n

In universities, ChatGPT created a widely felt earthquake that fostered discussions: are ChatGPT answers a form of plagiarism? How to evaluate students\u2019 knowledge if they can answer assessments using the program? Can researchers write scientific articles in collaboration with ChatGPT? To what extent using artificial intelligence in a field hitherto exclusively human can be considered ethical?<\/p>\n\n\n\n

As it could not be otherwise, artificial intelligence tools have also transformed journalistic practices. This, for example, is the first time that the reporter writing to you uses ChatGPT to verify the credibility of information. And this primary contact with this type of technology yielded some interesting insights, including about the very own relationship between humans and machines. For this reason, I have included comments throughout the article regarding the behind-the-scenes of the news report.<\/p>\n\n\n\n

How<\/strong> does ChatGPT <\/strong>not<\/strong><\/em> work?<\/strong><\/h5>\n\n\n\n

First and foremost, let’s be clear: ChatGPT is not Google. Yes, you can insert in the prompt (space intended for the insertion of the text or user question) the same question that you would type in Google. And, yes, the program will generate a response. But while Google offers a list of links among which the user might find their answer, ChatGPT does not clarify where its information comes from.<\/p>\n\n\n\n

If pressed to answer, artificial intelligence will say that it has been fed a large amount of data (news articles, books, academic articles, conversations) and that it uses this prior knowledge to generate its answers. This knowledge, however, is based on information available up to 2021.<\/p>\n\n\n\n

This is the first common mistake about ChatGPT: it is not constantly updating its database.<\/p>\n\n\n\n

Ask about the Covid-19 pandemic and the tool will tell you about the lockdown measures implemented by countries and the first vaccines developed. Ask about who won Big Brother Brazil 2022 and it will not know how to answer. Searching for recent news on ChatGPT is therefore useless.<\/p>\n\n\n\n

\n

“Sorry, but as a language model trained in 2021, I have no information about future events\u201d.<\/p>\nChatGPT<\/em><\/cite><\/blockquote>\n\n\n\n

OpenAI’s goal with ChatGPT is to produce an artificial intelligence capable of understanding and communicating with human natural language. It is not a search engine: it is a machine that knows how to speak very well.<\/p>\n\n\n\n

The truth is that this technology works more like a text editor than an internet search engine. To be more precise, it is closer to a text corrector.<\/p>\n\n\n\n

As it reports Joel Lu\u00eds Carbonera, professor of the Department of Informatics at Universidade Federal do Rio Grande do Sul (UFRGS), ChatGPT’s artificial intelligence was trained to perform a predictive task \u2013 given a set of words, it predicts the next most likely element. It is no different from what happens when you write a text message on your mobile and the app automatically completes the word based on the first letters you type. The difference is that ChatGPT prediction is capable of generating not only words or phrases, but entire texts.<\/p>\n\n\n\n

Understand how ChatGPT was trained in the infographic below:<\/p>\n\n\n\n

<\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n
\n
\"\"<\/figure>\n<\/div>\n\n
\n
\"\"<\/figure>\n<\/div>\n\n
\n
\"\"<\/figure>\n<\/div>\n\n
\n
\"\"<\/figure>\n<\/div>\n\n\n

Carbonera explains that ChatGPT generates the most plausible answer from a linguistic point of view, not factual. Therefore, we cannot assume that it is factually correct.<\/p>\n\n\n\n

The program\u2019s core commitment is not with the “truth” but with the production of a semantically correct text.<\/p>\n\n\n\n

\n

“Here’s the biggest trap: if we use ChatGPT as an oracle, we may not be getting the accurate and factual answers we need.\u201d<\/p>\nJoel Lu\u00eds Carbonera<\/em><\/cite><\/blockquote>\n\n\n\n

\n

Since I was concerned about the program not mentioning sources in its answer to a specific question, I asked ChatGPT which texts from its database had been used to generate that answer.

Very politely, the program apologized and said it was impossible to determine specific texts. “The answers,\u201d it said, “are not based on a single source of information, but rather on a wide range of textual data that the AI model used to learn natural language.\u201d

The program explained to me that ChatGPT is a model of language and not a source of knowledge or information. Its goal is to generate coherent and meaningful responses. It is up to the user to critically verify the information provided.

We then had a heated discussion about the difference between coherence and truth, until I realized that several of my questions began with \u201cplease.\u201d As in: “Please, could you give me the answer about x?\u201d. I was subconsciously worried about being kind to the machine. I ended the conversation thinking about how vulnerable human beings are to any simulation of empathy. This observation I preferred not to share with ChatGPT.<\/p>\n<\/div><\/div>\n\n\n\n

Digital hallucinations<\/strong><\/h5>\n\n\n\n

Ask ChatGPT something it does not know, and the answers may vary. It can apologize and say that the question refers to something that, for the program, is in the future \u2013 that is, after 2021. Regarding controversial matters, it can dodge the question or present multiple existing points of view about the topic. And sometimes it might invent answers.<\/p>\n\n\n\n

These texts about things that do not exist or have not happened are called \u201challucinations\u201d and are quite common. In an interview with the website Metr\u00f3poles, artificial intelligence researcher Anderson Rocha told an anecdotal situation: he asked the program the names of five artificial intelligence professors who had died in the past five years. The program presented the names and described the circumstances of the death of each one of them. However, out of the five allegedly deceased, three are still alive. One of them shares a room with Rocha.<\/p>\n\n\n\n

Curiously, hallucinations are very common when academics and scientific papers are involved. There are several reports on the Internet of ChatGPT responses that mention researchers who do not exist or with links to non-existent articles.<\/p>\n\n\n\n

Professor Joel Carbonera can’t explain why hallucinations happen, but he has a theory: during the machine’s training, programmers failed to fine-tune the responses, which he calls \u201cputting safety handrails on the experiment.\u201d This adjustment process is meant to prevent artificial intelligence from generating inappropriate replies, such as sexist or prejudicial responses. It also serves to prevent unreasonable responses, such as the so-called hallucinations. According to the teacher, hallucinations happen when the \u201cbugs\u201d exceed the filter acquired in the training.<\/p>\n\n\n\n

\n
\n

One of the first uses of ChatGPT for an agenda was in source search. I asked the program to generate a list of researchers that had already published papers about the use or the impact of ChatGPT in the academic environment. Only later, I realized that, obviously, the program would not know the answer, since ChatGPT itself and any papers written about it were released after 2021.<\/p>\n\n\n\n

This did not seem to be an obstacle to the program, which immediately delivered the name of seven researchers, the name of the institutions they work for and the subject of their research. I refined the question and asked it to add links to these researchers’ studies.<\/p>\n\n\n\n

Several were broken. One of them was directed to research about cell biology.<\/p>\n\n\n\n

It wasn’t even from one of the listed authors. It is important to mention that the program crashed several times when this question was asked.<\/p>\n\n\n\n

Interestingly, once I have altered minimally the content of the initial question, asking for \u201cBrazilian researchers\u201d, the AI program provided the appropriate answer: there is no information, because the database only goes until the year 2021.<\/p>\n<\/div><\/div>\n<\/div><\/div>\n\n\n\n

Academic prejudice<\/strong><\/h5>\n\n\n\n

The constant occurrence of hallucinations and the lack of source of information practically makes it impossible to use only ChatGPT to write scientific articles. Still, artificial intelligence can be used as a tool by the researcher. It can be used in repetitive tasks that do not require creativity, such as formatting texts, for example.<\/p>\n\n\n\n

As soon as the program was launched, the journalist and doctor in communication Marcio Telles started publishing on his Twitter profile a series of tips on how to use ChatGPT to write academic articles, to prepare classes and to other useful aids for researchers. In one of the posts, for example, he demonstrates how to ask the program to organize the references of an academic paper in the APA format, a style of reference citation.<\/p>\n\n\n\n

In his posts, Telles points out that ChatGPT is a working tool, and not a replacement for authors. Even after such warning advice, he could not avoid negative reactions on the part of some members of the academic community. Telles declined to be interviewed for this report, claiming that his comments in favor of the use of ChatGPT have not only generated \u201cunease\u201d among colleagues, but also raised suspicion about his own work, what made him decide not to talk about it anymore. Since then, he stopped posting tips on Twitter.<\/p>\n\n\n\n

According to Luiza Santos, researcher of artificial intelligence and professor of Communication at the Universidade Estadual de Ponta Grossa (UEPG), this cautious stance of the academic community is historical and comes from Ancient Greece: Plato warned us of the evil effects that books could cause to human memory. The same fear took over with the emergence of computers and even Google. Time has passed, and today these technologies are considered essential to human thinking.<\/p>\n\n\n\n

According to the researcher, having a stance of complete “fear\u201d or “fascination” with the academic use of ChatGPT does not help at all. The technology is still new, and a lot of opinions are based upon pure speculation. However, she believes that the community should establish consensus and guidelines. She also mentioned the scientific journal Nature, which published rules for the use of ChatGPT in scientific studies. One of them says that artificial intelligence systems cannot be credited as an author or co-author of a study.<\/p>\n\n\n\n

\n

\u201cWhat ethical rules should the Academic Community follow? We need to establish consensus and guidelines. I believe that consensus will vary according to each field.\u201d<\/p>\nLuiza Santos<\/em><\/cite><\/blockquote>\n\n\n\n

\n

Could ChatGPT write a scientific paper for me? I have tested it: I have asked the program to write a ten-page article already formatted in ABNT, containing a literature review on online deliberation in internet forums. The article should have, among other sections, an abstract in Portuguese and English, a conclusion section and another section of references.<\/p>\n\n\n\n

A paper titled \u201cfindings and trends of online deliberation in Internet forums\u201d was promptly generated. It was much shorter than I expected: it had approximately five pages, counting the bilingual summary. The system already generated the keywords, which was an interesting touch.<\/p>\n\n\n\n

Some studies were mentioned in the body of the text but were identified only by the year of publication (\u201ca 2014 study showed that…\u201d). The article ended with a list of references already formatted in ABNT. The years of publication did not match those of the cited studies.<\/p>\n<\/div><\/div>\n\n\n\n

To learn so you do not \u201cget swallowed\u201d<\/strong><\/h5>\n\n\n\n

Engineering professor at the Federal Institute of Santa Catarina (IFSC) Jesu\u00e9 Graciliano da Silva remembers the date he first heard of ChatGPT: January 25th. Overnight, that was all anyone was talking about. Most of the news and messages that reached him were catastrophic. All student assessment tests would become obsolete. According to some professors, education would be destroyed.<\/p>\n\n\n\n

Jesu\u00e9 had no doubts about how to proceed: he went to test the tool. And he acknowledged: \u201cIt was a little bit above my expectations\u201d.<\/p>\n\n\n\n

While testing the ChatGPT prompt, he asked the machine questions about its use in education. He liked what he saw. Since he works with distance education and digital learning tools, the teacher has no prejudices against new technologies.<\/p>\n\n\n\n

The result of his interaction with artificial intelligence was the digital workbook \u201cElevando a aprendizagem com ChatGPT (Elevating learning with ChatGPT)\u201d, written by ChatGPT and arranged by Josu\u00e9. The publication provides a presentation about how ChatGPT operates, its potential and limitations, as well as suggestions for its use in the classroom.<\/p>\n\n\n\n

Jesu\u00e9 is not naive: he knows that the program has biases and that it requires attention when handled in the educational context. Therefore, his suggestion is to work with the question, formulating it together with the class and then discussing the answer. If the answer turns out to be wrong, this is also an interesting point for discussion in the classroom.<\/p>\n\n\n\n

For those who disapprove of ChatGPT, Jesu\u00e9 has a warning: even if the teacher does not know how to use it, the student does \u2013 and the program will be used for plagiarism purposes. It is with this purpose that the guide was organized: to facilitate the transition of teachers to this new scenario of artificial intelligence.<\/p>\n\n\n\n

\n

\u201cThe more teachers try using it, the better. Teachers need to seek self-improvement. It is their obligation to expand their capacities. And it is up to the educational institution to open the discussion\u201d<\/p>\nJesu\u00e9 Graciliano da Silva<\/em><\/cite><\/blockquote>\n\n\n\n

More than interesting knowledge, more than being good or bad, it is about survival. Jesu\u00e9 understands that ChatGPT will change the educational environment forever. If teachers do not adapt, there may be no place for them in the future.<\/p>\n\n\n\n

“Some of my colleagues tell me I am too enthusiastic,\u201d he says. \u201cNo, I am not. I want to handle it the best way I can, so I don’t get swallowed up.\u201d<\/p>\n\n\n

\n
\"\"
Above, from left to right, images produced by the Artificial intelligence application Dream Wombo, representing (from left to right) Elza Soares, Elon Musk, Paulo Freire and Fernanda Montenegro as if they had been painted by the Brazilian artist Di Cavalcanti (montage: Fl\u00e1vio Dutra-Dream Wombo)<\/figcaption><\/figure>\n<\/div>\n\n\n

<\/p>\n\n\n\n

The stance of educational institutions<\/strong><\/h5>\n\n\n\n

Consider these two antagonistic cases: as soon as the use of ChatGPT became highly popular \u2013 it gained more than 100 million users in the first two months \u2013 The New York Department of Education banned its use in the city’s schools. At the same time, technology professor Ronaldo Lemos, who teaches at Schwarzman University, in China, began to force the use of ChatGPT in the tests of his discipline.<\/p>\n\n\n\n

What explains the difference between the two approaches? Would it be the bold personality of the Schwarzman University educator versus the slower, more cautious culture of the government? Maybe. However, the question probably boils down to an age factor: the students of the higher education institution are adults; those of the New York schools are children and adolescents.<\/p>\n\n\n\n

Leonardo Foletto, editor of the BaixaCultura website and free-culture activist, agrees that ChatGPT should be used cautiously in elementary and high school. According to him, adopting a liberal stance is more difficult when it comes to teaching children at the literacy stage.<\/p>\n\n\n\n

Foletto cites three different positions of educational institutions regarding the use of artificial intelligence.<\/p>\n\n\n\n