Shannon proposed that language could be generated based on the probability of the appearance of the next word in a sentence, based on the words that preceded it, but this idea was not widely accepted at that time, and even faced harsh criticism, most notably from the famous linguist Noam Chomsky, who described the concept of (sentence probability) probability of a sentence, as worthless and useless.
But 74 years after Shannon's proposal, the world witnessed the emergence of ChatGPT in 2022, a chatbot based on an advanced language model that surprised everyone, and even led some to speculate that it could be a gateway to super artificial intelligence.
The long time it took to move from Shannon's idea to ChatGPT is due to the need for huge amounts of data and supercomputing capabilities that were available only in the last few years. But how do large language models – on which chatbots such as ChatGPT, Gemini, and others rely-produce texts that look smart and coherent, and how influential are they in the field of creative writing
How do large linguistic models produce texts that look smart and coherent?
The large language models were trained on a huge variety of texts available online, including books, articles, websites, conversations, and code, and this wide exposure enabled the models to learn linguistic patterns, grammar rules, semantic meanings, and relationships between words and ideas.
Chatbots like ChatGPT and Gemini follow this probabilistic model of language generation, choosing the next word based on a Probabilistic Prediction, imagine it as if you were pulling words out of a hat, and the words with the highest probability are represented by a larger number of copies inside the hat, as a result, these robots produce texts that look smart and coherent.
Large linguistic models.. Random parrots or creative gadgets?
It is necessary to distinguish between the concept of creativity shown by large linguistic models and real human creativity, since at the initial stages of the emergence of such models, it was easy for people who had limited perceptions of computer capabilities, to attribute the attribute of creativity to the results of these models. While others have expressed a more conservative and critical attitude, Douglas Hofstadter, an American cognitive scientist, has described what these models offer as” an amazing emptiness hidden under their bright and superficial appearance".
The linguist Emily Bender and her colleagues also provided an accurate description of the way large linguistic models work, describing them as random parrots, which means that these models reproduce what is contained in the big data that she trained on in a random way.to understand this mechanism, it is necessary to consider why a linguistic model is generated for a particular word, the reason lies in the fact that this word has a relatively high probability of appearance, and this high probability is due to its repeated use in similar contexts within the training dataset.
Hence, the process of choosing a word based on a probability distribution is very similar to choosing an already existing text that has a similar context, and then using the next word that is mentioned in that text. Based on this logic, the process of generating large linguistic models of texts can be viewed as a kind of plagiarism.
The essence of human creativity, on the contrary, lies in the presence of unique ideas and visions that the individual seeks to convey and express. So let's analyze the creative process of a person who brews in his mind a bunch of unique ideas that he strives to embody and transfer to others, using the techniques of generative artificial intelligence.
In this context, this person will begin an interactive journey with the machine by formulating his precise ideas and turning them into a detailed textual prompt, and this prompt is considered an initial seed, from which artificial intelligence will take off to perform the production task, which may manifest itself in the form of a coherent linguistic text with specific meanings, or in the form of accurate visual images reflecting the user's perceptions, or audio clips with distinctive characteristics bearing his artistic imprint.
And in a scenario where the user is indifferent to the quality of the final output or its intrinsic nature, the choice of a textual claim becomes secondary and insignificant, in this case, the user does not pay much attention to the exact characteristics of the input, as long as the machine produces something.
But the situation is radically different when the user is very interested in the final results, looking for outputs that correspond to their vision and creative aspirations. And here, the role of the large linguistic paradigm becomes central.
Limitations of models in the simulation of authentic creativity:
Large linguistic models in their work seek to simulate the writing style that a random person might follow who wrote the texts trained by the model, but most creative writers, who are distinguished by their unique style and distinct vision, are not satisfied with texts bearing the fingerprints of a random person, they are eager to embody their authentic creativity in every word and sentence, and may look to use tools that help them produce texts that they might write themselves if they had enough time for this.
But large linguistic models usually do not have a wide enough collection of works of a particular author to learn from them in depth, and this lack of special data makes it difficult for the model to assimilate the unique style and exact preferences of this author, as a result, the author undoubtedly aspires to produce a work that bears its own imprint and stands out from everything else.
If the expected output requires a level of detail that goes far beyond what was included in the initial input, it becomes necessary for the model to add details on its own, and these generated details may fit and correspond to the previous intentions and perceptions of the writer, which enriches the text and makes it more accurate and deep, or they may be completely far from his original vision, which requires human intervention to correct them and direct them towards the desired goal.
This highlights the fundamental challenge that large linguistic models face in simulating authentic human creativity that stems from a personal vision and a specific goal.
The potential of generative artificial intelligence in creative writing:
The processes of creative writing and complex software development share fundamental similarities. while a specialized software developer is engaged in converting an abstract idea or precise functional requirements into a series of structured code, in essence, texts written in a specific computer language, the creative writer performs a similar task, transforming his perceptions, ideas, and inner visions into texts written in a natural language rich in semantics and expressions.
It is remarkable that large language models deal with writing code – including logical structures and executive orders, and writing texts in natural language – with its various meanings and linguistic contexts – in a similar way, and this unified treatment is due to the fact that the huge text set trained by these models contains huge amounts of natural language texts, which cover various aspects of human knowledge, and huge amounts of code written in various programming languages, which represent solutions to various computer problems, so the nature of the final product-generated by these models – Depends on the specific context of the initial inputs (text prompts) that you receive.
In this context, creative writers can draw some valuable lessons from the experience of software developers in using large language models, as these models excel in dealing with small and specific projects, which have already been completed by a large number of developers repeatedly, such as: creating specific queries to databases to extract certain information, or generating scripts for typical emails or routine documents.
These templates also provide valuable assistance in specific parts of large and complex software projects, such as creating text for a pop-up dialog box within the graphical user interface of an application.
However, if programmers or writers want to take advantage of these models in more complex and detailed projects, they should be ready to produce several possible outputs from the model, and then do a careful selection and adjustment of the output that is closest to the desired goal and the specific creative vision. The main challenge in software development has never been the process of writing the code itself, but always the process of determining the exact and complete requirements of the project with unambiguous accuracy and clarity.
Claims engineering.. The art of directing artificial intelligence:
With the rapid development of large language models, the need to formulate effective inputs to obtain the best possible results has emerged, and this has led to the emergence of what is known as prompt engineering (claims engineering). this emerging field aims to explore and develop specific techniques to improve the quality and relevance of outputs generated by existing models.
Experts in this field have proposed a variety of practical techniques to improve the output of models, including:
Multi-step prompts: this technique involves dividing a complex task into several sequential steps. for example, the user can first ask the model to create an outline or structure of the required content, and then submit a second prompt asking the model to create the actual text based on this previously prepared outline.It is believed that this method helps the model to better understand the task and generate a more organized and coherent output.
Chain of Thought: This technique is based on asking the model to reveal their thinking process in an explicit way. Instead of just providing the final answer to a question, the model is asked to explain the logical steps or thinking steps it took to reach that answer. The model uses these intermediate steps as part of the claim itself, which helps it improve the accuracy and reliability of its final answer. It is believed that this technique encourages the model to conduct a deeper analysis of the problem and offer more detailed and logical solutions.
However, it is necessary to realize that the nature of these tips and techniques used in claims engineering is likely to have a limited shelf life.if a particular technique in claims formulation proves to be effective and capable of achieving noticeable results, it is expected that this technique will be directly integrated into the internal structure of future versions of the model. this means that the positive impact of these techniques will become an inherent part of the behavior of the model, without the need for the user to manually apply these techniques each time.