Text generation

Purpose : Influenced by the hot chatGPT in recent years, the goal of this topic is to generate articles or sentences according to the existing text entered.

Dataset : IMDB.

Contains a large enough text file for training.

Development brief : Remove unnecessary elements from the data and use the TextVectorization layer to calculate the vocabulary to use. The concept of softmax temperature was introduced in the experiment, mainly to adjust the index of sampling probability distribution entropy.

Model :

Results from 60 epochs :

Because training this model requires a lot of computing power and more than 10 hours to reach the original preset 200 epochs. The hardware equipment is not good enough, so it only trains to 60 epochs. Therefore, the results are not particularly good.

Detail :

https://colab.research.google.com/drive/1PKq1VcoTm85eylyQBgoHuv9WZavEoYHZ?usp=share_link