Project 4: Google’s transformer T5

An example of a modern Transformer that uses the encoder decoder structure is the T5 model from Google. This model reframes a range of tasks into a text-to-text framework, including translation, linguistic acceptability, sentence similarity, and document summarization. For the project submission, submit the following:  

1. See T5 transformer model description via image and examples from the book and explain its major components and its workings.

2. Create and explain your own example of cross-referential attention in style that would T5 use

3. Based on working of T5, how would you design a generative model that is able to create a short video given only a text prompt as input.