How are the inputs look like

[nemo0526]

The HowTo100M require a huge storage that I can't make a training. May I ask that how the videos and the texts look like before put into transformer module(like[31373, 11, 1312, 716, 36945...]or[0.0524,-0.0960,-0.1728...])? Do you directly contenate the tensor after feature extraction and put into transformer module without any adjustment?