mcaledonensis

The trick with the F is a good one. Yes, tokenization is very likely one of the contributing factors.

The point of the eval is an attempt to exercise: attention to detail, tokenization, ability to stop, ability to follow instruction and ignore irrelevant instructions, biases, laziness, computational limits, etc.

Two hands have ~60 degrees of freedom (4 in each finger, 4 for extension/flexion/abduction and a few for a pose between the hands). You can speak with your hands! Or perform (shadowgraphy, puppeteering, magic).

In computer graphics rendering hands and hands gestures was considered very challenging for a long time (This was, for example, a PhD thesis of one of the Pixar co-founders, back when computer graphics was just starting). And it is still very challenging to render a close-up of a hand performing a natural looking gesture.

Yes, in this integration GPT3 connects with the image space only through text and doesn't consume or emit image space embeddings / tokens. With GPT4 that perception modality will be available directly in the model. And it'll be a better integration.

Likely yes. Considering that the pandemic was near perfect for VR, they are probably seeing contraction of their user base now. Even with perfect market conditions in 2022, that division of Meta reported $13.7 billion in operating losses for Reality Labs.