ctoa

When I did strength sports and would eat ~180g protein a day (which for me was 1.8g/100kg), I ate a lot less meat than you would think, I was carefully tracking all my food for a while and you have to count the whole diet.

I really like this study of a population of highly trained athletes and their diets/protein intake: https://pubmed.ncbi.nlm.nih.gov/27710150/

In that study they eat > 1.2g protein/kg body weight, but 43% of that is "plant sources", meaning grains, legumes, fruits and vegetables. Like one serving of oatmeal is 6g, things you don't think of as "protein" add up and you have to count them. The athletes in that study are Dutch and 19% of their protein intake came from bread.

But what always happens with protein recommendations is that they say "x grams protein/kg bodyweight" but people hear "protein is meat, you are telling me to eat x grams/kg bodyweight of meat." Very few people ever look closely enough at their diet to develop an intuitive sense for counting macros.

It's sort of an RNN, but it's also basically a transformer with shared layer weights. Each step is equivalent to one transformer layer, the computation for n steps is the same as the computation for a transformer with n layers.

The notion of context window applies to the sequence, it doesn't really affect that, each iteration sees and attends over the whole sequence.

You are probably thinking of Merrill (whose work is referenced towards the end of the article).