I believe the input state is the past 4 frames stacked or concatenated.