Yea JWT maxes out around 8KB vs DynamoDB's 400KB
Also, the CloudFront storage problem exists with many other serverless frameworks like Zappa so I don't blame you for not solving it yet haha.
JWT is designed for authentication claims, and not to completely replace $_SESSION data, although technically you can stuff as much as you want into the payload. The wording makes me uneasy about the implementation - but decent advice otherwise. (Also note that getting JWT implementations/crypto right is somewhat tricky, obvs. Caveat emptor)
On a quick note, storing big blobs of data in DynamoDB is asking for trouble, and AWS actually recommend S3 for this [0]. Best to not find this out the hard way (i.e. huge bill).
[0] https://docs.aws.amazon.com/amazondynamodb/latest/developerg...