Hacker News Reader

Token list, not token trees. There are official libraries for parsing token stream as rust code but you can parse it as anything (eg json, html) if you want to.

View on HN

I think you meant TokenStream. They are trees, behind the scenes, because matching delimiters happens early on between lexing and parsing. By the time the rustc_proc_macro::TokenStream is exposed, the rustc_ast::tokenstream::TokenTree is hidden to the proc macro API.

https://doc.rust-lang.org/stable/nightly-rustc/rustc_ast/tok...

https://doc.rust-lang.org/stable/nightly-rustc/rustc_proc_ma...

Reply on news.ycombinator.com