Hacker News Reader

Comment by carlmr | Hacker News Reader

It says you only plan UTF8 support and no Windows line endings. The windows line endings ok, and I get why in Rust it was easier with UTF8 (although there are some good crates for handling encoding). However this seems awfully restrictive to me.

View on HN

It’s a start

If it's a start it should still be planned though.

I've mostly built Amp to fit into my workflow, which is unix line-endings & ASCII/UTF-8 at the moment. Not opposed to alternative encodings, but I think that's a non-trivial addition. Amp uses a gap buffer under the hood and avoiding segmenting multi-byte UTF-8 graphemes/grapheme clusters was tricky. It's definitely possible, it just hasn't been a priority, yet.

Arguably this is somewhat of a kludge(?), but one option (which would involve less coding than making the engine work with different encodings) would be to use readers/writers to support other formats, and keep using UTF-8 internally.

UTF-16BE/LE and UTF-32, plus Windows CPs, are all really easy to do like this — probably others too.

(Note that depending on the range of codepoints used in files using these encodings, they may take more memory in UTF-8 form. This usually isn't an issue unless one is handling large files of Kanji)

I agree, especially as UTF-8 is standard inside Rust. Also the encoding library would make it easy enough to support other formats.

I'm not sure if the editor streams or loads into RAM right now. If it already loads into RAM this is a tiny change.

https://crates.io/crates/encoding

Line endings should also be unproblematic.

Reply on news.ycombinator.com