> I think that having different conceptual kinds of strings be distinct types would eliminate a lot of bugs. I'm not aware of any language or standard library that does this, though.
Foundation seems like the obvious one? NSURL vs. NSString?
Foundation seems like the obvious one? NSURL vs. NSString?
If you want type, yank the data out of the string representation and into an actual data type!
"The string is a stark data structure and everywhere it is passed there is much duplication of process. It is a perfect vehicle for hiding information." -- Alan Perlis, Epigram #34.
Well, the very act of "yanking" can be helped by more well-defined behavior policies of "typed strings". It is unavoidable that the programmer will have to confront a contiguous piece of memory (string) with no boundaries between characters whether it's a block of bytes on disk, network stream, or the clipboard buffer. It can be helpful to differentiate the various types of strings with a formal type system before the yanking takes place.
The Alan Perlis epigram is wise but I don't think it applies here. I think's he's talking about abusing strings as an adhoc data format by stuffing them with multi-value data and then turning right around and parsing them. For example, concatenating name+":"+id+":"+amount into a string with colons like "John:389432:$5.99", passing the "complex" string around, and then later parsing out the colons with string.split(). Basically, the programmer is making an in-memory csv (comma separated values) data format instead of using "C struct {}" or passing 3 separate formal parameters.
I take it that you are advocating for typed strings in addition to fully-formed types for things like URLs and SQL, but I think all that will lead to is a lot of gratuitous casting of strings just so they can be passed to the fully-formed types. Furthermore, few of these casts could be statically checked for validity.
But if programmers are not directly doing memcpy() of bytes directly to the stack frame, they are already doing a roundabout version of "casting".
If a programmer has to "yank" from a string to put it into a more organized data type, the "yank" has to be done within the framework of semantics. If one "yanks" or "slices", or "substr", etc characters 3 to 15, the exact behavior depends on whether contiguous bytes is ASCII, UTF8, UTF16, or binary blob (which means NUL(0) is valid byte in the middle of the block of bytes).
That "meaning overlayed onto those bytes to manipulate it correctly" has to decided somewhere. It is either implicit or explicit.
Likewise, if one pulls 4 bytes from disk or network, the programmer has to assign meaning to it and say it is the 4-bytes of an IEEE floating point (float) or 4-bytes of a integer (int). The programmer may later want to do a bitshift (<<) which is not applicable to a float.
>That "meaning overlayed onto those bytes to manipulate it correctly" has to decided somewhere.
Sure, but I don't think it has to be done twice. A string type serves the purpose of abstracting the stringiness of a byte sequence, and an URL type serves the purpose of abstracting the URL-iness of strings, and to my mind, adding something to abstract the URL-iness of a byte sequence is a gratuitous redundancy that conflicts with the principle of separating concerns.
But NameAscii.CharacterIndex(3) is a different meaning than NameUtf16.CharacterIndex(3). The byte index could be 3 or 6+n depending on semantics.
>Sure, but I don't think it has to be done twice. [...]abstracting the URL-iness of strings,
There's no redundancy. If I have a config file that has user-specified URLs in it. It is more likely that the programmer will know if that file on disk is ASCII or Utf16. The programmer can know that before the compiler or runtime can know it and assign a formal type to the incoming string. This is independent knowledge from a URL data type. A URL data type isn't going to have member functions to pull data directly off of the disk. That's a separate concern as you say.
>, adding something to abstract the URL-iness of a byte sequence is a gratuitous redundancy that conflicts with the principle of separating concerns.
We are envisioning different things. I thought of it as:
- bytes -> typed string -> URL
- -- typed string
- bytes /
- \-- URL
bytes -> string
-> URL
-> SQL
...
bytes -> string
-> URL-string -> URL
-> SQL-string -> SQL
...
bytes -> string
-> URL-string -> URL
-> SQL-string -> SQL
...
Edit: I don't think these diagrams are formally correct - I don't think of URL as a subtype of string, even though it has a string representation and URL objects can be constructed from strings - but I think they make my point clear.
I was thinking of ASCII/Unicode string types and not the URLString vs SQLString types. Yes, a URLString would be redundant. In C#, the URI data type contains a Utf16 string as a data member but it is not another type of string.
We were talking about different things.
The ASCII/Unicode string data types would be somewhat analogous to int32/int64/float32/float64/Money data types.
The way C# does it is that everything is Utf16 and there is no separate ASCIIString data type for pure ASCII strings. They also don't have pure Utf8 strings. Instead, the UTF16 classes has member "get()/encode()" functions that interpret as ASCII or UTF8. The ASCII/Utf8 data state is transitory. This architectural choice is fine for most apps but if you have 1 gigabyte of ASCII data, it's very memory inefficient to double the memory footprint to 2 GB for Utf16 just to slice & dice strings. If you use byte[] arrays to keep everything as raw ASCII to conserve memory, you don't get any convenient string-like slice & dice functions. The "string type" is lost in a opaque byte[] array.
You started by disagreeing with kazinator's objection to typed strings, and for several posts after that you explicitly defended the concept of typed strings. Now you are trying to rewrite the discussion as if it were about the internal representation of strings, which it has never been.
You misunderstood our subthread. Kazinator was talking about "fat" strings. I was pointing out that there can be "string types" that helps detect defects. We had 2 different ideas of "string types". I can't say we disagreed insomuch as clarified what each of us was talking about.
I also don't care for your accusatory tone as if I'm deceiving people. My examples of string types has been consistent from the very beginning. I was never talking about NSURL/UrlString but you did. I simply didn't pick up on that divergence which was huge.
Also, an internal memory representation has ramifications on external semantics. That's not a rewrite. That's been my point all along (see my earlier clipboard example). A sequence of of bytes on disk, or network port, or memory always requires correct semantics to process.
"The string is a stark data structure and everywhere it is passed there is much duplication of process. It is a perfect vehicle for hiding information."
My "string type" examples to help uncover defects in parsing & yanking were not relevant to Alan Perlis criticisms. The "hiding information" of strings is an orthogonal issue.
>My "string type" examples to help uncover defects in parsing & yanking were not relevant to Alan Perlis criticisms.
Did you mean to write that Alan Perlis' criticisms were not relevant to your examples? Anyway, this should perhaps have brought it to your attention that you were going off on a tangent, given that Kazinator chose to qualify his point with this quote.
It is not unreasonable to interpret that sentence as some kind of "universal" advice. If he meant it in specific circumstances, he might have said, "solve that kind of bug" instead of "solve bugs". (It's not even clear if kazinator felt that NSURL is a bad example of string-but-not-really-a-string. NSURL is an object with well-defined structure and not a pure string. It therefore doesn't fit the criticism of the Alan Perlis epigram)
If I went off on a tangent, then I felt the door was opened by kazinator's (seemingly) universal statement which expanded the scope of discussion. Whether kazinator meant it universally is irrelevant because that's how I read it and that point in time is long gone. All my subsequent replies flowed from that (mis)interpretation. In any case, writing universal replies to posts with specific examples is not uncommon on HN or any forum for that matter. It's typical ebb & flow of discussions.
Lastly, I looked at your history of posts and you're just a generally argumentative type of communicator. The mods have asked people to read posts with a charitable interpretation but you don't seem to follow that spirit. I'm fine with debate but I'll ignore your posts with needless hostility. Regards,
As it happens, I was considering the general case, and the examples of URLs and SQL came from Mike Ash's article, where he was considering the interpretation of text as information. My interpretation of cballard's post is that he regards an URL to be primarily a string, and therefore that an NSURL is, in some way, a typed string. I didn't think it was a good example, but as I was making a general point, I did not see that as mattering - I could make my point with Mike Ash's general examples instead.
My interpretation of Mike Ash's original statement is that it would be useful to have different string types where strings have different interpretations, and I disagree - the semantics of a type do not depend on its textual representation (if that were so, it would not be possible to translate those semantics between languages.) If I now understand your position correctly, it is that it is useful to have different string types because there are several different ways to interpret the underlying bit sequence as a valid string. I am not convinced that is the best solution to the problem created by us having incompatible bytes-as-text representations, but I agree that it is not wrong to consider it as an exception to the general rule.
Take it from someone that used to be very uncharitable in my interpretations of other's words and actions and was subsequently very argumentative and hostile: this is something you need to extract from yourself.
The sooner you take that step the better because it takes a long time to retrain yourself out of bad habits. I'm still working on it and every time I fail I wish I'd started sooner.
And even if you think I'm wrong, look at this way: You're not going to convince the person you're doing this to of your accusation and the other people reading it are going to ignore your argument and will just think you're being a hostile asshole. At that point you may as well be talking to yourself.
The correct place is at the earliest possible point: the input into the system. Parse the textual cruft into a data structure as early in the data processing pipeline as possible. (And treat any deviation from this cautiously, for the sake of optimization.)
Totally agree. But whether we "parse" or "yank", that processing requires semantics. Data types can help with correct semantics to interpret the textual cruft the way the programmer intended.
The other issue of strings acting as adhoc compound data formats is orthogonal.
That said, I completely feel what this blog post is saying: strings are _hard_, especially if you're not just doing ASCII.
1) 40 years of crazy encodings and languages.
2) human languages are wildly diverse and basically any assumption you wish to apply is broken.
For 1, any system that wants to deal with the outside world needs to deal with: operating system encodings (arbitrary bytes on unix, malformed UCS2 on windows), C representation (null-terminated strings), systems that only work with ASCII, systems that only work with utf8, systems that work with arbitrary encodings/languages (HTML). This is arguably unnecessary complexity that exists because of short-sighted decisions in the past.
2 is the necessary complexity; the fact that languages are really complicated.
There are thousands of symbols in writing. Do you try to encode these symbols in a monolithic manner, or in a compositional way? For historical reasons, you can often do both! ë can be a single character, or e with an accent modifier. How do you handle string searching in such a model? Do you match `noel` with `noël`? What's the length of noël? 4 characters? 5 characters? bytes? graphemes? codepoints? Can you correctly reverse noël (do it wrong and you can get leön)?
Different letters which have similar/identical representations but different semantics/origins! Is Ε "capital e" or "capital ε"? How do you upper-case or lower-case these letters? Do you expect to_upper(to_lower(char)) to roundtrip (it won't)? Do you expect capitalization to be doable in-place (it's not)? Do you expect capitalization to be region-specific (it is)?
Are any of these operations even coherent in a language like Japanese? Why are you trying to do them?
God help you if you want to display this text. Are you ready to handle right-to-left text? Are you assuming that your font is monospace (hey there terminal and text editors)? C̢̫a̘̺̯n ̘̜̦̹y̷̫̼̘̩o̶͉u̗̩̻̞ ̻ẹ͡v̴̤͎̹e̶̫̠̤̭̺̤̞n̛̞̹̣̩̲͉̮ ̜͖̪͔̖d̤e̘̯ͅa̺l̟̀ ͚̗̣w̭i̸͇̠̥̣̜̥t̸h̸̻̮̼̙̹ ̗̺̱̣̰̱̙z̟a̺͜l̠̦̖̟̰͍g҉̜͖͓̫ơ̩̹̰͕?̹̳̼̯̘̺̟
Each has their own unsized view: str, OsStr, Path, and CStr.
We also provide AsciiExt for those times where you really truly believe you want to be working with a String as Ascii.
That said, we generally try to make it as ergonomic as possible to pass a plain str where a Path/OsStr is expected. This is because utf8 is a subset of wtf8, so it's always fine to convert in that direction blindly (and it's really nice to just be like `File::open("foo.txt")` when hacking something together). This is why so many interfaces are riddled with something like `P: As<Path>`. The differentiation largely exists for the other direction, IMO. Paths and OsStrs aren't guaranteed to be valid UTF8, and shouldn't be provided where a proper utf8 string is expected.
Path is just a convenience wrapper over OsStr that understands the platform's seperator conventions and provides convenient utilities.