Your Language Features Are My Libraries
blog.bitquabit.com
blog.bitquabit.com
Of course it is hard to do programming without doing any metaprogramming at all, so new languages, like C#, have some basic metaprogramming features. But you have to market them as features of an otherwise fixed programming language or you'll scare people off. It's all about minimizing the number of things your customer has to think about. It's even worse if management is driving the choice/design of the language.
(I wrote a whole book on how to best use Catalyst. A year later, it's nearly all wrong. This seems to upset people, as now they "are wrong". But it doesn't really matter; they can incrementally move in the direction of what we consider "not wrong" today, and, wrong or not, it's not like their app doesn't work. Personally, I am happy that everything becomes outdated so quickly, as it means that I can be more productive, and my apps can be easier to write and maintain. But people are weird...)
I think programmers will also trade actual flexibility for an IDE that appears to make their language flexible. Are there any really popular languages today where bloated IDEs aren't "required"? (I consider Java and C# to be the "really popular" languages today. I don't use them, but millions of other people do.)
The main problem with all of this is that programmers aren't really responsible for deciding which programming technology will be used. A programmer wants macros. A hiring manager wants the ability to throw a dart at a list of names and find a programmer that knows that language. Guess whose input wins? (Combine this with the fact that people think programming is a "cool" profession, and you end up with 90% of the practitioners not actually knowing how to program or what programming is. Does it surprise anyone that cool GUIs beat complex computer science topics?)
The good news is, you don't have to be part of the problem. There is hardly any reason to use the "popular" languages. They don't have better platform support, they don't have better libraries, and they don't have better tools. So don't worry about them, they are for other people :)
These things together mean that using a popular language in the way that most people use it is easy -- it's the path of least resistance in a lot of instances. Using languages that are cool, have neat features (or the ability to build them), and/or have higher abstractions is fun for play, but it turns out to feel like you're swimming upstream when you have to do the boring requirements-laden task that you've been handed.
There's more than one reason for that, but collectively they add up to why I use PHP for things I just want to get working quickly, or for job-like things, and lisps or other small-community languages when I'm doing something just for fun.
Economics has given us the wonderfully pretentious sounding term "positive network externalities" to describe products that become more valuable, the more people who use them. Simple concept, big, impressive term.
http://www.welton.it/articles/programming_language_economics
This is why I'm not interested in Clojure, regardless of how nice a lisp it may be. Java libraries are poorly-designed, the prospect of wrestling with them again brings back memories of frustration. And you can bet that no one will ever write better ones, because the community will discourage people from "duplicating functionality."
It is easier to just steal Clojure's good features for CL. (funcallable everything is kind of nice.)
There are many Clojure libraries that attempt to reimplement existing Java libraries. For instance, Ring is a HTTP specification for Clojure that reimplements much of the functionality of Java servlets.
One of the best things about Clojure is how it makes Java libraries suck much less. For example, line-seq turns a Java BufferedReader into a lazy sequence of String's, one for each line in the file.
You still have the suckiness of the multiple lines it takes to instantiate and clean up the BufferedReader, but macros like with-open makes that suck a little bit less, too.
Stir in some macro-fu of your own, and now you have the benefits of Java libraries minus the boilerplate suckiness. Basically, it turns Java into a "your language features are my libraries" language, no mean feat.
I'm working with matlab currently. It is simultaneously the best engineering tool ever and the worstest, lousiest development environment ever. It has tremendous shortcuts for doing things quickly and powerfully but everything has been added in haphazard, half-cocked, half-assed fashion.
I can see how the two face of matlab have evolved together in the way that Perl has amazing power and horrible maintainability. I don't see that they would have to have evolved together. Ruby, for example, does what Perl does but with more sanity.
Details and exact choices of libraries can be discussed, but... could you be more specific?
Or were you talking of 1990s code, before e.g. testing took over the Perl world? Nah, straw men like that on HN... not possible.
Again, Perl would have more justification if Ruby wasn't visible
I was hoping that any answer might be insightful.
(Readability is mostly a function of familiarity. Everything has advantages/disadvantages, if I'd list the problems with Perl, readability wouldn't be there.)
http://www.dagolden.com/index.php/199/time-for-second-editio...
I'll research lambdas.
But I meant it also in saying that Matlab is the greatest engineering tool ever. You can do amazing things in 200 lines. Then you do 400 lines and get caught in some BS for days. But if you remember to do only 200 lines (even just 2 lines), then you can be a very, very good and feel very productive.
That doesn't change the fact that I'm very glad most Smalltalks ship with ROE or GLORP, or that C# ships with LINQ. It just says that I want to be able to add features as easily as the language designer can.
If a language does not have closures nor more complex control flow features, you really can't implement PL features as equivalently easy-to-use libraries. IMHO, that's why most mainstream languages continue to add superficial features rather than fix the core problems in their PL.
(edit)
Without eventually making your own?
If you don't need additional libraries then you are probably doing things that the authors of the language/platform/framework thought would be necessary.
The other language language/platform/framework, which I am familiar with, is ColdFusion. If you are doing web development and fit with-in only the functionality available to you by default then you are fine. If you want to abstract away certain things or do more than average web development then outside libraries are necessary, especially in the user-interface side of things.
I nominate Ruby as the first "your language features are my libraries" language to become legitimately popular. This is likely due to the fact that Rails provides the libraries that most web programmers want, in addition to the cool language features. But it also seems that the "Rails community" is enamored with how easy Ruby's flexible syntax makes it to extend the language in interesting ways. Thus, all of the fuss over adding methods to base classes, and whether or not, or when, it is a good idea to do so.
So I would say that Ruby has brought the idea of language features as libraries into the mainstream, as much as anything as geeky as programming languages can be considered mainstream.
From the standpoint of an editor hacker, I love simple languages. When I write Lisp, and create some "new construct", I just follow a few simple rules and my editor can understand the code perfectly. (Name your definition-creation macros define-something-or-other, and emacs will syntax highlight them correctly. Perfect.) When I am extending a more complicated language, the editor hackery is much more involved.
A bit of backstory, Perl now has a module called Devel::Declare, which lets you hook the parsing machinery and run your own code when you see certain things; this basically adds macros to Perl. "It's not a source filter." A number of people have taken advantage of this newfound extensibility to add some great features; try/catch syntax, class definition syntax, etc. I became a heavy user of this stuff, and I wanted emacs to understand it. 3 days of hacking and 200 lines of code later, it does. But that's a lot of effort to go to. When extending Lisp, I had to do nothing. When extending Perl, I had to hack on my editor for a few days to get everything right. (And don't ask, "what happens if you don't use the module that introduces the syntax; does emacs detect that?" because you won't like the answer. But, try typing "define-thing-that-is-not-defined", and emacs will syntax-highlight that as though it's a builtin, and not as random garbage you just entered. Oh well :)
I think this must be a mental barrier for those who are not comfortable with both the internals of their language and with the internals of their editor, and that's why users rely on The Language Designer Gods to give them features.
But programmers who have that mentality probably should wait for The Language Designer Gods to give them features. These are the same people who should not be allowed to write frameworks and should not be allowed CPP macros and the use of #doesNotUnderstand: in Smalltalk.
It happens in C++, too, which has an extremely complex syntax. C programmers are used to thinking of operators like + and - as language features because their meaning is defined by the language and can't be changed by programmers. C++ operating overloading continues to be extremely controversial because people who are used to C get angry that + and - can "unexpectedly" have meanings other than the meanings defined in the C language.
I know that's a special case, nothing like the general mutability of Lisp syntax, but I just wanted to point out that the differences pro (superior freedom/expressivity) and con (fear of confusion) happen whenever one language is slightly more programmable than another, not when a language has simpler syntax. (Of course, any sane person designing a programmable language would prefer minimal syntax.)
In C#, ?? should be understood by anyone who knows C#. In Smalltalk, ?? will be understood by the implementer and then need to be explained or learned by every fresh set of eyes. (Yes, they might expect it mirrors C#, but the problem still exists. The eyes know Smalltalk, not C#.)
It is similar to the situation in http://www.37signals.com/svn/posts/1729-coding-style-helpers.... link_to, easily understood, rss_link, not quite sure.
In practice, there are two factors that work in opposite directions, and which factor you're looking at changes pros and cons of language feature vs. library discussion.
* Time difference - for language features you have to wait until implementors implement them. For libraries you can start immediately. That gives you time advantage, sometimes for as much as years.
* Pressure difference - to make it standard language features there are lots of pressures to make it right, and effort is taken accordingly. Similar amount of effort is needed to make a library feature into standard position. But for those who wrote that library feature in the first place, usually their need is fulfilled easily, so there may not be enough incentives for them to push the feature into standard.
I feel the time advantage the 'library' approach gives me is crucial, so I prefer flexible languages (specifically I use Scheme). But in this side I do see the effect of the latter---there are tons of libraries, each of which is "good enough" for a specific circumstance but a bit short to be truly general, reliable solution.
nil ?? [ self halt ]
Your division between access by the implementer vs. the "fresh set of eyes" is a misconception you're taking from C#. There is nothing in Smalltalk that demarcates the "implementer" from the regular programmer. It's "Turtles all the way down!" You can actually implement a debugger as an ordinary Smalltalk app, and in 5 minutes, you can be browsing a stack. There is little difference between programming and meta-programming.
In other words, Meta-programming is easy and natural, not esoteric.
You just see Smalltalk as a funny sort of Blub where no one knows what the language/library will look like, because you see things in terms of the programming you know: Blub. Take it from me, I've been to dozens of Smalltalk shops over a decade: this is not a problem. It's sometimes a huge advantage! One can create very powerful domain-specific languages and tools this way.
False. The SQL-like snippet of LINQ in the article is just syntactic sugar for
people.Where(person => person.Type == PersonType.Adult && && person.Location != "New York")
public static IEnumerable<TResult> Zip<TFirst, TSecond, TResult>(this IEnumerable<TFirst> first, IEnumerable<TSecond> second, Func<TFirst, TSecond, TResult> resultSelector)
using (IEnumerator<TFirst> e1 = first.GetEnumerator())
using (IEnumerator<TSecond> e2 = second.GetEnumerator())
while (e1.MoveNext() && e2.MoveNext())
yield return resultSelector(e1.Current, e2.Current);
}
Yes my friends, we are deep in the Turing Tarpit, where the boundary between language and library is blurred, baroque architecture looms menacingly in the mist, and the bare metal is somewhere far below the fetid surface of the swamp. I hope you brought your safari hat and your elephant shotgun.
I like language features because they provide a clear distinction between the meta (things that I have to do due to the limitations of my execution environment) vs. the actual steps I'm taking to calculate some result or provide some service. When I see a language keyword or operator its clear to me that this is glue/implementation, and when I see a function call its clear to me that this is an actual step related to the algorithm. Classes and modules and templates and all these "objects" ideas have little to nothing to do with algorithms, they are simply the way we choose to implement those algorithms.
This varies language to language of course. Smalltalk has the advantage that I can create library features that "look" like language features. This is a big plus in my book. I think if I were using Smalltalk I would probably not have a problem with this at all. However, taking an example like JavaScript, I very much dislike this style (which many js frameworks currently use):
classCreate("MyClass", mySuperClass, properties);
//...
cosine(angle);
var MyClass = SuperClass.extend({ ... properties ... });
Essentially, the problem I keep running into with referential data structures is that haXe, like most popular languages, will only let you set them up in an imperative fashion, one dependency at a time, which tends to both spread out and bloat application code. What I want is to write my dependencies in any order and let the resolution be sorted for me.
I'm pretty sure a language with AST access and quoting like Lisp makes it relatively easy to write something along these lines, but in haXe, I had to write about 500 dense lines of code, most of it going to parsing and fighting the limitations of the type system. It still has some major bugs that I'm trying to resolve with a refactor at the moment.
The funny thing is, I know I'm not just fighting haXe here, I'm also running into the limitations of the platforms that haXe compiles to. The newest strategy I'm applying to resolve references is to add a "__parent" field to the reference stubs, which refers to the container(an array or record type) of that reference. I use that later, after all references are resolved, to drop the final value into the place of the stub, but as it turns out this causes difficulty printing out the working structure. Neko will recognize the value of __parent as "....", but Flash 9/10 will recurse into the field again and get a stack overflow.
So I had to add more code to write a pretty-printer which is aware of such traps.
1. Standard coding practices & constraints 2. Code review
So what were the reasons again that a team shouldn't use the most powerful language at hand ;)
Except for C#, PHP, Python, Ruby, Perl, Haskell, Io, Lua, Scala, Groovy, D, F#, and even Visual Basic.
Really the only major post-C++ languages to avoid operator overloading were Java, JavaScript, and Objective-C.
I've seen an argument that Java learned the wrong lessons from C++. They saw how complicated operator overloading was in C++ and avoided it, despite the fact that the worst complications in came from mixed pointer/value/reference calling conventions (which of course wouldn't affect Java). C# proves that it's completely possible for a Java-like language to have safe and easy operator overloading.
a == b // same object
a.equals(b) // same valueDo you mean that the present == for primitives should be replaced with .equals(), and the == for object reference equality should be replaced by perhaps === or .refEquality(), and then the .equals() for objects should be left alone. and then, == be added as syntactic sugar for .equals().... Are those assumptions what you meant?
If so, the same outcome can be achieved with by changing object == to ===, and object .equals() to ==. However, there are reasons for treating primitives as objects other than this.
I expect you've a model in mind from some other language that resolves this in along different lines (C# would be one), and so you haven't felt the need to articulate the assumptions needed to make it work. If I've guessed wrongly, can you elaborate please?
For immutable objects like ints or Strings the implementation of equals() should compare internal values, because reference equality for passive immutable things is practically never useful. This should already obviate the majority of explicit calls to equals() in actual Java code, because most of them are comparing Strings anyway.
For mutable objects the default implementation of equals() should compare object identity (hashCode), but be overridable if the user wants something exotic.
Algorithms and data structures relying on equality, like hash tables, should provide the option of supplying the equality operator or hash function at construction time, just like printf frees the user from the need to commit to one Integer.toString() implementation in perpetuity.
This all sounds like a reasonable scheme to me. I can flesh it out in more detail if you request.
Also, if you make Set.equals() depend on contents, you'll have to change the implementation of hashCode as well, wreaking havoc on sets containing mutable sets. So you really should only override equals() if your class is immutable or you're feeling especially exotic.
But I'll say that naming is tricky, since programming languages are really human languages, and as such are affected by the familiarity of their specific users as much (or, at times, more) than pure logic. Highly illogical, I know.
EDIT: The present implementation of Set.equals() depends on contents; it's independent of hashCode, and there's no problem with sets of sets.
Meta-programming is much more powerful than just plain programming. It's also not as widely understood.
And no, you can not put every language "feature" in a library: what about features like static typing, isolation of side-effects... Even features like threads can be a library, but the compiler has to know about this library and its particular semantics...
So yeah, we all know languages like Lisp or Smalltalk are very expressive. But current C# is also quite powerful and relatively pleasant to work in, and it's goal is rather to bring more features to programmers and to evolve this "blub". And of course I hope someday the average programmer will write in a Haskell-like-but-even-better language :)
Adding a feature like ?? to C# isn't hard in itself, at the parser/CodeDOM level. What makes it necessary to be careful about limiting the scope of the language is the tooling, the IDE, the ecosystem work.