optionalparens

Firstly, let me say you're most likely correct. I do find finger trees useful in functional languages as well and for creating immutable structures with them.

I want to add some general comments about anyone thinking about using a finger tree or anything else.

Give any potential data structure a try with real data, not contrived stuff and benchmark on the actual target hardware and in a real code path if possible. Further, try it with real-world access patterns like adding, removing, ordering, etc. at different times at different sizes - the results can be interesting. Even if something does well for one use-case, the other use-cases you need it for it may perform awful, so sometimes it's a compromise. Data structure performance is quite often not performance for a single operation, but a function of several operations. This makes it even more important to test it with real data when possible and access it different ways.

Moreover, too many times I see positive or negative benchmarks where someone is filling a data structure with exactly the same integer, or the same instance of an object or something like that. In practice, the underlying data can introduce other considerations based on the size, pointer chasing, memory allocations (especially if the data structure can dynamically resize) and so on in just about every language. Factor in hardware, and the results can change even further. Also ensure your benchmarking itself isn't flawed - we've been there and discussed it ad-nauseam and yet I continue to see bad benchmarking in the wild (particularly on the JVM or CLR).

As a related example, I've found that even the famed link list can suck in real-world. I've always been a "choose the right data structure for the job" kind of guy, and yet sometimes it's more deceptive than it seems. You might say, oh, my access patterns are exactly a linked list, and yet a simple array with lots of helper cruft code will smoke it in practice. Likewise, I've seen shocking results replacing something with a skip list or other data structure that seems like it should perform worse, but depending on the data, size, hardware, and implementation, I get interesting results. I've seen plenty of stupid things like someone who picked a hashtable for fast lookups, only to iterate through it in some other code path for some other use-case accessing the same data, completely murdering overall application performance due to selecting based on a single criterion. If you're using a language where you can create your own allocators, it can become even more interesting than you think - i.e. there are gains or losses to be had.

The moral is simple: "just try it." This comes up a lot in game programming for example and I've seen it in embedding programming have huge effects as well. I can't even begin to explain how much faster I've made things just by changing a data structure because of real-world conditions vs. lab conditions. Don't just assume that picking what the textbook says in the Big O table is what actually will dictate performance, positive or negative. There's a reason most good data structure books also contain explanation regarding behavior and performance, though even books don't and can't explain the real-world nuances when working with non-contrived data.

I am aware of the compilation into CLR bytecode and I've been a .NET user since the beginning, though I don't work with it much these days. The fact that it someone is using Clojure though doesn't mean it will translate always into the most efficient byte code. On the JVM, this is a problem sometimes forcing you to either write your Clojure differently to produce something more efficient, or requiring you to write a bit in Java and call it from Clojure (contrary to amateur practices w/Unity, people do need to share code that is not just abstracted by the engine such as some behavior). The same paradigm would be true of having to drop to C# and then call it from Clojure CLR. The problem here just becomes you get further from what is good about Clojure and instead write a version of Clojure that is closer to C# or something else.

Regarding speed, I think you misunderstood part of my comment. I simply mean you pick something decently fast when picking a scripting language with regard to the host language. Obviously Clojure is decently fast on the CLR in theory, but in practice may be another matter given the properties of a game which are far different than general use computing.

As for the speed with regard to Clojure, I've covered much of the general stuff in other comments, but I was in part writing to the point of using it independent of Arcadia in a scripting context. For this, I am contending there aren't many advantages to using Clojure over other approaches as a general scripting language for most engines, ex: Unreal, CryEngine, in-house, etc. Unity obviously benefits because of the CLR, but there are still issues with both Clojure CLR and using something that is yet another abstraction on top of Unity as others have pointed out. This makes life hard for newer or less-skilled game devs especially which it tends to sound like the original post authors are in general. More than anything, unless you have unlimited time, do you want to get a game done or do you want to help fix a new runtime?

Additionally, I am not comparing Lua to Clojure for the CLR specifically, rather stating that most of the time you add a scripting language to an engine in general it depends to be languages with certain properties. The reasons have little to do with Clojure as a language and more to do with scripting specific needs, which Unity itself doesn't really handle very well depending on how you define "scripting." That is any decent scripting engine must address some of the following below in a game, or it is not a scripting engine but rather just part of the main compilation target.

- Support hot code reloading, ex: from file system, user scripts, in game commands, etc.

- Compile/JIT reasonably fast if required

- Provide bi-directional calling semantics

- Easily integrate with the primary host language of the engine

- Decent experience for script authors

- Easy tool integrations, including with 3rd party middleware

- Minimize garbage if generating any, and make collection predictable if possible

- Not require the game code core to be recompiled in tandem (a primary motivator for using one from C/C++)

Clojure on the CLR certainly does some, but I'd argue not all well to be a huge benefit over C# in terms of productivity, performance, user experience, or user-facing scripting features. Of course there are some cool things due to Clojure's reader and macros, REPL, and so on, but the point is more the aggregate benefits. The reason people use Lua, Squirrel, AngelScript, and many others is not that they are the best, or fastest languages, but are generally good in all those points with regard to the primary language of the engine they are attached. Personally, I'd rather use other languages like Clojure, but there are tons of drawbacks that don't bite you until you are working on a real project, many that I listed elsewhere in this thread.

As I noted (and you repeated), I assume the code is alpha and there are still items to address. Nonetheless, most of these things are obvious and easy up-front, and many well-known in the Unity community, so I'm not 100% buying everything is a "later," just some of it. While in a normal game I wouldn't expect various optimizations or micro-optimizations, in an engine you definitely do. Even when working in C or C++, all the engines I've worked on we used our own allocators, minimized general allocations, added indexing tricks, branch prediction and inlining tricks, array packing, alignment tricks, and so on from the start, because that's the job of the engine and the end-user (game dev) cannot optimize away all of those things if the engine is the issue. Of course Unity doesn't do all these things well and things like Clojure layers are a lot further from the actual engine, but I'd argue that it means that everything needs to strive for equal parity or better performance when and if possible. Those are things you design around mostly from the start, the other optimizations that I think you are referring to happen later. I'm simply contending I see a mix of both.

Finally, following up again on the use of Clojure, as I've said in this thread, I'm a big proponent of Clojure. That aside, the CLR vs JVM implementations do not have parity because of lack of resources more than anything else. This automatically puts a lot more work in the pipeline and ultimately on the game developer, at least until things mature more. This isn't about using something like Arcadia at all, but rather using it right now given the original constraints and resources the article author mentions or implies.

I wrote a long comment, but was rate limited at the time.

I'll summarize instead with a simple list:

- It's better to map your input to some game actions than pass around the raw value

- You don't need to do anything creative like queue function pointers themselves, just the raw values.

- You can have an input "queue" but that is typically different from a queue that is going to act on those inputs.

- You need to deal with multiple inputs potentially, like someone pounding a button. Hopefully your input lib deals with this some already, but always be aware of validating the input and deciding which one matters most. To that end, priority queues are good sometimes. You also sometimes want to cancel inputs. If you want to read more about it all, I'd read some about "intention systems" as related to input.

- Once your input is mapped, you can operate it in multiple pieces of codes, systems, etc. that might be downstream in your game loop

- You can transform an action into yet other new state as you go along

- Queues are indeed great for doing multi-threading, just be sure you select the appropriate kinds of queues. Mostly this ends up being thread-safe queues that are lockless. But sometimes you want to just use queues for FIFO, and these queues are generally faster and/or more flexible if they use a non-thread safe approach.

- You can have many queues in your code as mailboxes to distinct systems. Again, think something that is agnostic to the world around it and just receives some state it needs, and has its own "inbox" of things it might need to process this frame. It certainly could be that you can't process them all that frame.

- Keep in mind with queues generally once you pop something off, that's it. So you need other ways of hanging on to state and things that aren't ready. To that end, simple arrays are your friend, especially if you can pull things out quickly by index in relation to some id (ex: entity id) and keep them tightly packed. Dynamic vectors are garbage and mostly not used in serious game engines unless the developer had a wtf moment or the use case specifically calls for it. That's a longer topic though.

- Attaching functions directly to input again is a pretty bad approach. If you need more than one function, you'll have to implement something more complex. If order of function calls matter, you again need yet more. The order should be well-defined in the game loop itself.

- Regarding order, it also relates to communication as well. Note that you don't need to always process every new input or piece of state every tick. Sometimes you actually don't want to do this and defer it to a later time. There's no 100% rule, but an easy thing that helps is to think if the player or game sim will suffer because you didn't process that state during that tick. For some physics related things, that can be bad, but for other things, it's more of a "as long as this happens very soon, it's alright."

- There's no 100% rules for any of this.

A keypress should never do much other than to update some game state to say that whatever key was pressed and it is mapped to whatever game concept in your input mapping. Nearly any code I've ever seen with an if/keypress, do this is an anti-pattern. Once you have your state, you just need to handle it somewhere later during a game tick, and keep in mind that might be even multiple places.

I would recommend thinking in loops, so it sounds you're already on the right track, you just need to expand that more to everything else. You generally want to do the same "thing" to many "things." Since you are newer to programming engines, I won't get too technical and I'll say that the gist is that the CPU likes this. The bi-product is if you take this approach, you'll tend to have better architecture as well as performance and solve some of the problems you listed. This is really the basis of an entity system approach if you want an example architecture.

Whether using entity systems or something else, my recommendation would be to separate out functionality and make them rather agnostic to each other. Again, this fits in with the loops paradigm. You still often need things to communicate as your problem suggests, but the way to do that is to often take some state, update it with new state, and pass it down the chain to see if anyone cares. That chain itself can be a loop, i.e. a bunch of systems in your game loop calling a method like update(tick, state, ... ). That said, you also need to be careful what and how much state gets passed around for performance reasons (ex: avoid excessive copying) and to prevent weird things from happening if you start introducing concurrent and/or parallel programming. To that end, I tend to keep things minimal as I can, or at least have some sort of way to hand things just what they need, not everything.

Queues also relate to all of this and can be your friend. A common communication approach is to use mailboxes for example and tends to be more straight forward to debug and optimize than using callbacks and event-based approaches (they tend to murder the cache and your stack traces). Queues also play nicer for being building blocks for concurrent programming.

Also keep in mind you may need to flip your thinking. Sometimes you don't actually need some action that happens during a tick to be processed the same tick. You can often or will often want to defer things to the next tick, or even several ticks after that. There's a Naughty Dog presentation on Last of Us port to PS4 that relates to this and how they flipped even the way frames are processed on an existing codebase to get better performance. The point is not for you to do what they did, rather to start thinking in terms of what the user will see as the end result. The user often doesn't care that you didn't finish processing some action last tick if it had no bearing on the gameplay.

I couldn't agree more. If anything, the pleasant experience argument tends to come up more in the context of discussing a secondary scripting language to be added on top of a fast, already working foundation. Even then, you mostly just end up using something reasonably fast like Lua or one of its cousins. I can't see Clojure really being a good fit for any existing engine I am familiar with as a main or scripting language (maybe ClojureScript if we stretch it for something using JS, which I disagree with), and I love Clojure.

FWIW, scanning the Arcadia source there's also some garbage being created that can be easily avoided. I didn't really trace to see what each thing was "doing" or if it even runs during a game, but it certainly felt eye opening to me to see lots of temporary garbage created in something that is supposed to be lower-level. I am not sure at this point what would get optimized away anyway (ex: foreach loops, extra tolist/arrays used to create garbage in the past), so here's to hoping the devs just didn't get around to optimizing that stuff away yet. Seems to me you would want to minimize garbage from Aracadia itself given Clojure is going to create yet more, not to mention cut stylistic corners for pure speed in this context.

Agreed, and I'll add a few things.

You can argue that lots of games can/do modify things while running. Entity systems allow components to be added at runtime. This not only allows for different code to be executed, but also I've seen code that reads components and hot loads additional code for systems supporting those components if necessary, which can be done a number of ways, and raw from the source even.

Smalltalk also supported this and isn't really functional. In addition to the Smalltalk environment itself pretty much being a live environment, I saw things like serializable continuations in Smalltalk. You could do fun things like a Gemstone Smalltalk app I saw that would take the image state and allow hot modifications via a bug tracker.

There's tons of other systems for doing this. And it's also a reason why many games write big chunks of the type code you would want to hot load in scripting languages like Lua.

There are tons more examples and has no relationship to functional programming other than to say that it was a common thing in the Lisp world many moons before most of these other things existed.

Clojure suffers some because of the reality of today's game dev ecosystem. Just looking at the source of some past attempts to wrap things like LWJGL or OpenGL (hint, none of these tend to look like Clojure anymore) should be enough to answer some of the questions why this is. There are many others reasons as well that only come to bite some people until they are in the middle of a project. Beyond those reasons, even just dealing the JVM or the CLR for a game present significant challenges to a developer.

Unity itself has had to work around challenges too, and the abstractions and development comforts it provides are not free. Even experienced Unity developers who use C# still have to be mindful of how they write their C# (i.e. don't write the prettiest code even if you can because it will punish you). Of course this is true of any language and for other platforms like in mobile game dev. The point is that it's especially limiting to have additional layers of abstraction largely beyond your control. Eventually, it reaches a critical mass where you're writing this weird meta-language to do what you want because writing a game with the ecosystem around you forces that instead of letting you write idiomatic code in the language you picked.

The further you get away from the metal for a game, no matter how simple, the more you will face problems. It's nice to use languages like Clojure, Python, Ruby, JavaScript and so on for games, but for serious work they often get in your way. For instance a common problem the average developer encounters is the game loop vs. frame rate - how do I get enough done during a tick to not grind the game to halt? Garbage collection, de/allocations, and so much more become your enemy and you start to feel like you're fighting some kind of magical force trying to slow down your game or make it less predictable, rather than being productive or even optimizing it in sane ways. And yes, predictability is vital to writing a good game, because the last thing a player wants is your game to do stupid things at inopportune moments like in the middle of a jump, never mind other concerns like debugging, multi-player, or platform requirements.

Of course there are workarounds for many problems you may face, but as game complexity grows, things tend to scale out of control for most people. Many of these problems cut so much in to the time or make you have other sacrifices that you start to feel like you're largely missing the benefits of working in these alternative abstractions. At some point you just end up breaking all the rules of your language/tools/libraries to get the game to the level you want. Worse, you're working on many problems that are quite far from actually finishing your game. Obviously for simple projects, much of what I've mentioned previously is not a problem, just to again make that clear.

Getting back to Clojure, I feel it really suffers from the aforementioned issues for non-toy games. This isn't an indictment of Clojure, just about picking the right tools. Immutability, atoms, refs, agents, CSP, sequences, transducers, recursion, and so much more seem like they would allow making a game quicker, easier, and with less headaches. What ends up happening to most people I've seen who try to use these kinds of tools, whether it is Clojure, Lisp, Haskell, Elixir, or anything else is that at a certain threshold of requirements, is what I mentioned earlier - it all falls apart. At this point, you spend all your time removing all the goodness the language and tools provide. You start writing your own libraries, often down to numbers, matrices, etc. because you have no other choice if you want things to run in a sane, predictable way and to integrate with anything like OpenGL, input libraries, hardware, SDL, and so on. You throw out immutability in huge parts of your game, and you realize that refs, agents, channels, sequences, and more are just making life worse, not better. Pretty soon the entire language is stripped down into something almost unrecognizable, left with only a few core nice things. You then descend into the next layer of hell and start porting things into Java and calling them from there. Even in Java this can happen to a large degree. Add in more unpredictable stuff and abstractions like Unity, Unity plug-ins/add-ons, multi-platform requirements, talking to other libraries you want to use, and so on. The author of the article mentioned simplicity as a selling point, but for non-trivial contexts, you will almost certainly throw simplicity out the door. It starts small and snowballs as I described.

For someone building a text adventure or other simple game, you probably don't even need Unity anyway. If you're building a smaller indie game or want to get something done quickly, just use Unity and C#, and you'll get it done quicker and benefit from the ecosystem better. If you can't/won't learn C#, you shouldn't be programming or making games. I know that sounds cruel, but at some point we all need to acknowledge our skills. A game developer should be able to learn any language and be productive in it quickly. The average game dev may not touch the entire game, but more sophisticated games often use several languages, especially if you count things like shaders and scripting engines as being distinct.

If you're just learning/new to game dev and/or really want to learn Unity, just use it as intended, otherwise you're adding more layers of abstraction and complications that make it actually harder to learn anything, and worse to get things done. It may often seem like you figured something out and using your favorite tool will get things done quicker, but most of the time you'll hit the ugly thresholds I described when you try to combine it with something more sophisticated like Unity. Use things as they are intended. If you want to make a game in Clojure, great, just keep it simple, write your own minimal engine optimized for Clojure or hope someone makes one someday, or go the ClojureScript route to again make something simple.

In summary, Clojure is indeed an awesome language and you can write a game with it, just I wonder what is to gain using it for Unity. In the general sense of things, I wouldn't recommend layering too many abstractions when building games. If you feel otherwise, I'll refer you to the graveyard of projects that have tried to take X and make it work with Y - it is a huge, sad place. That said, I'd love to write a non-trivial game in Clojure or another functional language one day, somehow.

As a semi-retired professional gamedev, I don't get this. Let me qualify what follows by saying that I currently program quite a bit in Clojure and ClojureScript on my personal/side project, and I love both languages. Additionally, Lisp was one of my first languages several decades ago. Sorry for the length.

Game programming like most programming largely requires choosing the right tools for the job. While I agree it is important to factor in the skills that you team has, the choice of Clojure + Unity + this write-up seems like retro-fitting for the wrong reasons. Regarding Clojure, I can appreciate what Arcadia is trying to do, but it seems like an odd choice for an unskilled game dev for anything but messing around/intellectual fun at this point. This is both because the software is quite honest and labels itself alpha, and because Clojure and Unity is not necessarily the best combination IMO for many reasons. The primary reason I'd contend this is because it is a huge pile of abstractions of abstractions with not enough payoff. Even just browsing Arcadia's source, I can also already see a few bits of code that aren't optimized as well as they could/should be for something like this. I can only hope the article author is not serious about pursuing a commercial quality game of significant complexity and power with this approach (i.e. not necessarily AAA level, but upper-tier indie).

As for functional programming in games, it is a big subject and an interesting one, not to mention a worthy pursuit. Indeed, huge amounts of the trends in game programming for the past two decades at least have been moving to functional paradigms. Entity systems are a great example of this. One way to think about Entity Systems in particular in relation to a functional language like Clojure is that it essentially allows you to think about your game state as a reduction of states (entities with attached components and systems operating on them) over time. Furthermore, as multi-core programming becomes more important, some of the constructs that make concurrent programming, or even parallel programming easier are commonly found in highly functional languages. Clojure is no exception and has many things are amazing to this end, but not all are necessarily useful in the context of a non-trivial game. There are many more things I could touch on, but suffice to say that programmers are bending over backwards to make C and C++ in particular, followed by C#, Java, Swift, and other languages behave like functional language. Ironically, I think part of the problem with most functional languages is that people are pulled from the other end and forced to make these languages behave like C++ to get acceptable performance and other traits.

Despite many OO/imperative languages used in game dev moving towards more functional trends, the promises, power, and tools that most of these languages provide are vital. There are too many reasons to list (some good, some bad) why C and C++ are still dominant in this world for serious game dev work, but a few include existing tooling, full control of memory, ability to work around language induced performance roadblocks, possibility of portability, optimization per platform if required, bountiful workforce, and low-level integration points with hardware, SDKs, and tools. That's not to say other languages don't have these things, but rather they have some critical flaws in some of these areas that make life harder than it needs to be at the cost of actually getting a game completed, which in the end is one of the most important things.

I know it's frowned upon, but I'm splitting this rather large comment into two parts.

The pay certainly is not good for a lot of these jobs, but nor is it good for a lot of white collar jobs. The difference is often the long-term ceiling which is not a guarantee. For instance I know people who work for non-profits or more niche professions that make very little, but they need post-graduate degrees. Even professors in certain fields don't make a lot vs. many types of jobs, so education is not a guarantee to riches, nor should it be. Likewise, going into a trade should not be a guarantee, but if money is your goal, there are trades that can deliver just as much if not more depending on what jobs are compared.

The AI comparison is a really good one, I like that. I'm in part a trained statistician, at least from the University level. I have gotten into machine learning more recently given my stats background and my previous game development background. This was discussed by other people in other articles, but I sometimes shake my head when the word "intelligence" is used to explain what amounts to applying some basic (or even advanced) stats to an input to produce some hopefully decent output.

Having seen the number of ways that supposedly awesome AIs can easily be confused, I am far from convinced. I don't think we are in danger of being replaced by robots when we can't even secure a web server, email, or now it seems, toasters. If the robots do take over, I know there will be so much sloppy code that we can either hack/backdoor them and shut them down after many long nights in a bunker typing away, or we can wait for them to just crash/segfault/whatever because I trust the creators to write bug-free AI as much as I trust a Lion to go vegetarian. On the flip-side, that all should also make it easy for someone to trigger them all to become self-aware or start the next great cylon purge. I'm ready and don't care about my plans next week anyway.

To clear up some common misconceptions around the Old Testament in part, but not in whole....

Firstly, Judaism is not the Old Testament at all. Christian and Jewish philosophy, and for that matter Muslim philosophy are all very different. As such, there are different interpretations of the same input data one could say. The problem is not even the input data is the same. (btw: the term Judaeo-Christian suggesting commonalities IMO is wrong and might as well not exist because it's lumping mostly different things together).

The Old Testament is a Christian edited version of various selections of Torah, namely 5 books, but even the divisions differ between Judaism, Catholicism, and Protestants. Beyond potentially reorganizing and excluding content, another disconnect is that the Torah according to most Jewish traditions is both a written and oral tradition. Most Jews who are knowledgable about theology would agree that Judaism is not explained, nor summed up, or even represented by the Old Testament, let alone the Torah itself. Further, oral traditions as well as cultural and religious duality are so important, that you can't actually read things like the Torah without knowing them. Philosophically, there's a pretty large difference between Judaism and Christianity on so many levels related to spiritual significance and directions of text.

Even inside Judaism there are huge divisions. Note that there are people who devote their lives to studying Torah and while I have a few interesting things to say about that (depending on the context, not nice things), I acknowledge it as intellectually interesting in that you have many people with different interpretations, including seeing the text itself as liquid or even adaptable (ex: to modern times). This kind of studying, questioning, and analysis is encouraged in Judaism. While it happens as well in modern Christianity, the views and treatment of text here are very different suffice to say without going into great detail.

Another large issue is translation. Ancient Hebrew is just that, ancient and not completely understood with 100% accuracy even among native modern Hebrew speakers and scholars. Even Aramaic and other languages of related texts have changed. Pronoun translation alone from a gendered language like Hebrew to English is bad enough for example, but factor in that often the path could be as muddled as Ancient Hebrew -> Greek -> Latin -> <Your Language, ex: English>. There are of course more modern direct attempts, but quite often translations are not from the source, but rather translations of translations. Between translation, time, and oral traditions, there's a lot of signal decay.

As for other things about Judaism related to the Old Testament, the mention of it being 2.0 splits a lot here. Judaism has all kinds of commentary, supplementary texts, additional scrolls, and other books that are various degrees of "holy" or even "canon." Pretty much none of these exist in Christianity and only a few play any role or influence on the New Testament (aside: New Testament also morphed over time and had translation issues and so on). To make matters more interesting, some interpretations become part of oral law and even influence the readings themselves. Indeed, words are often changed or substituted by some people, while others would rather die than do anything but what they perceive as read things in their original form. If you purchase a book even that contains the Torah and what the Old Testament is based on, each version will have wildly different commentary, and sometimes even omissions.

I could go on, but since it's HN, let me just summarize a few things about the Old Testament in computer terms to get back to the point:

- Christianity vs. Judaism is not like a Git branch, but rather taking a few conceptual things and writing different implementations. There is little if any shared code, but maybe you could think of it as people who worked on an old project together and then had a split and built new things.

- New Testament is not v 2.0 but more of a re-implementation with what could argue is some code refactoring and slapping on a new interface (some things in common at a high level, but not at a lower level). Some (a lot) features are missing in the re-implementation.

- New Testament is heavy on marketing Jesus and his disciples, while ironically this would be an offense punishable by death by Old Testament and the Torah (you are not allowed to call yourself or be called the Messiah, the conditions for the Messiah are specifically laid out and according to these texts have not happened). New Testament would fail build tests and constraint checks.

- Christianity and Judaism have different licenses, plug-ins, benevolent overlords, and so on...

- The human compiler pretty much is broken because people aren't following the source of either most of the time, probably thankfully. I know the goats, sheep, etc. are probably happy aboutr that, let alone humans.

Well I am sure someone is going to hate me now or kill me for sacrilege. For the record, I the only thing I believe in is the electric guitar. I just grew up around religious people and spent some time studying it to understand what it was I didn't like.

Wow, way to take the high road on a tech news site. I think you're trying to imply Jews are barbarians given Israel vs. Palestinians. Note that not all Israelis are Jewish, many are Muslims, Christians, Druze, Atheists, and so on. Secondly, I doubt you have lived, worked, or spent significant time in Israel to make such judgements. More than likely you probably do not even known Israelis on a personal level.

Secondly, I'll comment separately, but Judaism != Old testament. This is incorrect for so many reasons, and among them the same reasons that even Koran != all Islam.

Only anecdotal, but I have observed as much over the past few decades. It's hard to be certain why as there are so many factors at play, but I suspect that a part of it may be due to how we perceive certain jobs. There's a certain stigma with most traditionally "trade" jobs like electricians and plumbers. Many of these type jobs actually make more, especially if you own your own business or have a fee structure that is tied to your actual work.

I've known journalists, biologists, programmers, middle managers, government employees, and many white collar jobs to scoff at or look down on other jobs. In some conversations I have reminded these same people that the filthy guy in line they are making fun of might be pulling more than twice their salary, and often may still be university educated or have needed to go through some rigorous training. I am not sure what makes some people think they are above certain jobs or better than other people.

The Obama administration sentiment that everyone needs to go to college has done no favors helping certain classes of jobs. White collar jobs or being a celebrity, athlete, or famous in some other way is seen as success, while anything else regardless of quality of life, salary, long-term prospects, independence, and so on is seen as failure, second-class, or even undesirable. I respect anyone who works hard, and especially those that do things that I can't do. I need these people to survive or at least have a better quality first-world life, therefore I always make sure to treat them well. It helps when you come from a culture that considers it rude to not at least offer to feed your plumber or make a coffee.

I agree with this and your point brings up some other implications. I especially agree that there are a lot of companies with very questionable business models. Ironically, Goldman is helping many of these companies make money and be perceived in the market as "valuable" regardless of common sense, evidence, or more thorough financial models.

Most financial and economic theory seems to indicate that even moving all the Goldman employees to some other company will just result in other companies taking its place.

If we put in place rules to prevent another Goldman forming, another company will find a loophole at some point or even be far worse. I'm certainly not justifying their existence, actions, or anything else. Rather, I am pointing out that the market will just swallow brain power and redistribute it some other way that isn't ostensibly any different or "better."

If there is a problem that needs to be solved, it is not redistributing tech people from Goldman somewhere else, but rather fixing whatever societal, financial, economic, legal, and political conditions among other factors created the perceived evil entity swallowing the universe. If we chastise the tech people for working there or the situation itself, we should also instruct all the other people there to stop wasting their brains and apply them somewhere else.

I agree it's disgusting to think of people working for a company like Goldman, but not sure I agree with the rest. Also some people are just trying to put food on the table, and they could compromise, but they don't and it's not proper for me to judge them even though I am.

Firstly, Goldman can't correctly value a company - no one so far can. The best Goldman can do is use their position of power to effectively tell the market what the value is, but they can do that practically by creating magic numbers from the sky. Indeed, go and do an old school valuation on many companies and you'll get vastly different results. I have a degree in Finance along with various computer and math degrees. One of the big takeaways is that financial valuations are entirely subjective just like stats studies are, so it is the justification, evidence, techniques, and so on that matter. Your job is to convince someone, and Goldman exploits its position as a large player by doing just that.

If Goldman could actually set prices or guess them exactly to the millisecond, they would never have any losing bets and they could manipulate the market 100% to their will. One could say that in this sense, you are right as they do indeed manipulate things to some degree. They are so large that their actions influence the rest of the market and their failures and successes change the market, ironically screwing themselves up quite often.

Many principles though say that valuing a company is hard to impossible (just ask Black and Scholes). Random Walk and some of the associated studies like throwing darts at the financial pages (darts beat top Wall Street investors more than 50% of the time usually) demonstrates that there are elements we do not understand in terms of valuations and predictions, and that moreover people don't act rationally depending on your point of view. Given this reality, trying to build better models is a very interesting and wide open computing topic.

Another important factor is that valuations and other financial market indicators change over time. That means your models both need to account for time and be extremely fast at reacting to changes. In computing, this touches a lot of interesting topics from performance to network architecture to low-level protocols to CPU caching to distributed computing to machine learning and on and on. Outside academia, there are few companies that are focused and yet touch so many different computing topics and actually apply them to their business in production.

Personally, I would never work for a company like Goldman. My moralistic view point has cost me at times financially but I am OK with that given my internal priorities. For some people, it's all about money. Goldman might be evil to many like me, but many popular tech companies aren't much better if you take off the geek glasses.

It is not a popular sentiment around here, but I wouldn't necessarily put people at a lot of popular SV companies like Facebook, LinkedIn, Google, Uber, and so on above the Goldman people or on some higher pedestal/plane of existence. Surely some are doing interesting or altruistic things even, but most are just grinding out the day trying to keep the business afloat or make it rich.

Many of these companies on some level are still tightly tied to people like Goldman, have similar motivations, and arguably a lot of the work they are doing is intellectually less stimulating, valuable, academic, and challenging. Moreover, a lot of the "work" and "tech" that these SV companies think they are inventing has already been around the financial sector (or even tech sector itself, not to mention academia) a long time and exists in better, faster, more powerful forms with the caveats usually of being expensive or exclusive to a single company. Not many people realize what tech people at a company like Goldman are doing, and the people that work there often are bound by legal agreements not to share. That makes companies like Goldman very adversarial, anti-open source, and anti-employee. To that end, I think they are pretty disgusting, but I've experienced nearly as bad working at game companies and tech giants.

Very few companies are doing anything worthwhile with brainpower. Even academia where brainpower should be tantamount has often become more about stats like amount of papers published, grant money, or other financial results rather the brainpower. As a whole, computer science is a field that constantly reinvents wheels poorly, wastes tremendous amounts of brainpower, and is far from altruistic or progressive at times. We could say that about almost any field, so take it for what it is and remember that most of us (a select few make a smaller scale difference) aren't saving the world, even when that literally might be the company "mission." Most of us are wasting our brainpower most of the time and it's up to us to do better; working for a company is almost never the answer for maximizing your brain.

I'd also add compression time and decompression time to that list. Zip wasn't actually the best compression algorithm I remember seeing in its original days, it just was the best compromise of speed of compression, extraction, and compressed size. The TOC feature was a killer because even the fastest formats decompressed slow, not to mention space was more of a premium.

An additional feature that sometimes is important as well is making itself stream friendly so you can read the metadata without the entire file (or easily before streaming) as it is streamed over some context or using some technology.

The landscape is much different now and so some of features you'd design into a new format didn't exist then, while others that were concerns are less concerns now.

As parent hints, the compatibility alone makes zip still very desirable and actual compression is good enough given size of disks today for most people. Obviously there are exceptions.

I have indeed encountered many broken zip files. In truth it doesn't happen to me much anymore, bit it sure did in the earlier days of pkunzip/pkzip, and in general during the BBS days. Sometimes this was just due to all sorts of weird things going on with transmitting files over networks and modems. Mostly you could prevent these things with CRC checks or any other verification method, but that doesn't mean implementations checked this or that the check itself didn't just fail.

A few things that I've hit in the wild that screwed up zips:

* Viruses/Worms that were rather primitive and start adding weird bytes in all kinds of places in all kinds of files

* Incomplete transmission over a modem. Depending on the protocol, you might even have most of the data, so you could read part of zip header, but not the actual archive or vice-versa. Normally, you knew right away the file was incomplete with a CRC check so not the worst problem, but the check itself was slow on old hardware.

* Weird things that added metadata and screwed with byte order, ends of file marks, etc. For instance there were some early attempts in the BBS scene to add metadata formats similar to what ID3 is to mp3s. Sometimes the writers would ruin the original file. I hit a few cases of strange attempts at steganography with software pirates trying to be "3l337" or whatever.

* Tools people tried to use to fix broken zips, that didn't quite fix them how they thought.

* Not really the fault of zip, but I've seen people rename a zip's extension to another archive format, causing the unarchiver to assume the format based on the extension. Never trust an extension if you can help it. (ex: rename a zip to rar, arc, lzz/lha, tar whatever)

* Floppy and hard disk repair programs. Sometimes these things would end up corrupting the bytes of files instead of fixing them when they tried to be more clever than moving things around. Sometimes moving things around also would result in things being ordered wrong for whatever reason in these programs. Some of the DOS Norton/Fastback/etc. ilk were especially frequent offenders.

Most specs I encounter have seriously flawed implementations in the wild. This is regardless of the quality of the spec.

Here's a short-list off the top of my head of tech with specs and horrific offenders in the wild, including many popular implementors.

* Telnet

* VT-XXX, i.e. VT-100 or any term spec for that matter

* MIDI

* MKV, AVI, etc.

* GIF, JPEG

* A huge amount of tech in any browser

* SAUCE

* ANSI escape parsers/writers/sequences

* LZH/LHA

* Open Document Format

* XMP

* MAPI

* POP, IMAP

As I think of this, I realize I'll be typing all night. In some cases, the implementors all adopt things from the most popular implementors even if they are historical. In other cases, things just don't work or break in awesome ways. Zip bombs were mentioned, but there are plenty of awesome things.

I'll finish by saying I wonder if many authors of anything related to terminals even ever bother to read specs or care. Over the years I've started to wonder how some of these things even work in the wild, but the answer is most people don't notice what actually isn't working, while anyone who needs to care has to draw from what become unofficial new specs.

You're making me seem old I guess, but I've encountered this a bunch of times. This was often done as a way to add metadata to things that otherwise didn't have metadata or needed additional info. Typical use cases include music players, trackers, drawing programs, media viewers, and anything that scanned files for metadata (ex: BBS).

The SAUCE spec adds trailing bytes after the EOF mark for instance. I wrote a few SAUCE reader/writers embedded in various codebases (and will release a stand-alone good one some day soon).

Generally, a trick was that if you wanted some sort of metadata on a file, you could add it before the contents of the file (as in MKV allows ASCII at the start) or after the EOF marker. The latter made sure you didn't interfere with a lot of existing programs on various operating systems of the time, including but not limited to DOS, Windows, and CP/M. This trick still works for a lot of OSs or in other forms today, we just don't rely on markers for EOF as much so there are other methods, and people have started to bake metadata into specs.

As for the BBS scene, although SAUCE was used, for zips DIZ tended to be used more and was just packed into the archive and could be found using the internal TOC, or otherwise just uploaded separately.

Former game engine dev and for a long while, proprietary engine programmer here....

Crytek focused too much on pretty graphics and not enough on anything else that is important to the larger experience of developing a game. Shiny graphics and cool fringe features are nice, but an engine is supposed to help you get a game done faster and without reinventing wheels.

Building an engine to be the wheels for other people is hard. Most of the time you build an engine, you make it work well for a specific game or subset of games you make. It seems like everything in an engine should be and can be generic, but that's not really true. Once you try to make things super generic, you start to make tons of sacrifices either for the sake of makings things easier or at least repeatable. The issue is that as you move farther towards abstractions away from the original problems, you complicate everything and the end result is actually often less productivity (more steps, more hoops, etc), less performance, or even having to drop features altogether. Specialization is often looked at as something terrible in programming by some people (my code should be reusable!), but it really can make a huge difference in performance and features-specific contexts like games.

There's also a difference as well between building an engine for yourself and selling one. If anything, I'd argue that Crytek may have done a pretty OK job for the former, but egregiously failed doing the latter.

If you're selling an expensive game engine, you need to do at least some of the following:

* Provide reasonably understandable source

* Offer good support, especially in English

* Stable, highly adaptable tooling

* Wide support of popular formats

* Clean, adaptable, fast asset pipeline

* Community support and interaction

* Cross-platform tooling/compilation options

* Integration points for custom code, middleware, etc.

* Ability to patch/fork the engine itself within reason when needed

* Avoid huge amounts of allocations

* Use fast or safe allocators or provide the ability to do so

* Provide useful feedback during debugging, logging, tracing

* Plug-in system/replaceable parts

* Offer a scripting language and/or support for popular ones

* Tools to prototype quickly if necessary

* Create one or more games that is understandable to demonstrate features, but is still comprehensible

* Support modern asset formats

* Provide decent, predictable, hardware support

* Ensure a stable experience

I'll let the reader interpret which of those CryEngine does wrong.

Some versions of CryEngine actually do some of these things well, but the problem is that relative to other options like Unreal, everyone I know has found it hard to justify buying or using CryEngine going forward. CryEngine in terms of raw features often looks attractive and awesome. Most people I've known including myself have had issues arise once you try to build something real with it. Quite often things just don't work how they should or are incredibly quirky. There's always tons of things promised that will be "fixed" or "redone" and never get done in a reasonable amount of time.

Just look at the Star Citizen situation. Hiring former CryEngine devs really helped the project they say, and that's not just because those people are awesome developers, it is because they understand the crazy quirks that most people who didn't work with that code every day don't and in some cases never will.

The game engine market isn't an easy one, but frankly they brought this on themselves.

I think the musician Jarvis Cocker explained this best in his song, "Running the World." Particularly the lines, "In theory, I respect your right to exist, I will kill you if you move in next to me."

People love to free speech, except when it involves supporting all the things they don't like. For the record, I don't love any extremes and certainly not the alt-right, but censorship is worse. It is never justified, under any circumstances.

If you don't like something, don't listen. People are and can be protected plenty of other ways, for example by laws against murder. The law anywhere isn't perfect, but censorship is even less so.

https://www.youtube.com/watch?v=xRGGbyZzuTg

What you are saying is mostly right, perhaps somewhat of an oversimplification. Interestingly, these days I work with quite a bit of Clojure.

Most of what I've been describing came about because people were in OOP hell, and decided to move towards composition vs. inheritance. As multi-core machines, GPUs, and more sophisticated games arose as well, there was more of a need to start thinking more in terms of what the CPU wants and pipelining things for the CPU, GPU, and such (think PS3 for instance). To some degree, this thinking was always there and circling back to 80s game programming, but a lot of people got off-track. For example, memory allocation was always a thing to the point where no sane person every used anything in C++ from std because of allocations and bad memory layouts. Structure packing has always been a thing of course too. Another was thinking about v-tables and how they just get in the way, especially when working on more limited platforms like the NES, 286, etc. in those days.

Games are mostly about enumerating a lot of stuff, very fast, per some measure of time, and then repeating. If this sounds a bit like a map or a reduce in functional terms, it should because it is. You have loops inside loops, and you try to operate on similar blocks of data, rather than jumping around all over the code or iterating mixed collections (or hybrids using co/contravariance). It turns out computers, especially x86, really like it when you enumerate things in a line. There's a fun presentation somewhere where Stroustrup himself shows how an array can be faster in many situations than a linked list when the linked list fits the exact description of the correct data structure.

Anyway, you can rig a lot of this into Java as you say, but fighting the GC is still considerably hard. Clojure itself is even worse because it generates garbage like nuts and you also end up doing things like transient! or using volatile! to work around other speed issues. I'm not sure about Kotlin would be, but as a JVM language dealing with JVMisms, I suspect largely the same as Java. Other languages that are more predictable or not GC'd fair much better for games, they are just not used as much because of the people in the industry, technical debt, tool chains, know-how, risk, etc.

Where I'd disagree with what you said is that those constructs are all you need. For certain parts of your game you still need quite sophisticated stuff. Threading primitives and complex pieces of code help that are a pain to write with just what you described. But these things are generally higher level. For example, scripting engines are a good thing to have and often not written in the same language (ex: lua). Editors are another example. They aren't part of your game directly necessarily, but these days they increasingly are on some level, at least some form of them. Either way, you benefit from building toolchains in a variety of languages to help a game. It's not just a single code-base and ultimately you only have so much time, so you take shortcuts like throwing in Lua to make life easier.

Regarding SDL2, it's probably great for what you are doing. It's more low-level, but if you want to build things from the ground up, it's a fine and well-tested choice. There are other alternatives, but mostly in the form of full game engines. Certainly you could do this with something like Unity and it'd be good enough. If you're going 2D, the one piece of advice I'd give you is I'd say probably these days you can get just as much done by making things 3D and fixing the camera most of the time, or using a hybrid. 2D is more difficult than people think and often sucks in certain engines (ex: Unity :) ). Anyway, it's just a matter of some head banging and time. If you go SDL, you'll learn a lot, just remember, build games, not game engines.

Regarding entity systems, it's best not to implement as slots in a struct or something like that. Of course there are lots of ways to do it and varying performance. Interestingly, taking a data-first approach and iterating over the same kind of components within a system tends to be good for performance.

An entity is an int, long, or something else depending on your needs. It should be light-weight and not cause garbage to be allocated, thus why a long is good. A UUID/GUID is bad because you cannot use the entity id to cheat into pool rosters or other uses, not to mention it is hungrier for memory and slower to generate. Entity IDs should be allocated, often atomically and in one place and from a single thread.

A component is the data that you mention and should be a class or map/dict depending on the language. It is just a collection of fields, usually flat. Allocation should be done from a pool. It should be attached to an entity and detached as necessary. This is like a foreign key relationship per component to the entity ID if you want to think of it in DB terms.

A system operates on collections of entities, usually per frame/tick. Generally it is looking for one or more types of components per entity and isolating the logic there.

You can go outside the entity system for performance reasons like rendering if you must. Purity is good, but a lot of people go crazy and wreck performance. Be smart. The other hard thing is communicating between entities and between components. There shouldn't be any hierarchies or hard-coded things like that. Usually what is best is a queue or mailbox approach that the system can read from and deal accordingly, and then perhaps something more sophisticated beyond that as required per type of game.

My advice for getting games done in general if that's what you want to do is to just build a game and not an engine. From that regard, do dirty things like using pre-built engines such as Unreal or Unity. Better yet, just make a simple game. You will learn less, but be far more productive as a single person. If however you just want to muck around and learn things, then go wild. One thing we used to do to test architectures and game concepts that might be of interest to save time and explore me is we would build big chunks of a game with wire frames, stick figure-like things, whatever. If it's fun playing as a cube, it will be fun playing with a fancy sprite or 3d model.

Thanks, if you do nothing else, please (links to follow):

1. Read/watch the naughty dog presentation on Last of Us port where they talk about their time step. I have to say if there's one company who puts out stuff like videos and articles/books you should read, it is Naughty Dog. They know what they are doing optimization-wise, I just happen to find most of their games boring and lifeless from a design point of view, but I can still respect them greatly.

2. Read the GDC presentation about entity systems and the "roster" approach linked below.

3. Read probably everything on the Bitsquid site. Some of it is dated or even not the best, but I pointed some people here before and it really opened up their minds to what I was talking about when they were looking at my code. Changed their view from "black arts" to "wow, that really is simple and makes sense."

4. Read up on Entity Systems. But do not read things written by bloggers, authors of "frameworks," and various Java stuff out there (ex: Artemis). I hate to be so negative, but what I usually found out there is mostly garbage, wrong, and the authors disappear when challenged or asked about the hard, non-obvious parts. If you want good info, read code that's out there and figure out what seems good/bad, and listen to some of the people in the industry like at GDC talking about it. I tried to find one more recent talks, but so far only came across the Amazon one which seems pretty weak (though still interesting) and basic with regard to low-level detail.

Games knowledge is a tough thing because everyone is professor know-it-all and usually just hobbyists and throw away mobile game devs. Most people working on games who are competent are too busy or too sick of it to write about it. I hate to sound like one of those people. And that's why I don't blog about it or normally comment much. The only reason I'm mentioning it now is to pass time while watching the election. My games career was making me go insane and it was either family or video game development, so now I have more time in life to rant here. So, yeah.

http://www.gdcvault.com/play/1022186/Parallelizing-the-Naugh...

http://www.slideshare.net/TerranceCohen/terrance-cohen-dynam...

http://bitsquid.blogspot.com/2011/09/managing-decoupling-par...

Have worked on many games and game engines for several decades. Here's some quick advice that applies mostly regardless of the JVM.

1. Allocations of any kind are often the most expensive operation per frame, or at least some of the ones that are fixable (you can't just not have physics in some games). As such, avoid allocations per frame.

2. Notice transition screens, loading screens before levels, brief artistic animated overlays (things like ROUND 1 - READY!). These just aren't for fun or to hide just disk IO, they can actually be used as a time to do controlled allocations. Let's say you're doing a Metroidvania for instance - perfect time when changing areas to load new stuff.

3. Use fixed-sized data structures like arrays. Vectors, Lists, any kind of dynamically resizing data structures are generally your enemy. Sure you can use them in non-essential parts, but it's an easy thing most of the time. These help you control the amount of allocations, and moreover ensure that your game runs predictably, especially when pushed hard. You shouldn't be letting your game get into insane states that would allow allocating 5 billion bullet objects even if your vector and a machine with 64 gigs of rams can handle it. Obviously some exceptions, but there's sound reasoning.

4. Operate on fixed-sized data structures in sequence. It's harder on the JVM, but in most platforms you can start trying to use this to help the runtime optimize for you, or better yet, get things to fit nicely into cache lines. Things like "events" and "callbacks" may seem helpful, but they are actually your enemy, even on the JVM. You are better off using a queue or mailbox approach if you need more performance and less GC surprises.

5. Pool. But don't just pool and allocate, make sure you are pooling efficiently. On some platforms, this means tightly packed structures with no holes. One cheat you can do is actually use an ID system and work with that to index into an array-based pool by getting via index instead of a search. Using dicts/maps for this is the wrong approach for so many reasons. There's an older GDC presentation about this somewhere I can't remember, but it talks about this in relation to entity systems and uses the term prius a lot. Free list might be something else on your google for this.

6. Pack your textures when you can. Do things like use a sprite sheet. It's not just the GC pauses that are the enemy on the JVM, but also hitting the file system. This will help reduce it at the cost of potentially more memory for a single texture.

7. Don't go wild with classes. Try a data-based approach. This cuts down on GC and is the basis for some flexible architectures like entity systems. Of course you may still need classes as containers if you are using Java, but you'll end up with less ugliness and perhaps less classes and instantiations of those classes this way.

8. Check out your JVM flags and tweak things more if you must. Check out https://docs.oracle.com/cd/E40972_01/doc.70/e40973/cnf_jvmgc... for an example. Not the best guide, but my first google hit to just demonstrate. On a semi-related note, don't do stupid stuff like reflection or dynamically load in things that are just going to make the JVM angry. Simple is better than clever here.

9. Use a good time step for your game loop where you separate your sim time. It doesn't necessarily fix GC pauses, but it does ensure that if things are running slow you can respond and interpolate.

10. Related to the last point, try to do more per frame and don't assume all work is the classic game loop of render per tick. Naughty Dog did an interesting presentation on The Last of Us port to PS4 you can read about where they talk about optimizing the way they process frames. If your game permits similar approaches, it gives you a chance to squeeze out some performance this way.

I have many more tips, but this is just off the top of my head. Some of these might seem like micro-optimizations, but they actually end up shaping your game's architecture so it is definitely important to think about it early if you are serious about not putting yourself in a performance corner.

You can certainly develop games on the JVM. I personally don't recommend it unless it's for pleasure. C or C++ is still the way to go. I am sure you can have fun with Rust too doing it, but there's just so much ecosystem and people talent tied up in the former choices that it effectively makes anything else far behind. That said, you can write a good game in any language, and if you are just sane about your game it will be fine. Even with GC pauses, for some type games, ex: strategy that isn't real-time, your users would barely notice and it wouldn't effect gameplay. In short, be pragmatic no matter the language, stick to some of the above, and enjoy yourself and just get it built. Language is the least of your worries in the end, but still worth paying attention to on some level.

I agree. Lots of startups are created with this goal in mind. I have a tough time believing a lot of the popular ones are because of some inner calling, vision, or passion, despite the CEO press tidbits and about pages.

My view is you should always start with a problem that is significant to you, and solve it. I know that making a living is important, but if the problem applies to more than just you and you can charge actual money for it, I feel it is worth trying. The next question is if you can afford to take on such a risk, as it is a great one. This is substantially more risk than thinking of something to think of something because you want to run a company and/or make money. In the end, if you solve your own problem, you at least have that if you lose completely or go nowhere with it.

This of course is a rosy view and things like debt and moral responsibility to employees increase the cost of such a strategy. From an optimistic point of view, this is how I would hope people try to start their companies - with good, utilitarian intentions.

I for one am so tired of all the companies that are in search of a problem that does not need solving. I am also tired of all the cash given to companies that solve some problem, but don't meet the 2nd part of what I said - make money. If you can't tell me how your company will make money in 1 sentence, you have at best a hobby, not a real company. Unfortunately, we have a billion dollar industry of glorified hobbies.

I also feel the same about advertising-based funding, but that's another minefield and to some degree one could argue at least is necessary in some spaces - ex: TV, newspapers. Too many people use ads as a hand-waiving excuse to explain how they will make money.

Unfortunately, the cold-hard capitalistic view of things and use whatever advantages you can get. Sometimes this means screw the morals, passion, soul, ethics, etc. But you know it does work and some people convince themselves they are happy. Maybe they are, but for many of us like me, I seek more intellectual purity than financial glory (as much as I want the latter).

I agree conceptually, but this points out some fun things.

So in other words, we tried that and failed in many ways in a much simpler domain. Now we are trying it again with a much bigger, more complicated scope and set of technical issues. Just like in the VTxxx days, people say something is compatible, and yet it is not 100%. Just like in those days, we deal with the craziness around all that as developers.

I've been working on some stuff dealing with old formats, including VT100. Diving into the source of some very big and well-known projects along with many lesser known things, it's amazing how wrong many implementations are when you check the standards. Related, I've seen the craziest implementations of telnet servers, ANSI SGR codes, and more. Translation: I have 0 faith in humanity to implement standards to the standards. I have less faith in the standards people to create sane standards.

> It's deployed everywhere and that will not change.

Firstly, why do you assume I use x86? I spent a good chunk of last decade programming against PowerPC, Cell, and ARM in my day jobs. From what I am seeing WRT to x86, there is a small but increasing chance that it might change long-term (or needs to). But pulling your leg aside.....

Anything can, will, and does change. People seem to think the browser belongs to the people, but history has spoken to the contrary. The browser is in the hands of a few large companies, some more benevolent than others. There are also committees, and those are often stocked with people from companies as well. The average person has little say over the progress of the web itself, just like they have little say over the OS. We have literally said what you said about <pick any technology> since the 60s.

I'd argue that the browser is not in fact deployed everywhere and does change with great friction. The browser has a terrible record with regard to universal compatibility and deployment. Every time they improve, they get worse again in some other way. Go look at MDN and the cross-browser tables they list for certain features like video. Further, browsers have a long record of making dramatic changes, and APIs with new or breaking changes. There are also a long list of technologies with regard specifically to media experiences that are effectively no longer around or won't be long-term: ActiveX, Applets, Flash, Silverlight, VRML, the list goes on.

Additionally, tons of people are still using ancient browsers, and probably more on at least out-dated versions which introduces lots of inconsistencies, incompatibilities, security holes, and performance issues among other things. There are just as much if not more problems in this regard with compatibility vs. popular operating systems. Writing compatibility with older apps/standards/formats for an operating system is also miles easier as worst case, ancient stuff can be run pretty readily via emulation or simply ported. Obviously it is by no means easy, but generally time and individual efforts are the main obstacles rather than corporations.

There are things about the browser I like, for example that it allows content to easily be network accessible (in conjunction with http/servers). But we have an entire Internet and huge set of people to come up with other solutions too. I'm not saying the browser can't do many things, of course it can, but rather it is a sinking ship that we pile more baggage on and we refuse to stop. While speed is a concern, I've been more bothered by things such as development experience, access to hardware, security, threading (VLC author directly mentions this), rendering, graphics API, and many other things. If the browser is just going to be a frame around something that could run more directly in the OS, I am not sure if I see the point long-term. People make fun of Emacs for far less.

I obviously understand the needs of the archive, but with all due respect I think they overestimate the range of their audience is when it comes with regard to browsing MODs and such (i.e. not your mom/pop 99% of the time). This is a bigger discussion. I could just as easily pull something down from a web server via a native app like Kodi for arguably a better experience, though I am not suggesting that as a solution. What I am saying is I am disappointed by the lack of creativity and the pigeonholing of the internet as a browser-only experience, when as of a few years ago, 1960s/70s/80s problems such as aligning text reliably was seemingly out of the grasp of the average website. "This time we got it right" is not a good attitude either and rarely, well, right. The browser track record is not a good one and saying "a lot of people use it" has never been a good excuse for much of anything and hinders progress. A lot of people supported a lot of bad things throughout human history, but that doesn't change the fact they are bad. I get pragmatism, but for something like an archive I think we can do better and have more time to do it. Write what you can today in JS if you must, quickly and get some experience to the users, but don't move mountains to do it. If the effort to pigeonhole something takes as long as just developing a more correct solution, what is the point? There is also a larger issue with is more upstream to the issue of where it is viewed and involves things more on the level of how they are browsed, searched, preserved, and served to the end-user; the browser is just one end-point of that.

With regard to WASM, I don't care if it is fast or slow. I am more concerned with the development experience from feel to debugging to maintenance. Having worked with multiple architectures and more direct experiences, I am not filled with confidence when attempting more complex things. Even writing code for lets say Cell or 32 vs 64-bit architectures on your PC and then hoping it works is not without its flaws. If anything, we have proven we are really bad at "write here deploy there" style development as computer scientists, even if the write-side is compatible or the same as the target. But again, the problem isn't so much WASM in of itself, it is the ecosystem it has to exist inside and thus be limited by.

I'd answer the rest of these replies, but my main, much better written response was lost between writing it, being rate limited, and posting it. I zoned out and shut down. Neither OS nor browser helped :)

And nearly every one of these does not do it correctly. I haven't recently checked this in VLC either since normally I just use the original trackers and other programs for playback on the real hardware.

From the github of the project you linked:

  "None of the player classes fully implement all the
  features and effects in each file format, but all the major
  ones should be implemented. In addition, there most 
  certainly will be some playback bugs in each player class -
  let me know if you run into some bad ones."

Additionally, I did not see support for some other tracker formats listed (though perhaps supported) such as IT. There seems to be some sort of culture of "cool web page" or "on github" means "works PERFECT" to many people. Worse, sometimes these things do work well, get abandoned, then something breaks them like a security change, so we're back to unsupported again.

As someone who has a passion for what some describe as "scene" formats, I have to say that 99% of the stuff I come across, whether for MOD, ANSI, RIP, DIZ, whatever is fundamentally broken or missing key functionality. The same holds true to other related tech such as vt100/vtxxx emulators. Music and ANSI are some of the worst offenders. Music is interesting since even at the time, different programs would play things back differently, however there was some degree of consistency to make things playable. Literally every ANSI parser in popular places I check have some combination of horrible bugs, missing parts of the standard, abysmal performance, inaccuracies, and worse. It's as if the authors didn't read the specs, because, well, they probably did not (or at least didn't understand them).

The situation gets worse when you think about preservation. Some of the formats had multiple versions and iterations, not to mention were built with different processors and architecture in mind. I was working with one format recently that used a little-endian packed struct that added fields between versions and stored this data as binary. If you read a previous version of the format with the most of the code out there (supporting the latest), it would actually crash. I've actually hit this with archives too, such as with LZH/LHA that are especially used in retro settings or on some platforms.

There's a lot of complexity to older formats and the evolution of things that appeared in more wild circumstances and/or times. Implementations that work perfectly are a rarity and worthy of attention. VLC in this regard does a better job than most. So I think part of the logic whether I agree or disagree is that VLC is widely used, maintained, and active, so it's a bit better for preservation at the moment than random collections of github libraries.

You are right that certainly one could lets say edit the source of chrome and add support. There is no technical limitation here, however see https://developer.mozilla.org/en-US/docs/Web/HTML/Supported_...

Specifically, one thing they mention is licensing with regard to browsers.

Parent's comment has relevance IMO because we're trying to shoe horn things to make everything work in the browser and thus via HTTP as if the browser was an operating system. The browser was built to browse web pages, and at that, didn't even necessarily need to be graphical (see lynx). It's becoming a monster and many of us don't agree with the hand-waiving style justification of adding more and more.

Once upon a time, some people had a vision of using something optimized for files for files, another thing for chat optimized for chat, and so on. There were protocols for these, and apps on top of those. Not all these things were built well, efficient, and/or easy to use. The browser and http were good compromises in many ways, but certainly not without massive problems either. Fast forward and we've just compounded the problems and we keep working harder to lock ourselves into another limited set of technologies we already knew had fundamental issues (people used to/still do complain about processor architecture in similar ways).

I like the browser as a unifying, networked experience, but HTTP + the web browser are not really designed particularly well for some of the things we try to use them for from many perspectives. On top of that, with the issues of JavaScript, security, state, and other things, it gets even more messy. Somewhere along the way we seem to have become disinterested in protocols, lower-level networking, and generally challenging the norms.

Ironic that a lot of web developers preach things like KISS, and yet the browser is a huge example of a project where one could argue that KISS + don't throw in the kitchen sink has run amok. I understand this is not a popular opinion with some people, but I still think it is a valid one. Sometimes I think a little more pausing and thinking things through on lower-levels and with regard to the bigger picture are undervalued and could save us from some of the nonsense of the last few decades.

I'm currently putting what little few hours I have into a few side projects that touch things like XM/MOD and other things like ANSI, RIP, etc. I always seem to share Jason's nostalgia and I get his meaning, but I often don't see eye to eye with him on a lot, whether his treatment of BBSs or some things like emulation.

There are so many red flags here, from licensing to browser vendors to JavaScript to threading to security, I don't know where to begin. It's not that these things aren't doable (even stupid hacks like transcoding are readily doable), it is just that I do not agree with the notion that this must be in the browser for the sake of digital preservation or some sort of user experience.

I know this is a very unpopular opinion in some parts (perhaps HN), but why must everything be in JavaScript and in the web browser? I understand we want things to be cross-platform and network accessible, but I must say that the drift from native to the browser is becoming increasingly disheartening and inspiring constant wtf moments. I get that a lot of people use the web and some people even see it as the computer itself now, but we rarely pause to ask if this is a good thing. Moreover, we invest a lot of time and effort in things that aren't really worth it in the end. I wish in computer science there were more sanity checks and willingness to make progress, rather than to paint flat tires or more crudely, shine turds. The term sunk costs comes to mind.

I like the web and the browser as much as the next person for browsing web pages, but it is not an operating system. Stop making it do everything and anything; that's why I bought a computer not a web browser machine (yeah, I know chromeos). I did not buy my computer so I could stab a few of its cores in the face or make it feel like it is running several orders of magnitude under-clocked. I feel like this is what my browser is doing to me when I see <insert latest abuse> or visit a page displaying a few poorly aligned paragraphs of text resulting in more RAM usage than total RAM I had in my computer for several decades.

What are the benefits and do they really outweigh the costs? I understand that building binaries is full of issues, as is compilation, processor architecture, and so many other parts of the native experience. But the thing is we spend so much time on maintaining and improving our already dated and flawed OSs that at least generally run fast and sanely.

Why would we want to shoehorn yet more things we already do happily in our OS into the browser? Why move something from efficient environments to inefficient ones? Why have to lobby or wait on a standards committee to allow something? Why wait on browser vendors to agree or disagree to make it more complicated? Why am I trading taskbars, terminal sessions, or other OS constructs for browser tabs (can we have new or other ways to mix content types)? How is the browser better as a unifying experience than my OS? Do I really even want videos on web pages or do I want something that is built better for video content? Why not just use what's inside my computer, like that expensive GPU, CPU, RAM, etc. without some ridiculous workarounds or barriers? These are all just random topics to think about, however the point is that it has to be more than, "Wow, that was cool but stupid" moments or "it's in the browser."

We already have put herculean efforts into making safe, pleasant, efficient, cross-platform languages. There are many successes and failures, and the jury is almost universally still out. But the barriers there are much smaller and the environments much more sane. In light of the move to multi-core architectures and bumping up against x86 limitations/physics, and eventually the same for others, it seems insane to me to throw away what we do have right now and relegate it to some browser environment that often struggles to things I can easily do faster/smoother over a terminal session.

I feel with every new "XYZ, but in the browser" we are taking a step away from both the Internet and the general utility of the computer and stomping on what remains. Instead, if we want experiences that must use the Internet, I would like to see more lower-level things implemented. I want to see the next browser, the next usenet, something new that can be built-on and other higher-level things for the every day user. One day it is mobile apps, the next day browser apps, and so on. Let's forget what is trendy or fun and get back to being creative and making progress, leaving the rest to fun "because" side-projects.

Finally, getting back to what Jason and/or archive does specifically, how does this help preservation? It seems to me that browser vendors and browsers in general don't have the best track record in general here and are an added layer of complication vs. <insert OS/programming language/VM/runtime/shim/emulator>. We've also seen any kind of ancillary/hosted/browser-native solution come and go, whether it was applets, flash, activex, whatever. I wish the browser were better with video, but VLC is not what would make it better, just designing and allowing the proper support from the ground-up would be the right way, and at worst, requires the help of the committees and the vendors. I do love VLC.