2,731 karma · joined September 24, 2018
I've played Star Trek: Resurgence a couple of months ago, and I must say I enjoyed your game much more than Resurgence, which is incredible considering that it was written just by you. IMO your game stays very close to spirit of the classic Trek I grew up with.
The post I submitted refers mostly to optimizations to those extension methods.
This clicked for me after having learned Haskell. It also shares some of Haskell's features/pitfalls, such as laziness.
There are pitfalls, sure, and honestly I wouldn't advise a team having no one somewhat experienced with basic functional idioms (including laziness) to use it. It can lead to obtuse and slow code if used indiscriminately. I try to lead by example myself.
The json storage we currently use is kind of "secondary" to the database, and so there currently we don't verify it. The relational database we use does have a schema, of course.
Not verifying the json schema is something that is sorely missing, since the files get corrupted from time to time and this is discovered only during deserialization. Unfortunately this is something I don't have enough time to add, as we're swamped with other work (our app is huge and we are a very small time). We just tell our internal users to re-serialize the database to fix these corruptions, which is unfortunate and costly, but the best we can do at the moment.
I wrote more about that aspect of my internal app here.
https://news.ycombinator.com/item?id=25005993 https://news.ycombinator.com/item?id=38111860
Like I wrote, when in a future version we'll drop the database completely, and work only off those json files, I'll also introduce schema validation. That won't happen for some time though.
Thanks for your answer, I understand now why you didn't measure the performance of simdjson etc. into account.
I have thought in the past about using BinPack for my json documents, but: I want them to remain as human readable as possible, since the reason to move to json from a DB, was to make database diffs into readable diffs, and BinPack isn't readable. I also want users to be able to use existing tools on the json files (e.g. the jq tool), but existing tools don't understand BinPack (yet?).
I think BinPack would shine in an RPC/IPC setting. Just recently there was this big discussion here about systemd replacing D-BUS with a json-based IPC and a huge discussion around the waste of using plain json to do that.
https://news.ycombinator.com/item?id=41687413
I think it's a shame that BinPack wasn't even considered, as it would have made moot some of the arguments there.
I might get in touch with you in the future through your github.
In my day job I have multiple large json files, used in an internal in-house application, into which I serialize a database, and working with them is much faster than using the database (for some workloads). Although the main reason to do so is just for using git for source control.
In the future I plan on migrating all the code to working off the json files, but that won't happen for some time yet.
I was looking for a fast json library and that's why I discovered your paper and your project, although that was some months after I have started working on that project and it was too late to change.
Currently I use the (very slow) json.net library for most serialization/deserialization, it's fast enough for the specific places where I use them, but for some specific workload that has to be fast I use RapidJson (the code in that part is already c++), but only its streaming parser, which blazingly fast, at the cost of some development. I tried using simdjson but had trouble compiling it at the time.
By the way, in your dissertation you don't mention RapidJson, nor json.net, nor simdjson. Is there a reason why you didn't compare them?
So I go to sleep around 12:30 AM each night (sometimes even after 1 AM), and I wake up at 5:50 AM.
That's not enough sleep at all. I do sleep till very late during the weekend.
I've been doing this for 20 years now, and some day it's probably going to catch up on me.
Must be some sort of SO culture we're missing. Because in normal day-to-day conversation between reasonable adults, this would have been an acceptable answer.
During the translation phase, we inject a call between every 2 lines (of the original script language) similar in spirit to the macro above, that updates a global array with a counter. This global array is also built during the translation phase.
There's a separate thread which polls this array and sends updates to the IDE, so users can see in real-time what code was run and when, without needing to stop and debug the code or insert printing statement.
There's a somewhat complicated algorithm which during translation gives a unique ID to each callee in the call tree, so that thread sends to the IDE basically just a number with a counter. This doesn't deal with recursion BTW, I just limited reporting on recursion to a shallow depth. It still runs but just isn't reported.
It's not a complete replacement for a debugger (we also have a debugger) but it's good enough for most simple cases.
There's a similar hack for variables (including local variables).
Recently, some of my less technical users have been disappointed to discover this feature isn't present in other languages / IDEs...
You can also sort of do the same trick in any language, if compiled without optimizations the compiler will usually insert NOP assembly statements between each source line (disclaimer: not all compilers, not all languages, depends on a lot of factors).
This gives you both runtime visualization of running code and potentially time-travel debugging.
So one might be able to run a post-build patching phase and replace these NOPs with a call to a reporting function as above.
Out of curiosity I actually did this with C#, replacing NOPs in the MSIL level, since MSIL is easier to reason about than pure assembly, and it worked very nicely, I got a full "log" of program execution, including all lines executed, when, and all values of local and global variables.
I used the Cecil library to make it slightly more pleasant to read and manipulate IL.
Didn't go on with it other than writing a basic proof of concept.
There are of course tools which do the same for C++, like undo.io and RR, but I'm not aware of any tool doing this for C# / dotnet code. I'm not sure why since while not trivial, it isn't very difficult (and I'm not an expert by any means in this sort of thing).
Roslyn (C# compiler infrastructure) has code generators now, which is nicer than working with IL, but as far as I was able to see they don't support this scenario.
Show HN: I've built a C# IDE, Runtime, and AppStore inside Excel
The last paragraph seems to sum up my view of the world.
In school when learning Stopping By Woods I'm pretty sure the death/suicide interpretation was mentioned by my teacher, and seemed pretty obvious to me even before she mentioned it.
This should be dealt with by pressuring the publishing industry not to inflate books and fill them with fluff. This could be done by not buying these kind of books, and publicly shaming publishers who engage in this behavior. It's easier in non fiction books since the amount of fluff in fiction books is a more subjective matter.
And thanks for the Sorcery recommendation, I haven't played it myself and from reading about it in the past I wasn't sure if it's age appropriate.
I was very worried when we transitioned to git that history will not be preserved and tried to preserve it, but it proved too much hassle so I dropped it.
In fact that proved not to be a problem. Well, not a problem for me, since I remember all the history of the code and all the half forgotten half baked features and why they are there. But if I'm gone then yes, it's going to be a problem. It's in a dire need for a rewrite, but this has been postponed again and again.
First off, I loved your presentation. And your book. As someone who actually bothers to read most of github's "Highlights from Git" blogs, that the, I was somewhat familiar with some of them, but it was still very informative.
Also liked your side-swipe at people who prefer rebase over merge, I'm a merge-only guy myself...
I also took a look at GitButler and it looks like it could potentially solve one of my pain points.
If you're looking for things which are confusing to beginners, for a future version of your book, there are many useful / interesting / sometimes entertaining git discussions/rants here on HN. One of the recent ones is:
For complex tasks, like fixing someone else's mess (or my own), I always start with a visual tool to look at the history and the commits, also look at the reflog (again, in a visual tool, it's much faster for me), understand what the mess is and if I can find anything to salvage, look at some diffs.
Then if it's just a commit I need to return to, I do a reset --hard. If I need to combine stuff from several commits, then I usually use the commandline.
After thinking about it, my intuition is that:
- the Vec was passed by value to the function, which in Rust means something different than passing a std::vector in c++ to a function, due to...
- Due to the language semantics, its reference counter remained 1 while inside the function, but decreased to 0 when returning to main, since in Rust if it isn't a borrow (ref) then it's a "steal" - the callee "steals" the Vec completely from the caller. Or something like that.
- Since the reference counter is 0 when returning to main, the Vec is released, akin to calling its destructor if this was C++.
- SOMETHING changed the value of either the internal pointer to the heap inside the stack-allocated Vec (so now it points to garbage), OR something overwrote the Vec's content with some garbage. I'm not sure what that something is, and why would it do that.
I apologize if my explanation/intuition above is garbage in itself, like I said it's been a while since I programmed in Rust.
Edit: formatting
Learned most of it from a wonderful book whose name I have forgotten, it had a wrench on its cover, I think?
Anyway, still rocking Pascal to this day, since I still maintain 3 moderately complex installers written with InnoSetup, which uses RemObjects Pascal as a scripting language.
4 years ago, a new guy on our team, fresh from school, who never even knew this language existed, picked up Pascal in a week, and started maintaining and developing our installers much further. He did grumble a bit about the syntax but otherwise did a splendid job. I thought that was a tribute to the simplicity of the language.
Glad to have found about Lode Runner: The Legend Returns, Lode Runner Online: The Mad Monks' Revenge, and its remakes from the comments here. There goes my weekend...
Another very good and free remake (which doesn't even include ads!) for mobile platforms is/was Lode Runner 1 by Nexon. It appears it's no longer on Google Play, but it can be found on APKMirror. I still play it from time to time.
Review: https://toucharcade.com/2017/05/19/lode-runner-1-is-a-well-m...
True, in its current architecture it has no way to do it since it doesn't have a daemon or any background process which tracks user actions in real-time, all tracking is ad-hoc, only when git is invoked.
So it basically just side-steps the question, and internally a renamed file is recorded as a file which was deleted and another file which was created, "magically" in the same commit.
When you run merge or log, git tries to guess that a file was actually renamed based on similarity statistics between the text representation. With small, similar files, changed this leads to false conflicts.
I'm not talking theoretically here. I'm talking about something that has caused me, personally, and my organization, many developer hours and real money. I'm talking about something that has hurt productivity and confused me and my users. I'm talking about something that has caused me, just this week, to be alerted to go to work since this happened on a critical project and I'm as the "git expert" was deemed the only one with the know-how of how to deal with those conflicts.
I have spent 2 weeks during covid lockdown, 2 years ago, writing a complicated function which silently identifies those false renames and fixes them before calling git merge. This eliminated many, many support calls. Just last week I have fixed what I hope was the last bug with this function, a nasty corner case.
However this is specific to my situation, where the files are json files that were serialized from the database, and each one of them has a globally unique ID that users can't change. So I'm able to verify which files were renamed in which side and automatically make a "fixup" commit on each side so the 2 sides are as equal as possible (leaving real conflicts in place).
The situation isn't good since this function only works on a single branch merged from the remote. It doesn't work yet between branches. In fact I ought to begin working on changing it to work between branches as I am writing this very comment...
I have many, many other gripes about git, but many of them have been voiced on the comments in this page. But I generally mourn the fact that it is the de-facto source control solution in its current form. Since now its maintainers, quite justifiably, won't break backwards compatibility, and are mostly into fixing bugs and adding some small new features that are QOL improvements.
I still think that some of the bigger pain points could be addressed without breaking backwards compatibility. Similar to what was done with splitting the checkout command to switch and restore, while still keeping the original command.
E.g. for the rename problem above:
1) an optional daemon could be introduced that tracks renames in real-time and records them, both for display purposes in the log, and more importantly to avoid false conflicts during merges / rebases.
2) And/or, users could mark during merges that have conflicts, which files were actually renamed to other ones, in which point in history (perhaps by modifying a generated an input file like rebase does), to help git merge/rebase make more informed decisions instead of relying on statistical similarities.
4 years ago I wrote an internal tool to serialize records of our hierarchical database to a bunch of hierarchical json files, which are then managed in a normal git repo. Each record is serialized to json, "root" entities in the hierarchy are serialized as to contain all their child entities. This is done as the user works - the database is updated and then the correct json file is read, updated and written back. The current SHA is always updated in the database, so there's a way to recognize users worked outside our app with git on the repo, we can detect that and automatically the database based on the diff. There's also an opposite sync.
This was/is a very specific solution to our very specific set of problems. So not applicable to the general problem of "versioning a database".
In fact users now complain that this duality of database <-> repo is confusing and we should ditch the database. Our users are unfortunately not very technical and sometimes require a lot of hand holding.
In theory they're correct but in practice this would mean ditching our ORM and rewriting one from scratch that will work on top of the json files. While keeping the same API outwards. Which is potentially a lot of work. Maybe that will happen, we haven't decided yet.
I wrote much more about it here: