HNHacker News
TopNewBestAskShowJobs

JelteF

1,084 karma · joined April 27, 2013

Building pg_duckdb at MotherDuck

[ my public key: https://keybase.io/jeltef; my proof: https://keybase.io/jeltef/sigs/ZMan3toJpggBpU3_dAtlav5NbAMU_TcEoMp4LaxqiwA ]

submissionscomments
JelteF··on Colorado Town Offers 1 Gbps for $60 After Years of Battling Comcast
Downloading Steam games is one thing that actually takes advantage of my 750Mbps.
JelteF··on Ask HN: Are you ok?
It's not dating advice, but this comedy show might make you feel a bit better: https://www.netflix.com/nl-en/title/80223685?preventIntent=t... (Daniel Sloss: Jigsaw, in case link doesn't work for you) There's some good life lessons in there about the feeling of needing relationships and it's very funny at the same time.
JelteF··on Why All of Our Games Look Like Crap
Completely agree that it doesn't have anything to do with the fact that they are low res. However, it's not just the colors. For me it's mostly that the lighting across artwork is very inconsistent. The artwork separately is quite good, but because some stuff is much lighter than the rest it feels like it's gathered from multiple separate games and put together in one.
JelteF··on YAML: Probably not so great after all
The reason this is a fault of the DDL and not the parser is that the DDL spec decides that it has label that evaluates a command. The parser then has two options, either implement it or not conform to the spec (and essentially implementing a different DDL). For programming languages it makes sense to have an eval label/command. For configuration/serialization DDLs I think it's a terrible choice.
JelteF··on Go Contracts – Draft Design
> whereas the contract version would be "a slice of a type that implements io.Reader" but it would have to all be the same type.

From what I understand, you could still pass a []io.Reader to a contract version of the function. A slice of interfaces would still implement the contract (and is the same type, i.e. io.Reader). However, you could also pass a []os.File, which is not possible to do with the interface version of the function.

JelteF··on Submit Your Claim in the Equifax Breach Settlement
In the US yes, in a lot of other country in the world they don't have a credit score as crazy as this. The only thing that's registered in the Netherlands for instance is how much you have to pay every month in interest and if you have failed to pay any of the bills for loans you have. This is submitted to the government and requested and updated by banks.
JelteF··on Common mistakes in PostgreSQL
I think what the parent tried to say is that there's no such thing as timeSTAMP without a timezone. There's like you said a time without a timestamp, but it does not contain a date as well.
JelteF··on Common mistakes in PostgreSQL
It's a wiki right, so you can simply add it
JelteF··on JIT-Less V8
As far as I know Apple doesn't require browsers to use the same rendering engine as Safari. It's just that so far any rendering engine + JS engine required a JIT to make it performant. And because a JIT is not allowed because apps are not allowed to write to executable memory. That's why the 3rd party browsers all used the same rendering engine as Safari.
JelteF··on Google cans the Chromecast Audio
For 20 bucks you can get an hdmi audio extractor[1]. So for a little bit extra money you can simply connect the regular chromecast to your amplifier.

[1] https://www.amazon.com/slp/hdmi-audio-extractor/xc32yz93wutk...

JelteF··on Sqlfmt: an opinionated online SQL formatter
Looks really cool, but I have one suggestion. Have you considered using leading commas? It makes big SQL statements much easier to edit and it improves diffs. I normally use trailing commas for my code, but most sql dialects don't support it so I fallback to leading ones there.
JelteF··on Go 2 Draft Designs
Overall really great that Go is addressing it's current biggest issues. I do think the argument for the check keyword instead of a ? operator like in Rust is quite weak. Mainly because a big advantage of the ? operator (apart from the brevity) is that it can be used in method calling chains. With the check keyword this is not possible. AFAICT the check keyword could be replaced for the ? operator in the current proposal to get this advantage, even while keeping the handle keyword.

Furthermore the following statement about rust error handling is simply not true:

> But Rust has no equivalent of handle: the convenience of the ? operator comes with the likely omission of proper handling.

That's because the ? operator does a min more than the code snippet in the draft design shows:

    if result.err != nil {
	    return result.err
    }
    use(result.value)

Instead it does the following:

    if result.err != nil {
	    return ResultErrorType.from(result.err)
    }
    use(result.value)
This means that a very common way to handle errors in Rust is to define your own Error type consumes other errors and adds context to them.
JelteF··on Winds 2.0: It’s Time to Revive RSS
All these services have free plans either if you have little traffic or if it's open source. So it's definitely possible to self host this (although maybe not completely trivial).

Disclaimer: I work at Stream (but haven't really worked on Winds)

JelteF··on Algorithms Behind Modern Storage Systems
There's is one important detail that is never fully articulated in this article. It says B-trees are read optimized. This is true, but there's only a big difference for random reads and not sequential reads. Because seeking a key in LSM is relatively expensive, because it has to read multiple indexes. But after this is done, reading the a lot of keys an values from this point on is not expensive (or at least not much more so than in a B-tree). The reason for this is mentioned in the article, which are the SSTables. These allow for quick through the keys, because they are stored on disk in sorted order.
JelteF··on Hijack of Amazon’s domain service used to reroute web traffic for two hours
Usually yes, but that's only the case if you try to spoof DNS/routing for end users. If you can fake the DNS/routing for the certificate registrars you can trick them in giving you a certificate.
JelteF··on RSS is undead
We're currently working on an Open Source and good looking RSS + podcast app that has personalization built in. If that sounds interesting to you be sure sign up here to be notified when the beta is released: https://getstream.io/winds/
JelteF··on Former Uber Backup Driver: 'We Saw This Coming'
These people are basically giving driving lessons, just to cars instead of people. Regular driving instructors can do their job fulltime in a safe way without getting distracted. They don't control the car most of the time, yet they still are able to stop it in case of a student error.

So to me the question is: What is making this difference? Is it the level of training for the instructor? Human interaction? Is it the sign on their car that shows it's a lesson car to the other drivers? Or is it something else?

PS, this post is based on the way driving lessons happen in the Netherlands. I'm not sure if it's completely comparable to the US.

JelteF··on Why GitHub is not a great package repository
That's exactly the problem. People can just remove their Go packages.
JelteF··on Regexper – Regular expressions visualizer
Looks neat, but after a quick look I think I still like https://www.debuggex.com/ better. Going step by step through the regex for a given string is really a killer feature for me.
JelteF··on Stream and Go: News feeds for 300M users, built on RocksDB and Raft
I have to say I'm not very familiar with Riak. A quick google makes it look similar in functionality. I'm not sure about speed though, RocksDB is really really fast and storage efficient. We tried out a lot of other embedded KV stores. Also Riak does seem to miss one important feature for us, iterating through the keys in lexical order. We use that a lot to build features on.

Even if this feature was/is available in Riak, I still think this was the better choice for us. Bringing this core component in house has been a real boon in gaining good and more importantly predictable performance for our API.

JelteF··on Stream and Go: News feeds for 300M users, built on RocksDB and Raft
Engineer at Stream here and I'm really not the guy to talk to about pricing.

However, I can tell you that our features that are hardest to implement in a scalabale way are aggregated feeds (e.g. one message with "tom watched 10 videos") and custom ranking (reddit/HN style ordering with time decay and points). If you're feed system doesn't have either of these features it might indeed be possible to do it cheaper than our more expensive plans.

However keep in mind, a lot of the cost is in the availability and reliability of our service. Basically for every type of server we run (including databases), we need to run at least two to be able to stay online in case one dies. Finally building and maintaining a feed system is not cheap in terms of developer cost (try finding a dev for $10k a year).

JelteF··on Stream and Go: News feeds for 300M users, built on RocksDB and Raft
Not as far as I know. We did benchmark a lot of embedded storages before deciding on Rocksdb though.
JelteF··on Stream and Go: News feeds for 300M users, built on RocksDB and Raft
I also think it's really interesting stuff and love working on it. I'll definitely write something about that if/when I do a blog post about it. But it's quite simple at its core. Like the current post mentions, we use Raft to do it. We simply have a cluster of 3 nodes, each in a different zone. If one zone goes down, there's still a majority of nodes up, so enough to keep the cluster running. I recommend reading the raft paper for more details, it's one of the easiest papers to read and understand I've ever found. https://raft.github.io/raft.pdf
JelteF··on Stream and Go: News feeds for 300M users, built on RocksDB and Raft
Main developer of Keevo at Stream here and you ask a very good question. I've asked myself the same question quite a lot. And I think it actually warrants it own dedicated blog post at some point. To give some idea though, the main reasons are, cost, simplicity, control and wanting to have a very good understanding of the database internals.

Cassandra is very scalable, but it's not very efficient. The hosting costs for our cassandra cluster were so big that it was infeasible to run it in another region as well.

Appart from that we've had (partial) downtime a couple of times because one node just started going crazy because of unclear reasons.

Keevo solves this by not trying to be cassandra, but be much simpler. It doesn't do schema's or indexes. All it is is a very fast ordered key value store that is stored to disk and replicated automatically to multiple servers (using raft). Any other features we need, we build on top of this usually outside of Keevo itself. This simplicity saves us a lot of hosting costs and makes its performance much more predictable and easier to debug.

Last but not least a very important advantage is control and understanding of the database internals. Because we build Keevo ourselves, we know the performance and consistency tradeoffs it has and can change/improve them when needed.

I hope this helps in understanding our choice. It's definitely not something I would recommend for most companies, but since our product is storage at its core it makes sense for us.

JelteF··on What I learned in 2017 writing Go
All our production applications don't have their vendor committed. By running "dep ensure" it dep restores the exact same vendor directory from the lock file, we do this in CI where we build our binaries. For dependencies that don't support semver dep uses the master branch by default and pins a commit to support reproducible builds. The only case where this has caused problems for us was when one of our dependencies force pushed to master, so the commit that was pinned didn't exist anymore. This resulted in a failing build though, not an inconsistent one.

Other than working around that issue the only case where you actually need to commit vendor (at the moment) is when you want your project to compile reproducibly by only running "go get project". If you are fine with telling your user to run "dep ensure" (or make) first this is not needed. This is not usually an issue when working with colleagues, but can be nice for publicly released projects.

JelteF··on What I learned in 2017 writing Go
Always nice to hear :). Be sure to report any issues you have on GitHub btw, the setup and usage should be as painless as possible.
JelteF··on What I learned in 2017 writing Go
My main method of keeping track of this is creating a symlink to all my actual projects in my regular ~/work directory. Appart from this most people use vendoring, but I use (and maintain) virtualgo instead: https://github.com/GetStream/vg

It supports working on a dependency of a project locally with ease, by using "vg localInstall" which is a extreme pain with vendoring. And also it supports version pinning of executables.

JelteF··on What I learned in 2017 writing Go
Dep (which is very similar to bundler) really solves most of the problems with Go dependency management and is the official tool everyone is converging to now: https://github.com/golang/dep

For the problems it doesn't solve I've created virtualgo at my job, which is similar to rvm or virtualenv: https://github.com/GetStream/vg

JelteF··on What I learned in 2017 writing Go
Running "dep prune" after "dep ensure" should remove a lot of unnecessary files (this will be integrated into ensure somewhat soon). Also, in a lot of cases it's not needed to commit the vendor directory, so its size doesn't really matter then. Finally, I really recommend checking out the #vendor channel on the Go slack if you have any more questions regarding dep, everyone there is really helpful.
JelteF··on What I learned in 2017 writing Go
Main developer of vg here. In principle vg should work for packages outside of GOPATH when using full isolation mode (which has its own set of problem, see README for details). Its dep integration doesn't work then though last time I tried. As far as I can tell the main reason why Go tooling needs GOPATH is because everything uses absolute imports and relative ones can translated to absolute ones by always working from inside the workspace.

I've solved my main issue with GOPATH in a very easy way though. I just create symlinks in my normal directories (like ~/work) to my projects in GOPATH. This allows me to go to my own projects easily while still having everything in GOPATH.

← PreviousPage 3 of 9Next →