1,318 karma · joined November 6, 2016
Formerly at Palantir, Apple, and co-founder at Symbolica AI. Currently at GitHub.
That is how change happens in the world, not by trying to ruin the lives of individuals emblematic of a flawed system but quite likely perfectly decent people in and of themselves.
My expectations are that a candidate can identify that there is a concurrency consideration to be made in the first place, and have some knowledge around how to handle concurrency issues in their language of choice, since different languages take very different approaches.
For JS/node that could mean identifying that the main execution is single threaded, so you need to use async/await or callbacks for asynchronous tasks to prevent blocking all other execution. In go that could mean understanding that passing messages is generally safer than sharing memory. In java that could be an awareness of the concurrency primitives that the language has like atomics and concurrent data structures (I could never expect a candidate to fully grasp the concurrency nightmare that is java).
I also usually give candidates a laptop and free use of the internet. I try to ask questions that are more grounded in the actual work we do and not just toy algorithms, which means letting people look up documentation, data structures, and whatever else they want. If we're being honest with ourselves, the ability to effectively search the internet is just as important to software engineering as the coding itself, if not more so.
It's 2020: computers have multiple cores, and it's great to know how to use them, but time and time again I see candidates get tripped up with basic concurrency concepts. Deadlocks in go, event loop blocking in node.js, unsafe memory access in Java or C, they come up all the time in interviews I've done.
"Track the median value in a stream of integers"? No problem. Clean code, no errors.
"Track the median value of two streams of integers which are coming in concurrently"? Data loss, deadlocks, concurrent modification exceptions... Sad times.
I know it's a bit more complex than basic algorithms and data structures, but it also is much more important in the "real world", at least in my opinion. Every language has libraries of data structures already written, but concurrency solutions are often context specific and need handwritten solutions.
“Scooby” => “Scooby Doo” => “Clue”, making “I haven’t got a clue, mate”
“Ruby” => “Ruby Murray” => “curry”, making “let’s go for a curry after this”.
After dog detective scooby doo resolved to “clue” I was hoping that there was at least internal consistency between the rhyming phrase and the intended word, but as far as I can tell there is no relationship between a Northern Irish actress from the 1950’s and Indian cuisine.
The rest are soon to follow I’m sure.
The catalog functionality is just more complex given the granularity of our permission model (configurable down to the package level), so it unfortunately didn’t make the cut for the beta feature list.
You should already see `Container` listed as a package type in the UI, and can begin publishing packages by creating a personal access token with `read:packages` and `write:packages` scopes as appropriate - https://docs.github.com/en/github/authenticating-to-github/c...
If you hit any compatibility issues, especially any with regards to the OCI spec, please let us know!
After we go GA, we'll be charging for storage and data transfer on private images only, the same the current registry offerings. If your popular image is public, you'll incur no bandwidth charges, regardless of how often it's pulled.
Our only namespacing enforcement is that packages must be published under a user or organization namespace for which you have write permissions. For example, I'd be able to publish under `ghcr.io/taywrobel/redis`, or under `ghcr.io/github/redis`, but would be disallowed from publishing as `ghcr.io/saxonww/redis`.
Other than that restriction, you can have nested image namespaces with arbitrarily many segments; i.e. `ghcr.io/taywrobel/dev/redis` would be valid.
The general approach would be “give us 2-4 weeks to integrate your data and give you a demo”, and would give organizations a demo of the software using their own data.
There were some well known stories internally, of things like police agencies forgetting to ask about the product at all, and instead taking notes on suspects and investigations being shown to them in the demo for further follow up, because it was making connections and breakthroughs on stalled or cold cases. Those were the ones that they knew were hooked.
The flip side of this is the cost of deploying anywhere from 2-20 engineers into the field to handle the system setup and integration on the fly.
Reducing binary size makes a difference when the app is deployed to remote servers thousands of times under normal operation.
Storage and bandwidth is cheap, yes, but it adds up.
Tim Cook spelled it out pretty clearly to Congress when he testified last week; Apple does not have majority market share in any market they’re in. Not in smartphones, tablets, laptops, desktops, or wearables. And they don’t want to have more market share. The inflated prices and being seen as a “luxury brand” is a great way for them to continue making a huge profit while restricting market share to the point that they can skirt around most monopoly law.
The downside there is to maintain growth, they need to continually push into new markets, which they’ve been doing about once every 5 years (desktops -> laptops -> phones -> tablets -> wearables -> automotive/AR?)
Palantir makes that easier, sure, but at what point is a tool sufficiently generic so as to not be held accountable for what it’s used for? And if there is no amount of generalizability that absolves the author of software from what its used for, how do you square that against open source software?
Should the elasticsearch authors feel guilty if their software is used for something unethical as described here? No money is changing hands is one difference, but for a decision of morality does the payment matter?
To be clear, I don’t think there’s one right answer here. Different people will absolutely have different opinions, and that’s not the type of disagreement that’s likely to be swayed on a forum, I just wanted to provide you a different way to think about it, if you’re open to doing so.
If you think your best hope for addressing government behavior you disagree with is to try to take their tools away, you may as well stay silent completely because it isn’t going to do anything.
If your grievances are with the government, take your complaints to the government, no?
Now take that, add a few more database technologies to do remote indexing and on the fly imports on federated databases, and some geospatial hotness, give it a swanky UI (now web based, not java), and that’s pretty much the platform, or at least was as of 2014 when I left.
And the thing is, they’re remarkably open about all of that... it’s just that people like the boogeyman narrative more, so all the openness about their tech is completely overshadowed, but there’s a reason it and Pinterest were the two hottest internships for college CS students in 2012/2013.
They don’t even source or store data themselves... they just provide a glorified federated search engine on top of data organizations already have, with fancy visuals and user interface on top.
Apache is great at software engineering, but sorely lacking in product design. Because open source software is almost definitionally not a product, but a tool.
With that comes increased bifurcation of the tooling when different requirements arise, and increased complexity with running it. Kafka and pulsar both have zookeeper as an external dependency, for instance. Pulsar has an extra dependency even in bookkeeper, one of the few things I’ll readily fault it for. It’s a stark contrast to openly commercial products like CockroachDB, which has a single static binary, with symmetric nodes, and built in management UI. It’s a product, not a tool.
FWIW, Apache has a project directory - https://projects.apache.org/projects.html
But it’s a far cry from a comparison matrix as you (and I think many other confused and disheartened engineers) desire.
I get it, I really do. Your manager has heard of Kafka, as has your PM and CTO. Nobody has ever fired for buying Kafka. It’s the new IBM.
“Buying Kafka? It’s open source, so it’s free!” you say, my naïve friend who hasn’t heard of Confluent.
Yes, Kafka is free until you go to production and need things like mirror maker, for multi-site replication. Then it’s time to pay up to the company that has taken over and monetized the project.
But its a pain to run, a pain to debug, uses avro as a binary schema, because everyone uses avro, right? And partitions are great! Until you need to change them and then you’re in for a world of strange and potentially unnoticed bugs from a discontinuity in partitioning as the topic grows.
Or... you could have Pulsar. Dynamic partitioning, more expressive subscription models, multi-active as part of the core product. No BS marketing claiming “exactly once delivery semantics”, aka more than once with receiver side deduplication, aka what TCP does and has always done.
There is no reason to be building something new on top of Kafka in 2020.
I don’t. Your “lost interest“ on a currency that has nothing beyond speculative value is literally worthless to me. The increased energy production and subsequent pollution that proof of work costs, however, has a real value and does real damage to the environment.
Proof of work currencies need to be shut down, for environmental impact reasons if nothing else. You keep playing with your speculative assets all you want. I don’t subscribe to the idea but it doesn’t bother me, until you start damaging the real world with it. Then I have to insist you stop.
Since they’re claiming something provably false, it’d be nice to have some empirical evidence as such.
It seems absurd that a two letter combination not used for nearly a century is enough to disqualify a name on an entire continent.
We won. It’s over. We can reclaim their terms and symbolism now.
Well, yeah, it's meaningless if you cut off the second half...
> ...to process and distribute data.
That's what it does. The adjectives before it aren't the meat of the sentence.