HNHacker News
TopNewBestAskShowJobs

metheus

343 karma · joined June 4, 2010

http://bringingfire.com/what-is-bringing-fire/
submissionscomments
metheus··on MLab is being acquired by MongoDB
The same features exist in Atlas, so you should be good to go there.
metheus··on MongoDB gets support for multi-document ACID transactions
Think about it less in terms of relational/non-relational and more in terms of tabular/document. Tables can only model things in terms of relations. Documents can model them either as embedding or as external references, depending on access patterns.

A decent example is a person record with their email addresses and phones. In a relational DB, you would always and only model those as three sepearate tables, and you would quite frequently need a three way join to assemble that person again.

In MongoDB, you would definitely have an array of emails and an array of phone numbers embedded in that person document, sparing the join on those queries. But in an ecommerce context, you would likely not embed an array of all past orders with all the line items into that person, instead you’d have an array of previous order numbers.

But an order document would have an embedded array of line items, sparing the DB a bunch more joins (and the attendant indices you’d need for joining line items to orders efficiently).

Getting the 10 most recent orders from a customer would involve joining the customers collection with the orders collection (MongoDB’s join is the `$lookup` aggregation stage), but it wouldn’t involve joining the line items to the order.

Is that replicating a relational structure? A little column A, a little column B.

metheus··on MongoDB gets support for multi-document ACID transactions
Goodness, no.
metheus··on MongoDB gets support for multi-document ACID transactions
The EnterpriseDB benchmark is just a hit piece. See https://newbiedba.wordpress.com/2017/05/26/thoughts-on-postg...
metheus··on MongoDB gets support for multi-document ACID transactions
>Aggregates carry with them a memory limit.

I think you are referring to the 100MB RAM limit, but that’s not a hard limit, it’s more of a bad default. The `allowDiskUse` option lets MongoDB write intermediate results to the disk (which is exactly what SQL databases are doing).

> But aggregate queries really do not give you the power of SQL. They are great for transforming stuff for a report, but I would avoid using them for anything else unless 1) it can be cached (and therefor properly invalidated) and 2) doesn't need to be "real time"

I really don’t see a difference between what you can do with MongoDB and SQL. I can’t say much more without knowing specifically what impediments you have in mind, but I would certainly like to hear more. For example, why do you cite results caching and lack of real-time requirements?

> Being a document, schema-less database,

I guess if you’re on 3.4 you can’t take advantage of JSON Schema yet, but keep that in mind as a part of your upgrade plans. In the meanwhile you can still use document validation?

> your "quality of life" as you scale with MongoDB is going to be heavily dependent on how you structure your documents and the types of their fields.

This is completely true, but couldn’t we say that just as much about any database? I’d put money on there being way more grief out there over bad tabular schema than over bad document schema. I mean, who’s worked on large-scale systems that hasn’t put off implementing great ideas, or had to hack up app code to compensate for a restrictive schema, because you can’t take the pain of ALTER TABLE?

> Are you treating your collections like SQL tables?

Ouch. Please don't!

> Have any many-to-many relationships in hot code paths? Welcome to hell.

That's probably fair, but if you're in hell to a vastly greater than you would be with Postgres, I'm pretty sure that's a modeling problem. Again, can you tell me more about the particular example?

> Its type system is also limited compared to modern SQL DBs. Storing IP addresses, and want to query them based on a given CIDR range? Postgres makes this eas, MongoDB has you writing code that does sub-queries

That's 100% legit. MongoDB needs to do a ton better with that... types rule.

metheus··on MongoDB gets support for multi-document ACID transactions
While doing so, I recommend you think some about the meaning of embedding relationships vs. referencing them in other collections. MongoDB can do both (the aggregation framework provides joins in the form of the $lookup stage).

If you have a “one-to-many(some)” kind of relationship, embedding is a good option, and gets you ACID semantics without even resorting to multi-statement transactions.

If you have a “one-to-many(thousands or more)” kind of relationship, objectID references (a la foreign key) is more likely what you want.

metheus··on MongoDB gets support for multi-document ACID transactions
> Having to resort to doing things like map/reduce for a simple group by / order is not the way to go IMO.

Perhaps you are unaware, but that is a straw-man. That’s not how you’re supposed to with MongoDB. You’d use the aggregation framework:

db.sales.aggregate( [ { $group : { _id : null, totalPrice: { $sum: { $multiply: [ "$price", "$quantity" ] } }, averageQuantity: { $avg: "$quantity" }, count: { $sum: 1 } } } ] )

https://docs.mongodb.com/manual/reference/operator/aggregati...

https://docs.mongodb.com/manual/reference/operator/aggregati...

This has been the case since MongoDB 2.0, in 2012.

metheus··on MongoDB gets support for multi-document ACID transactions
John radically overstates his case in that article based on incomplete information; note that none of the code he points to is even 1.0 code.

Nonetheless, it’s accurate enough to say that the 1.0 BI connector used Postgres code. It was definitely the fastest way to deliver it.

2.x was built from scratch, it uses no Postgres whatsoever.

metheus··on MongoDB gets support for multi-document ACID transactions
Quibble: “schemaless” isn’t accurate, MongoDB supports schema (JSON Schema, in fact). We often say “dynamic schema”.

But your main point is good—- JSON isn’t the essence of it.

And yet it’s part of it, in that you could store XML and still be a document DB; MongoDB definitely chose JSON as the medium deliberately.

metheus··on The Marketing Behind MongoDB
That EnterpriseDB benchmark is a comically inept hit piece. See: https://newbiedba.wordpress.com/2017/05/26/thoughts-on-postg...
metheus··on The Marketing Behind MongoDB
What you wrote isn't a fact, it's a claim, and a shoddy one at that. If you're basing it on anything other than hearsay, I'd love to see your proof.
metheus··on Farewell, Solaris
See Aly's top-level comment re: patches and download numbers.
metheus··on Startup Engineers and Our Mistakes with MongoDB
I suggest then that you get the latest versions of MongoDB and PostgreSQL and run that test and report back. Verifiable isn't the same as verified.
metheus··on Startup Engineers and Our Mistakes with MongoDB
If you're going to get on the "MongoDB was overhyped" bandwagon, you should probably not post links to performance comparisons you pulled off of an EDB Postgres website.
metheus··on Why Did So Many Startups Choose MongoDB?
> I wouldn't be surprised if half of the commenters haven't used Mongo, or stopped in the 2.0-2.2 days.

You would be correct, as the mountain of ignorance about MongoDB on display in this and all HN threads on MongoDB attest to.

metheus··on CouchDB vs. MongoDB
Just to be clear, a better term would be "dynamic schema". You can ask for data type validation since 3.2.
metheus··on CouchDB vs. MongoDB
Since version 3.2 you've been able to enforce data types with validation.
metheus··on CouchDB vs. MongoDB
Ok, but even prior to 3.4, in what sense did you mean it couldn't guarantee something was a number at all?
metheus··on CouchDB vs. MongoDB
> a database that can't properly store a number as a number or even guarantee that it's a number at all.

What do you mean?

metheus··on JSON Feed
We don't prefer JSON to XML for any reason other than that XML is terrible by comparison.
metheus··on Why Twitter Is Failing to Grow
Twitter isn't growing because it is a bad product. Everyone using it has to train in a host of workarounds. Posting pictures of text to get around the character limit. Tweetstorms to get around the character limit. ".@mention" to get around the reply behavior preventing users from composing text naturally. Lousy feed grooming. (I love <some tech> but I hate <some sport>. Too bad! <some luminary in that tech> loves <that sport>!)

Twitter has some great aspects. It's really lightweight and doesn't enforce mutuality, making it possible to form communities on events on topics, or to learn in realtime what's going on somewhere. But as a product it is complete and utter garbage.

And twitter can't improve, because it's hardcore base is super convinced that its critical flaws are features.

metheus··on How Uber Used Secret Greyball Tool to Deceive Authorities Worldwide
Given #deleteuber, it clearly matters to some consumers. I suspect that it matters enough to give competitors another few % of the market per major fiasco, but not enough to alter the overall trajectory of the market.
metheus··on Testing Linearizability with Jepsen and Evergreen: “Call Me Continuously”
If you have any questions about getting Jepsen to run in your CI, or setting up Evergreen, let us know.
metheus··on MongoDB 3.4.0-rc3
> Is there a comparison to Jenkins available?

Not exactly, but we published a blog post about why we took the time to write a new CI system, and that post does touch upon why we didn't use Jenkins + plugins.

https://engineering.mongodb.com/post/evergreen-continuous-in...

metheus··on MongoDB 3.4.0-rc3
I hadn't noticed this before, but aphyr has an elaborate and articulate ethics policy: http://jepsen.io/ethics

I like it.

metheus··on The future is fewer people writing code?
I just had a waking nightmare of AppleScript.
metheus··on The Strange Politics of Peter Thiel, Trump’s Most Unlikely Supporter
Ok, I understand. You literally mean that both of them will only introduce objectionable policies, not that "less objectionable" might mean, in some cases, "good."

I still don't see the outcome you predict.

TLDR; power consolidation is a risk factor that leads to actual bad policy, not something with actual, palpable negative value in and of itself. E.g. the power of the federal government to end slavery is a win in my book.

* * * *

Let's map "objectionability" onto a number line.

A neutral policy is zero. An objectionable one is on the negative side, a non-objectionable one is on the positive side. The magnitude of the objectionability or virtue yields the magnitude of the number.

I'll go with your presumption that both Clinton and Trump's policies are only found on the negative side of that line. And we'll deal with their policies in aggregate. We'll also assume that current policy is perfectly neutral, at zero:

<---------T-------------C-----0

T = Trump APV (avg policy value)

C = Clinton APV

0 = CPV (current policy value)

Let's map effectiveness to a float between 0 and 1. Simple. If Trump's policies are twice as bad as Clinton's, he has to be half as effective as she for them to equal out. Here's a little table:

APV, Effect, Total value, Who, Worse

-20, .05, -1, Trump, O

-2, .95, -1.9, Clinton, X

-20, .1, -2, Trump, X

-2, .9, -1.8, Clinton, O

-10, .05, -0.5, Trump, O

-2, .95, -1.9, Clinton, X

-10, .2, -2, Trump, X

-2, .8, -1.6, Clinton, O

The problem with that simplification is the assumption that CPV is at neutral, zero. And mapping effectiveness to a scalar factor of APV is also incorrect -- instead, it has to be understood as a normalization factor, attempting to return CPV to APV. The greater the effectiveness, the more quickly CPV achieves APV.

Should any factor bring CPV further negative than Clinton's APV, her effectiveness would become a force that lifts CPV. Meanwhile, Trump's is always attempting to pull CPV further negative towards his abysmal APV.

metheus··on The Strange Politics of Peter Thiel, Trump’s Most Unlikely Supporter
Shall we discuss hard epistemology? Fine.

> You have no idea how he got to his position

False. I cannot prove how. The objection is pure sophistry. Were I to ask him, "how did you arrive at this position", his answer could be a lie. Would you commit the fundamental attribution error and assume that he would tell the truth? Perhaps he secretly doesn't even believe what he claimed to believe!

I doubt very much that you live your life doubting all things beyond your own thoughts. Trotting out a single fallacy mode and applying it in a scope carefully limited to bolster your argument is a rhetorical method for scoring debate points, not sound logic.

You are attempting to use "intellectual honesty" as a cudgel. Who wouldn't want to be intellectually honest?! Of course, you'll be the judge of when someone has met the criteria for honesty, and that will depend on whether they agree with you or not.

I am under no misunderstanding as to what I can and cannot know. I am completely comfortable with using mental shorthand to navigate the world. I do not need to stop and listen to every individual born-again christian to discover if they are the one that holds the key to my finally knowing the truth.

I do, in fact, have a very good idea as to how Thiel arrived at his position. I have a very good idea about why Strom Thurmond opposed the civil rights act of 1964. I have a very good idea that the sun will rise tomorrow.

> you go on this big rant that places you above him in terms of virtue

Yes, precisely. We all hold values. I find his to be in conflict with mine.

metheus··on The Strange Politics of Peter Thiel, Trump’s Most Unlikely Supporter
> Once you have a few dozen million dollars, it's not so hard to maintain and grow that (unless you go on idiotic spending sprees)

Or unless you're Trump! rimshot

http://fortune.com/2015/08/20/donald-trump-index-funds/

http://www.moneytalksnews.com/why-youre-probably-better-inve...

metheus··on The Strange Politics of Peter Thiel, Trump’s Most Unlikely Supporter
Under what rubric is the more effective rollout of less objectionable policies a greater loss to society than the less effective rollout of more objectionable policies?

This dogma holds consolidation of power to be bad /in and of itself/, rather than because of a consequence, such as "consolidated power leads to suffering." If that's your opinion, fine, but it's no different than any other religion.

← PreviousPage 2 of 3Next →