HNHacker News
TopNewBestAskShowJobs

meritt

6,947 karma · joined February 24, 2011

submissionscomments
meritt··on MessagePack: like JSON, but fast and small
Sure, but if performance is the actual issue, then you'd likely pick an even faster and more compact wire format (proto, avro, thrift).

I guess the ability to have a schemalass and no type checking format (just like JSON), while enjoying some performance benefits? It just feels like a weird niche to me.

meritt··on MessagePack: like JSON, but fast and small
I fail to see the use case too. If you want a compact wire format, there are numerous options (protobufs, avro, thrift, etc) and their size is achieved by storing the data in a compact binary format and the schema separately.

With MessagePack the schema is embedded alongside the data, exactly like JSON. So it's just binary instead of text encoding, which saves some space, but as you pointed out, standard text compression algorithms are going to likely perform similar or superior.

meritt··on Stanford cancels in-person classes for rest of the quarter
I disagree. When someone looks at an epidemic that just made landfall a month ago and compares those numbers on an absolute basis to a virus (seasonal flu) that's completely embedded in the fabric of the entire US, and concludes that since A<B, it's not a threat, they are indeed a complete fuckwit.
meritt··on Stanford cancels in-person classes for rest of the quarter
I look forward to HN's resident statistics gurus to explain how the traditional flu and drunk driving kill more people per year than the paltry 14 deaths in the US and therefore everyone is overreacting.

Edit: Apparently my sarcasm wasn't thick enough so let me make my point crystal: COVID-19 is a real, genuine threat that's just getting started, but HN is filled with armchair fuckwits [1][2] who dismiss it because they lack simple math skills.

[1] https://news.ycombinator.com/item?id=22507532

[2] https://news.ycombinator.com/item?id=22508819

meritt··on Netlify raises $53M Series C as microservices approach to web development grows
> but it seems like you still need a backend

Of course. It's just managed by someone else and you often are paying per usage. And you'll find yourself needing to wrangle numerous serverless platforms to actually build out relevant business logic if you want to do something meaningful, one third-party microservice at a time.

> Also, because it's just a static site, there's no backend or database, so you get better security.

It's not as trendy to think of it this way, but "static sites" are just caches with manual invalidation.

meritt··on Not being an asshole will make you more money
> by backing out all of our changes and only putting in his changes

That so feels like a mistake because that's the sort of tactic that would completely and utterly remove their capability of ever working as a corporate lawyer ever again for any company.

If that genuinely happened, that lawyer needs to be disbarred, but that's a pretty damn egregious accusation.

meritt··on Running Istio In Production
> Maybe there's a good reason to have as many services as they do.

Or maybe we're in a tech bubble and if you want engineers, you have to acquiesce to their demands of working with the latest shiny tools while they create mountains of technical debt, because if you don't, they'll just go to another startup that allows that behavior, or they'll go twiddle their thumbs at a FAANG while banking $300k+ total comp for their 5-years of experience.

I'm not trying to say one could replicate this business with a few scripts, but it sure as hell doesn't need a service mesh that looks like a mutated SARS-CoV3.

meritt··on Clickhouse Local
You misinterpret my comment as being against databases, when in fact I'm simply against the notion of utilizing a purpose-built OLAP database (Clickhouse) merely as the storage layer for yet another analytical platform (Pandas). In this hypothetical scenario, before the data ever makes it into Clickhouse, there's a very high likelihood it would already live in an OLTP database such as Postgres or perhaps parquet files living on S3 designed to be read by Athena or Redshift Spectrum etc.
meritt··on Clickhouse Local
Where did I suggest storing the data on a local machine? Common usage when using pandas/spark/etc is storing source data in parquet/orc/etc and they fully support data partitioning. That doesn't need to come from a local machine by any means.

My bigger point to the OP was why bother using a DBMS that is specifically tuned for running fast analytical queries if they only intended to use it as a storage layer and pushing all of the analysis into Python. Use a solution focused on storage, not analysis, if that's the use case.

meritt··on Clickhouse Local
Why would you bother using a database like clickhouse to store data if you're just going to analyze it in pandas? Just store it in a csv, parquet, or orc.

> As i'm sure things like pivot tables and rolling windows are a PITA in SQL

I can't speak for clickhouse, but group-by and window functions are a very standard part of any SQL analysts toolbelt.

meritt··on Covid-19 precautions for Stripe employees and customers
I've ran a remote-only company for 8 years and generally struggled to find employees who are suitable working remote. Especially junior employees I've had no luck. The best remote employees I've found are those who tend to be entrepreneurial in nature and usually more senior. They seem to possess the necessary self-discipline and taking ownership of getting work accomplished despite nobody directly keeping an eye on them.
meritt··on So you wanna buy a used IP address block?
https://aws.amazon.com/ec2/pricing/on-demand/#Elastic_IP_Add...

Pricing is about the same and it's guaranteed to you, not to a instance that may get shut down

meritt··on Modern Data Lakes Overview
Fairly new to this topic and coming from a traditional RDBMS background. How do you go about deciding how many rows/records to store per object? And how does Athena/Bigquery know which objects to query? Do people use partitioning methods (e.g. by time or customer ID etc) to reduce the need to scan the entire corpus every time you run a query?
meritt··on Fake Goods Sold by AliExpress, Amazon, eBay, Walmart and Wish
Amazon ships products from their warehouses, in their packaging, charges credit cards as "Amazon.com", provides customer support, and accepts product returns. This makes it generally indistinguishable to the consumer if they're purchasing from Amazon or a sketchy third party vendor, as simply the "best" deal gets populated into the buy box. Unless a consumer drills down into the full offer list, the specific 3rd party merchant and the corresponding ratings are completely hidden.

That's significantly different from someone running their own ecommerce site where they are very clearly the merchant of record.

meritt··on Building a static serverless website using S3 and CloudFront
Static site generation is just a trendy reinvention of caching. Put a CDN in front of your site, or run one of the many wordpress cache plugins [1] and point your httpd at the cache directory instead of PHP.

[1] https://wordpress.org/plugins/static-html-output-plugin/

meritt··on Building a static serverless website using S3 and CloudFront
I didn't vote on the comment but it came across to me as emblematic of what I feel is "wrong" with technology trends over the past decade. Where instead of solving business problems it's mostly building tools on top of tools or reinventing the wheel in the latest language/ecosystem. Programmers rarely produce anything of value anymore, but instead parade around their mastery of an increasingly complex toolchain that delivers the same end result.
meritt··on Building a static serverless website using S3 and CloudFront
It's amusing to me how programmers look back with disdain at web development in the 90s and how "you just uploaded files to a FTP server" and now we've circled back to the same thing, just with a whole lot more plumbing in the middle.
meritt··on Scaling Bitbucket’s Database
I'm also salty. What's your plan? I'm either gonna migrate to git (and use a different host), or self-host something like hetapod.net
meritt··on Parsoid in PHP, or There and Back Again
> I'm actually curious why PHP was chosen

From the article: "However, by 2015, as VisualEditor and Parsoid matured and became established, maintaining two parallel wikitext engines in perpetuity was untenable"

They didn't write it in PHP for speed, that was merely a side effect. They wrote it in PHP so they could have a single language for the system.

meritt··on Hard Drive Stats for 2019
We backup data that's not on S3 outside of AWS (code, operational databases), but most of our S3 data is effectively stuck due to the insane export prices. It's not the end of the world if we were to lose everything in S3 anyway.

To anyone reading this: Don't store lots of small files on S3. It's a terrible idea.

meritt··on Hard Drive Stats for 2019
Because the rest of my infrastructure currently runs on AWS and aws egress charges are far more expensive than the b2 savings.
meritt··on Modern cloud architecture on AWS: server fleets and databases
RDS (postgres, mysql, maria) is basically just a managed ec2 instance. The instance cost is about ~2x the comparable price for an ec2 instance, which seems reasonable to me. Storage costs are pretty comparable to EBS. You can do push-button upgrades to increase the capacity, but it's slow. The main benefits I think you get over self-managed is automated / on-demand backups, pretty seamless software upgrades/patches, you can quickly spin-up a duplicate instance from a snapshot for testing/distributing workloads/etc, and replication works pretty seamlessly.

Provisioned IOPS is one area that can get expensive very quick, but people often don't realize that you get 3 PIOPS included with every 1GB of allocated storage, so you really don't need to pay for provisioned IO if you have a decent amount of storage.

If you want auto-scaling you need to look at Aurora or Redshift, which are quite different and significantly more expensive. I've not used those.

meritt··on Peerd: an AWS VPC peering connection management tool
I still use bitbucket because I also use mercurial, but they've deprecated that and will only offer git beginning later this year.

So, no, I don't know why anyone would use bitbucket at this point.

meritt··on Amazon soars after earnings beat
> Wow those margins are shockingly thin

That's by design. Amazon has found it's far more effective to deploy their excess capital toward company expenditures in the form of acquisitions, real estate, capex, employees, stock buybacks, etc rather than letting it get taxed as corporate operating income.

Those dollars do ultimately get taxed, to be clear, but in the form of employment taxes, individual capital gains, individual income taxes, and property taxes.

meritt··on Why We Terminated Our Partnership with Microsoft
> I wonder what can you do to stop a big company from devouring your share in the SaaS market

Make a product that seamlessly integrates with the rest of their ecosystem so they're more likely to acquire you than replace you.

meritt··on PHP in 2020
> to handle the same number of requests/sec you need much more workers, which has a lot of overhead

When doing multiprocessing, it's advisable to fork() instead of creating fresh processes (or launching them in containers etc). As long as you preload the necessary libraries/modules in the parent process before the fork, children will reference the same memory in a copy-on-write fashion. It keeps overhead extremely low.

This [1] article is for the Unicorn (ruby) webserver but the way it explains forking, signals, and pipes for inter-process control it's absolutely illuminating. These techniques are applicable to any unix-based applications, including php daemons.

[1] https://thorstenball.com/blog/2014/11/20/unicorn-unix-magic-...

meritt··on Effectively Using Materialized Views in Ruby on Rails
Yup, exactly. Kimball's stuff is the best. You can achieve on a modest machine what modern techniques would require an incredibly expensive horizontally scaled MPP database. It does require a lot more planning and forethought to be certain.

https://www.amazon.com/dp/product/1118530802 and possibly https://www.amazon.com/dp/0764567578/

meritt··on Effectively Using Materialized Views in Ruby on Rails
Likely too boring and old school data warehousing, but this is what the notion of rollup/aggregate tables achieve. You define a table that's the result of some sort of an aggregation query (time is almost always involved: minutely, hourly, daily, monthly plus other business-specific dimensions) and at the conclusion of each corresponding interval you insert the next respective batch.

    e.g. insert into rollup_clicks_hourly select hour, count(*) clicks from clicks where created_at >= ? and created_at < ? group by 1
Then from a reporting perspective, users/dashboards/etc read from the highest relevant table that meets their criteria.
meritt··on Atrium lays off lawyers, pivots to tools
> how they got $75M

Justin Kan sold Twitch to Amazon for nearly $1B after 7 years, Socialcam for $60M in 18 months, and Exec for "under $10M" in 2 years.

That's how he raised $75M. The company/product was completely irrelevant.

meritt··on I almost sold Baremetrics for $5M
Sure, Amazon does it all the time to small startups. They fly you into Seattle, setup a fancy meeting with strategy teams and M&A, make you run through a pitch deck, explain every aspect of your business. They take diligent notes until they fully understand your inner workings, they tell you thanks, you'll hear from us soon.

And nothing happens, 6 months later AWS launches your exact product.

Founders need to be extremely careful when talking to potential acquirers. Dollar signs cloud your vision and you need to understand why they're talking to you. It may be genuine interest or it may be deceit in order to gain a competitive edge.

← PreviousPage 5 of 34Next →