HNHacker News
TopNewBestAskShowJobs

dhd415

2,236 karma · joined March 7, 2015

email: hackernews.dah328 [at] 0sg [dot] net
submissionscomments
dhd415··on Making SQL Server run on Linux
Cassandra really isn't a columnstore in the same sense that the word is used for OLAP systems. In the latter, efficient queries can be run across lots of values in a small number of columns across many rows. In Cassandra, queries spanning many rows are not particularly efficient. Sometimes wide-column tables are used for OLAP-type use cases in Cassandra, but that's something of a different approach. More info on that here: http://dbmsmusings.blogspot.com/2010/03/distinguishing-two-m...
dhd415··on SQL Server on Linux public preview
While the licensing details can be somewhat complicated, the retail price of ~$7k/core is published on Microsoft's site at https://www.microsoft.com/en-us/sql-server/sql-server-2016-p.... Oracle's pricing is harder to come by but is closer to $25k/core for the Enterprise edition on x86 CPUs. There are often lots of extras involved in Oracle licensing, too, so the total cost differential between the two is generally higher than the per-core license cost differential.
dhd415··on Implement table partitioning
No, for example, a composite index on (tenant, date) would be highly non-selective whereas an index on date in each individual tenant partition is highly selective and therefore higher-performance (in my case, much higher performance).
dhd415··on Implement table partitioning
There are other possibilities as well. For example, if your partitioning strategy is such that it improves the selectivity of an index, it could improve query plans for queries that were on an index that was less selective. As an example, I once had a table with over a billion rows distributed among ~100 tenants on which queries were typically run by tenant and date range. Partitioning that table by tenant dramatically improved the performance of those queries because those queries no longer had to scan through rows of which only ~1% were for the tenant of interest.
dhd415··on Amazon Aurora Update – PostgreSQL Compatibility
For many installations, an Aurora read replica can eliminate the need for a multi-AZ master instance which cuts the cost of your master instance in half.
dhd415··on Amazon Aurora Update – PostgreSQL Compatibility
I suspect you're right. I wonder if that's part of the reason why they claim a lower performance improvement of 2x over "standard" pgsql installations rather than the 5x improvement of Aurora/MySQL over standard MySQL installations.

If so, though, hopefully there's a silver lining of faster incorporation of new pgsql features into the Aurora/PG service.

dhd415··on Amazon Aurora Update – PostgreSQL Compatibility
For Aurora, you mean? They support 5.7 for RDS, but a consequence of forking MySQL for Aurora is that it's unlikely that new MySQL features will quickly be incorporated into Aurora/MySQL.
dhd415··on Amazon Aurora Update – PostgreSQL Compatibility
I'm curious how much of the underlying pgsql storage engine was rewritten since Aurora/MySQL uses a completely different engine than the default InnoDB.
dhd415··on Cassandra is not row level consistent
Some SQL databases (e.g., SQL Server and PostgreSQL) offer key-range locking which, along with a serializable isolation level, will prevent this race condition.
dhd415··on Microsoft announces the next version SQL Server for Windows and Linux
They haven't made all of the Enterprise Edition features available on all lower editions, but they have made many big ones such as table partitioning, Always Encrypted, in-memory OLTP, etc. available in most lower editions. There are a number of operational features such as piecemeal restores, Resource Governor, online index rebuilds, etc. that are still exclusive to Enterprise Edition.

There's a more complete discussion of these here: https://sqlperformance.com/2016/11/sql-server-2016/big-deal-...

dhd415··on SQL Server 2016 SP1 Released, SQL Server VNext Available for Download
Some of the huge news here is that as of SP1, the following features that were one exclusive to the Enterprise Edition of SQL Server are now available in Standard and Web Editions: Change Data Capture, database snapshots, column stores, table partitioning, data compression, in-memory OLTP (Hekaton), Always Encrypted, Polybase, fine-grained auditing, multile Filestream containers. I wish I had access to all of these features back when I was building an app that was required to support SQL Server 2012 Standard Edition.
dhd415··on Microsoft announces the next version SQL Server for Windows and Linux
I love pgsql and it has some features that SQL Server does not, but SQL Server also has a range of features that pgsql does not including index-organized tables, automatically-updated materialized views, much more mature support for query parallelization, etc. Lots of the discussion in this older thread is still relevant: https://news.ycombinator.com/item?id=9464505

Edit: fixed link.

dhd415··on Microsoft announces the next version SQL Server for Windows and Linux
Lots of previously Enterprise Edition-only features such as table partitioning and data compression are now available in Standard, Web, and local editions. This is a pretty big deal. More detail here: https://sqlperformance.com/2016/11/sql-server-2016/big-deal-...
dhd415··on Amazon RDS now supports PostgreSQL 9.6.1
Interesting, I've never heard of striping tricks like that. Do you have any links on that? And I suppose that could be applicable to EBS since it rather than S3 is the backing store for RDS.
dhd415··on Amazon RDS now supports PostgreSQL 9.6.1
I'm using RDS MySQL and have never been able to utilize more than ~25% of the provisioned IOPS or more than 100MB/s transfer rate. Also, as best as I can tell, EBS latency is pretty high so making lots of small, serial database calls is a lot slower than it would be on local disks.
dhd415··on Some thoughts on asynchronous Python API design in a post-async/await world
I'm surprised there aren't more mentions of Tasks in C# or F# on the .NET platform as examples of asynchronicity done well.

From the perspective of uniformity and availability, while C# provided asynchronicity via callbacks before the introduction of Tasks in the 4.5 release of the .NET Framework, all the core libraries that used callback-style async (as well as some that had been strictly synchronous-only) were updated with Task-based overloads, so there are no problems with Task-based async being inconsistently available. Additionally, adoption of Task-based async in third-party libraries has been high, so it's relatively uncommon to encounter code that does not support it.

From the perspective of code productivity, it's hard to get much better than simply adding the async and await keywords where necessary. As a very simple example, consider a typical server application that receives requests via HTTP, processes them via an HTTP call to another service as well as a database call, and then returns an HTTP response. The sync code (blocking with a thread-per-request model) might look something like this:

    void handleRequest(HttpRequest request) {
        var serviceResult = makeServiceCallForRequest(request);
        var databaseResult = makeDatabaseCallForRequest(request);
        sendResponse(constructResponse(request, serviceResult, databaseResult));
    }
In order to make that same process async (non-blocking with a dynamically-sized thread pool handling all requests), the code would look like this:

    async Task handleRequestAsync(HttpRequest request) {
        var serviceResult = await makeServiceCallForRequestAsync(request);
        var databaseResult = await makeDatabaseCallForRequestAsync(request);
        await sendResponseAsync(constructResponse(request, serviceResult, databaseResult));
    }
It could even be taken one step further to make the service request and database call concurrently if there were no dependencies between the two which would reduce processing latency for individual requests:

    async Task handleRequestAsync(HttpRequest request) {
        var serviceResultTask = makeServiceCallForRequestAsync(request);
        var databaseResultTask = makeDatabaseCallForRequestAsync(request);
        await sendResponseAsync(constructResponse(request, await serviceResultTask, await databaseResultTask));
    }
I've added asynchronicity into a C# server application as above with substantial improvements in both individual request latency and overall scalability. I'm now working on a Java8 system and bemoaning the comparatively primitive and inconsistent async capabilities in Java8.
dhd415··on Guide to Remote Work
I would agree. One of the things I felt when I first moved to working remotely was a sense of isolation due partly to losing the awareness of what other people or teams were working on that you have when working in an office due to just seeing them meeting or talking together. Sococo gives that back to you because you can see other people and teams meeting in the "virtual conference rooms". There's been more than one instance where I've dropped in on a meeting to assist with a production issue or a development question that came up because I could see those conversations were happening, something is more difficult if a chat tool such as Slack is your primary or sole form of communication.
dhd415··on Ask HN: What laptop should I get instead of a Macbook Pro?
That's along the lines of what I do. I get a used Thinkpad for occasional travel and use it to remote into my beastly dual Xeon workstation that is stupid fast and less than half the cost of a 2015 MacBook Pro.
dhd415··on Debunking Misleading Benchmarks Of Redshift vs BigQuery
+1. They'd have a much stronger case if they didn't drag in silly complaints such as "[Google BQ] does not support the ANSI SQL syntax of 'substring'. Need to change to 'substr'".
dhd415··on Visualizing a Job Search Or: How to Find a Job as a Software Engineer
Having gone through this process recently, there's a lot of truth in this post. I experienced more companies "ghosting" during the process than slow turnaround times, but it is remarkable how short-sighted companies are during the recruiting phase. It's as if common courtesy has gone completely out the window in tech recruiting. Basic things like not missing phone interviews at times that have been previously scheduled or informing candidates politely that you've decided to move forward with other candidates can't even be taken for granted anymore. Candidates' initial experience with a company is typically through the recruiting organization and negative impressions during the recruiting phase are often applied to the company as a whole. I could see it being a competitive advantage to be known for having a good recruiting process even if it is very selective. For example, I have a very positive impression of DigitalOcean despite not receiving an offer because of the professional and courteous way they handled their interview process.
dhd415··on The Stack Overflow Developer Story: A New Technical Resume
It's an interesting idea, but I prefer to take my experience and tell a story that specifically targets the role for which I am applying. From that perspective, having a single "story" out there for all potential employers isn't optimal for me.
dhd415··on Using R to detect fraud at 1M transactions per second [video]
The presentation is a little light on the technical details of how the demo was run. What I could get from the presentation was 1M fraud predictions/sec via R stored procedures on data streaming into SQL Server 2016 stored in in-memory column store tables on a 4-socket "commodity" server.
dhd415··on PostgreSQL 9.6 Released
Hmm, I've never run SQL Server on anything other than NTFS where 64KB was definitely the recommended block size. In either case, it's great to have choices. When the license fee is coming out of my pocket, I'm definitely not choosing SQL Server.
dhd415··on PostgreSQL 9.6 Released
I'm a fan of both PostgreSQL and SQL Server, but I think these numbers are very workload-specific. I've gotten 1GB/s throughput on SQL Server 2012 on spinning disks and CPUs older than the E5620, so I've no doubt that same workload would exceed 2GB/s on your hardware. The apples-to-apples comparison here is between the two versions of PG where the performance improvement is clear. It's harder to do an apples-to-apples comparison between PG and SQL Server because the optimal schema and queries for a particular workload are likely to differ for each of them.
dhd415··on PostgreSQL 9.6 Released
>>In InnoDB all the data is stored in the PK while in PG it is just a pointer.

This is just a consequence of the PK being a clustered index in InnoDB which has both pros and cons. One of the big cons is that all of the columns of the PK are implicitly added to every secondary index as the row identifier. That isn't a big problem if your PK is a single column int, but if it's multiple columns, that often results in unnecessary bloat in your secondary indexes. Ideally (as in, dare I say, MS SQL Server), you'd have the option of a clustered or non-clustered PK for your table so you could choose the optimal index structure for your workload on a per-table basis.

dhd415··on PostgreSQL 9.6 Released
PostgreSQL, for better or worse, doesn't have pluggable storage engines. There's some discussion on their dev mailing list about the possibility of adding that capability in PG10, though: http://postgresql.nabble.com/Pluggable-storage-td5916322.htm...

Some earlier (2013) discussion on the same topic: https://wiki.postgresql.org/wiki/2013UnconfPluggableStorage

dhd415··on Oct. 4
The "compact" phones in the Sony Xperia line are really the only ones that come close. My wife is still happily using her Z3 Compact though its successor, the Z5 Compact, didn't review as well. The newest Xperia X Compact is getting decent reviews.
dhd415··on Old Geek
Getting somewhat far afield of the OP, but in healthcare, the bigger trend seems to be a move from HL7v2 past the dumpster fire that was XML-based HL7v3 to JSON-based FHIR.
dhd415··on Lessons from a 45-year Study of Super-Smart Children
The article is short on actual lessons learned from the study. Two representative paragraphs:

  SMPY researchers say that even modest interventions—for example, access to challenging material such as college-level Advanced Placement courses—have a demonstrable effect. Among students with high ability, those who were given a richer density of advanced precollegiate educational opportunities in STEM went on to publish more academic papers, earn more patents and pursue higher-level careers than their equally smart peers who didn't have these opportunities.

  Despite SMPY's many insights, researchers still have an incomplete picture of giftedness and achievement. “We don't know why, even at the high end, some people will do well and others won't,” says Douglas Detterman, a psychologist who studies cognitive ability at Case Western Reserve University in Cleveland, Ohio. “Intelligence won't account for all the differences between people; motivation, personality factors, how hard you work and other things are important.”
dhd415··on Amazon Aurora: Parallel Read Ahead, Faster Indexing, NUMA Awareness
CitusDB is an option for scaling out PostgreSQL workloads, but it's not a direct equivalent to Aurora. That's both good and bad in different ways. E.g., it's good that CitusDB is not a fork of PostgreSQL as Aurora is a fork of MySQL. It's bad that CitusDB as pure software can't offer the same kind of high-performance, auto-replicated storage layer that Aurora does.
← PreviousPage 9 of 11Next →