Hacker News Reader

Comment by optimiz3 | Hacker News Reader

Detection of false positives: Run candidate solutions through the CPU

Detection of false negatives: Compare solution distribution and frequency to expected models; switch to debug kernels if outside tolerance.

However, this works because the mining problem space is stateless and follows strict mathematically predictable models.

A DB is stateful and the answers generally can't be verified without consulting a secondary copy, which is why I'm super curious how they would engineer correctness and reliability in a cost-effective way using GPUs.

View on HN

That's very interesting. Have you collected statistical data on these bit errors? Is it always a single bit error?

I'm assuming you are using GeForce cards and not Tesla cards which have an ECC memory protection mode?

I've tried to collect some statistics on GPU memory errors rates but have found them to be normally extremely rare. The only time I've reproducibly seen them is due to faulty hardware, where the errors become highly reproducible and the GPU needs replacement. The other theoretical cause of bit flips is supposed to be random errors due to cosmic radiation but I've never been able to observe that using memory testing software (though I did only run the experiments in AWS).

Could it be that you have faulty or low grade GPUs? I assume these are all low-cost OEM parts, given your application? Or maybe there's something odd about your data center environment?

Regarding the GPU database application, I think the answer is to just use the Tesla grade GPU with ECC memory enabled.

Generally we prefer AMD cards as most (profitable) mining functions are memory bandwidth dominated. Usually it's a shader unit that gets unstable in the 70-80C range (note that most silicon is rated for higher ranges).

AMD's hardware specifications are more open too which lets one build your own shader compilers and get direct access to the iron.

We've been working on an interposing library for guaranteeing GPU computation and it would be great to get your feedback. Any chance we sync up? My email is in my profile.

Reply on news.ycombinator.com