HNHacker News
TopNewBestAskShowJobs

runningdogx

735 karma · joined April 7, 2009

submissionscomments
runningdogx··on 9 million hits/day with 120 megs RAM
It's not just about steady state memory footprint. It's about the whole stack, how apache and mod_php and mysql (if you use it) interact.

Excessive traffic means a spike in simultaneously served connections, which means a spike in apache threads (assuming worker MPM, iirc there's 1 process per 25 threads by default). With the mod_php model, the per-thread php memory usage can be very expensive when you have dozens or hundreds of apache threads serving requests. A spike in running php instances leads to a spike in mysql connections for a typical web app. If you haven't tuned mysql carefully, which most typical environments have not, mysql memory usage will also skyrocket.

Then for the coup de grace you get stupid apps which think it's perfectly fine to issue long-running queries occasionally (occasionally meaning something like .1% to a few percent of page loads). When that happens, if you're using myisam tables which were the default with mysql < 5.5 (and which therefore dominate deployments, even if "everyone knows" you're supposed to be using innodb), then those infrequent long-running queries block the mysql thread queue, leading to an often catastrophically severe mysql thread backlog. Since php threads are issuing those queries, the apache+mod_php threads stack up as well, and they do not use trivial amounts of memory.

The result is that you have to severely over-engineer the machine with excess memory if you want to survive large traffic spikes. If you don't, you can easily hit swap which will kill your site temporarily, or worse, run out of swap too and have the oomkiller kill something... either your webserver or mysql.

The benefit to fastcgi is it takes the memory allocation of php out of apache's hands, so every new apache thread is more limited in how much bloat it adds to the system. With a limited pool of fastcgi processes, you can also limit the number of db connections which further improves the worst-case memory usage scenario.

The advantage of in-apache-process php is that it serves php faster when there are few parallel requests, but it's on the order of single-digit milliseconds difference (the extra overhead of sending requests through a fastcgi socket), which is dwarfed by network rtt times even if none of the above pathologies rear their heads.

The apache+mod_php model is to do php processing for all active connections in parallel. The fastcgi model is to do php processing for at most x connections where x is the php fastcgi pool size, leaving all other requests to wait for an open slot. It may intuitively seem like the fastcgi model is going to be slower because some requests have to wait for a fastcgi process to become free, but if you think about average php request time it's going to be better for high parallelism, because the limiting factors are cpu and i/o. The apache model ends up using ridiculous amounts of resources just so no php request has to wait to begin getting processed by php. The high contention particularly for i/o created by apache and mysql when they have large numbers of threads is what makes the fastcgi model superior.

runningdogx··on Nym Wars (Google+ "real name" rules)

  site:twitter.com steve wozniak
Not all famous people will have a twitter account, but if they do, it'll probably be in the top few results.

Those people who want to use pseudonyms, and have less visibility, can do that too.

I do not think a social networking site itself needs to distinguish between famous people, ordinary people, and pseudonyms. Link popularity and user-generated status within the social networking site will do that, and search is pretty good at picking up on those cues.

If anything, perhaps G+ and twitter need [better] tools that enable the userbase to ensure that a "real" Steve Jobs account can be given more status (let's call it whuffie), but Google and Twitter themselves do not need to take it upon themselves to unilaterally declare one account or another as special or reserved. That way madness lies. It will stress out the companies who have to pay employees to vet accounts. It will stress out the companies when they get it wrong or get into disputes over well-known identities. It will stress out users when they see network effects being overridden by corporate administrative fiat.

runningdogx··on The Legislation That Could Kill Internet Privacy for Good
The article starts off on the wrong foot.

"Every right-thinking person abhors child pornography."

Wrong. The way child porn is defined, I do not necessarily abhor it. One 17-year-old in a legal relationship [in case anyone out there thinks age of consent is 18 everywhere, google age of consent; you're in for a shock], taking nude photos of him/herself and sending them to his/her partner, is classified as "child porn" under current federal law.

The average person, hearing "child porn", conjures images of minors being abducted and abused in some basement, or being abused by relatives. If only child porn laws were tailored that narrowly.

(I completely agree with the main premise of the article, but this bill is hardly a new phenomenon. Pedophilia (technically pre-pubescent minors), ephebophilia (post-pubescent minors) and/or child porn has been one of the four horsemen of the infocalypse[1] for many years.)

[1] http://en.wikipedia.org/wiki/Four_Horsemen_of_the_Infocalyps...

runningdogx··on What are good keyboards for programming?
The MS X6 lets you move the numpad to the other side of the keyboard (or possibly leave it detached, but for mouse access those are equivalent options). I don't have one, and it's reviewed less favorably than the X4 which doesn't have the removable keypad, but it's something.
runningdogx··on Why I Hate Science
"It's not unusual in school to hear this religion of science put another way, science is the beautiful application of reason and logic that slowly and inexorably moves from ignorance to truth.

"Bullshit."

At this point, I stopped taking you seriously.

No. That is exactly what science is: an ongoing dialogue between scientists, using experiments to collect data to validate (improve certainty of) or disprove theories, theories which over time paint increasingly accurate pictures of how the thing we know as reality really works.

The garbage claims you ascribe to scientists are usually not made by scientists; instead, the claims usually come from the media who either don't understand the underlying science, or who do understand it but are sensationalizing the scientific result in an effort to get more people interested.

A problem only arises when Joe Sixpacks (who not only dominate society but Congress as well), who are completely ignorant of science, end up using media reporting of science to make decisions.

runningdogx··on Sony's Captcha. View source to see why they don't "get" security
There are two objectives for captchas:

1. Prevent prefab'd spam bots from being able to spam your site. Almost anything, including trivial questions and Sony's "captcha", works for this. Some bots will randomly try values for unknown fields, and note if a value apparently succeeds where others failed; against such bots, questions like "2 + 2 = " or "type 'orange':" are bad. A decade ago, people were writing irc bots that could answer most trivial captcha questions. A question like "Who was the U.S. President in 1993-2001" has a better chance of going uncracked.

2. Prevent a targetted spam bot from being able to submit arbitrarily many forms on your site. "orange", "3 + 5 =", 31337e^(-pii), and sony's captcha will not work for this. If there's a question pool, someone will solve most or all of them and program the bot to respond correctly, most of the time / all the time.

The idea that someone would only ever want to do (1) is flawed. If someone's bot is flexible enough, and your question pool is shallow enough, it may be worth their time to code in responses to your custom questions even if your site caters to a relatively small audience.

runningdogx··on Don't Let Congress Order Internet Companies to Spy on You
Did you ask the router provider to cite the law requiring them to keep logs?
runningdogx··on Aaronsw indicted for hacking MIT network to download millions of JSTOR docs
This is the most technically competent charging document I've ever read. I guess there must have been some hackers on the grand jury.

Paragraph 35 & 36: which "protected computer" on MIT's network did he access? Certainly they're not trying to claim his laptop was a protected computer? Are they talking about the DHCP server or whatever registration frontend MIT has for the DHCP assignments? I have trouble with the concept that a violation of a computer use agreement (when there are no operative security barriers in place) constitutes a violation of the computer fraud and abuse act. Then again, I've always thought that act was vague and therefore overbroad.

Obviously what he did was bad in some sense (at least from the perspective of JSTOR and MIT), but even if it should be a crime rather than a civil dispute or internal disciplinary action at MIT, I don't like the fact that just about any misbehavior on the internet becomes a federal case because the probability of no interstate resources being used is very low.

Finally, I take issue with the notion that someone who is accessing a service through a public interface is criminally responsible for downtime if too high an access rate causes service degradation or an outage. The claims that JSTOR's servers were overloaded and (one?) even went down at some point are clearly there to set up a later claim of damages. Haven't they heard of rate limiting (in this case, since it was a rogue laptop stashed in a data closet, rate limiting by IP)? That wouldn't work against a concerted denial of service attack, but this was no denial of service attack. JSTOR seems to have been relying on manual intervention to stop article leeching that could lead to a (partial) outage. That's naive, and not a good idea.

runningdogx··on Why I will never pursue cheating again
What about shifting to a teaching model where students review material and watch lectures (MIT OCW, Kahn, or even the professor's own recorded lectures) out of class, and do actual assignments (problem sets, essays) in class -- in an atmosphere that encourages working together so nobody feels "stuck" or gets stressed to the point where they feel like they have to cheat, while still preparing solutions separately? That seems to be direction favored by many of the advocates of reform in higher learning... advocates like Dr. Tae.
runningdogx··on What Google's Famous Cafeterias Can Teach Us About Health
Indeed. Quaker chewy granola is identified (by price -- $0.15) as a good food, which may be true if you're out hiking, but as part of an urban lifestyle, granola bars are a glycemic disaster.

By contrast, someone who eats an otherwise moderately healthy diet can eat the caloric equivalent of several granola bars worth of walnuts -- identified by google as a "yellow" food -- with no ill effects, and certainly no insulin spike.

runningdogx··on My wife's and my holiday weekend project: A database of intro books
I object to the site's notion that there is a "best" intro book in many areas, but I'd like to see listed on the site:

Griffiths' _Introduction to Quantum Mechanics_ is quite commonly used, even though it's a bit calculation-heavy. Beyond that or for more theory, perhaps Ballentine, or Dirac's original monograph, although neither is a very common text (if that's what someone's looking for, then Shankar [mid-level-ish], Cohen-Tannoudji, or Sakurai); Ballentine because it's new (relatively... 1998), and Dirac because it's old and it's what a lot of the mid-to-late 20th-century physics giants started from. Feynman's Quantum Mechanics and Path Integrals is also a no-brainer because it's been republished and it's now a $12 Dover edition.

runningdogx··on TrueCrypt User Held in Contempt of Court
What if your passphrase to your truecrypt container (let's say it contains, for the purpose of this thought experiment, child pornography) is the true statement: "I am [for this thought experiment] a child pornography collector."

Wouldn't it be a violation of the 5th amendment to be compelled to provide that passphrase, because it is an admission against interest and therefore would be admissible if you disclosed it? Wouldn't it also serve to waive 5th amendment privilege, and possibly put you at risk of being forced to take the stand?

If it were an admission of a different crime, a court could grant you immunity on those unrelated charges, but if it is relevant to the crime the government is investigating by asking you to reveal your passphrase... how can anyone, luddite judge or not, separate "key" from "testimony" in that circumstance?

runningdogx··on TrueCrypt User Held in Contempt of Court
The truth you can tell is only a fraction of a larger truth encompassing the entire universe of facts for a case the government may be investigating. Even if your portion of that truth is completely free of any wrongdoing, when combined with the rest of the facts that may include lies or mistakes by other witnesses or incorrect analysis of physical evidence, you can end up getting into trouble for telling the truth.

But we have never held, as the Supreme Court of Ohio did, that the privilege is unavailable to those who claim innocence. To the contrary, we have emphasized that one of the Fifth Amendment's "basic functions ... is to protect innocent men ... 'who otherwise might be ensnared by ambiguous circumstances.' " Grunewald v. United States, 353 U. S. 391, 421 (1957) (quoting Slochower v. Board of Higher Ed. of New York City, 350 U. S. 551, 557-558 (1956)) (emphasis in original). In Grunewald, we recognized that truthful responses of an innocent witness, as well as those of a wrongdoer, may provide the government with incriminating evidence from the speaker's own mouth. 353 U. S., at 421-422.

-- Supreme Court in Ohio v Matthew Reiner, http://caselaw.lp.findlaw.com/cgi-bin/getcase.pl?court=US...

runningdogx··on TrueCrypt User Held in Contempt of Court
The legal apparatus you identify is based on the 5th amendment, interpreted through the lens of mythological, historical ideas of how "thought" is different from information recorded external to the brain.

If you think things are confusing now, wait until merely sticking electrodes on someone's head and making them think about certain things by talking to them, without requiring any voluntary response, can generate usable evidence.

Requiring someone to reveal a passphrase or access procedure to reveal encrypted or otherwise secured evidence, external to the brain, is the tip of the iceberg.

runningdogx··on Verizon to end unlimited data option
The article is wrong about t-mobile. Quoting directly from their site (shop->plans->mobile broadband) (and note how all of them are described as "overage free")

"No overages! (After 10 GB, data speeds are slowed until the next billing cycle)"

e.g. http://www.t-mobile.com/shop/plans/cell-phone-plans-detail.a...

So there is not unlimited access at the highest network speeds, but the plan is nevertheless unlimited data -- as in you don't have to watch your consumption to avoid getting dinged with $$$/GB charges.

runningdogx··on US Senators ignore unintended consequences of criminalizing embedding videos
I believe you got snookered by the aide. The crime is conditioned on $2500 of income to the infringer, _or_ $2500 loss to the copyright holder, _or_ a $5000 or more licensing fee for said content.

It is not clear, if those damage requirements are met, that there is any requirement -- other than non-zero economic advantage from the infringement, as required by 506 (a)(1)(A) -- that the infringer actually made lots of money as a result of the instance of copyright infringement.

It is also unclear what "total retail value", "total economic value", and "fair market value" (all of which are used in the bill) mean in the context of a form of property that is given artificial monopoly status. If someone uploads an unreleased Justin Bieber video to youtube and it gets 1 hit, then Bieber releases that video in a limited collector's edition at $2600 USD, is that youtube uploader a felon?

The text of the bill is very short: http://thomas.loc.gov/cgi-bin/query/z?c112:S.978: (include the last ':') And 18 USC 2319 which it affects is also very short: http://www.law.cornell.edu/uscode/18/2319.html

runningdogx··on US Senators ignore unintended consequences of criminalizing embedding videos
More churn in Congress would decrease the value of any one congressperson to private industry, and would also reduce the tendency of members of Congress to amass large numbers of markers that they call in later.

It would also change the entire dynamic of Congress, to be less of a club and more of a transitional job, people coming from some other job -- some of them perhaps even hoping to do the right thing regardless of political fallout -- and then disappearing from the public stage and moving on to another stage their lives, rather than turning elected service into their lives.

A seminal paper on this subject from 1968, The Institutionalization of the U.S. House of Representatives, by Nelson Polsby: http://crespin.myweb.uga.edu/polsby.pdf

runningdogx··on Chrome to block insecure resources by default
The new Chrome hack forces embeddable content hosting sites (e.g. imageshack) to choose between supporting https, or giving up the analytics data they gain from being embedded on so many pages. It also forces sites with embedded content to implement hacks, converting img and other embed tag uris specified as http into https uris. But if the underlying page is loaded over https, obviously you either want to try getting the resource over https or not at all. What other sane thing is there for a web app to do other than convert all such uris from http to https when serving a page over https? Many uris will break that way, but it's better than not serving any of them. And, if that's the only sane thing for a webapp to do in that situation, why not put that logic into the web browser? As I mentioned in reply to gst, browsers supporting HSTS already have several modes of "force https for a uri specified as http". Adding one more hardly seems like an unacceptably ugly hack.

Regardless, embedded content sites that would be swamped if all requests were over https, and sites that map http://foo and https://foo differently, are going to have to figure out solutions, or they'll stop being embedded as more sites enable https. Even under the status quo, mixed content warnings and symbols are a little bit scary and sites wishing to go https-enabled or https-only will encourage members to use embeddable content hosting services that support https.

runningdogx··on Chrome to block insecure resources by default
Is it more of an ugly hack than forcing arbitrary sites to https based on a list that's partially hard-coded, partially user-specified, and partially added through an HSTS header in which case the entry also has a limited lifespan?

Ugly hacks to make the web even marginally more safe by encouraging or forcing use of https (in places it isn't explicitly specified) may be worthwhile. The thread's chrome hack alone reduces the usability of https, to the point of (I would guess) encouraging sites to use http and disable https so that visitors can actually see embedded content. Average web users know nothing of https, but if they happen across a site using https and embedded content doesn't show up, they'll blame the site. Certainly _that_ is not good. If this google hack is truly worth the pain, and even a small fraction of embedded content works with the uri changed from http to https, that's a benefit because it makes https more usable.

Would you think the same about simply changing the HSTS header so that sites could tell a browser to force loading all embedded http:// content through https://?

It seems to me like Google's long-term goal is to encourage tls everywhere. If all sites that have user accounts (thus logins) are using https, and all content embedded on those sites has to be referenced over https (or another secure protocol), that's probably the bulk of the web.

So the long term costs (in the far future) are an irrelevant cruft of a hack that ends up doing nothing because there are no http:// resource links anywhere. A hack which could, at that point, be removed with no impact whatsoever.

runningdogx··on Chrome to block insecure resources by default
In an https page, why not rewrite http resources to https by default? Then it could omit the content if there is a cert error or if there is no ssl server there at all.

I'm thinking of a use case of a typical not-ssl-mandatory online forum/wiki/whatever. There are going to be tons of historical img and script tags, either to local content or to other sites, that do not use https even when they could; when those links were set up years ago very few sites (other than ecommerce) took ssl seriously. The solutions are going through and changing links in the database, rewriting links in the app, or rewriting links in the web browser. Is there a compelling reason why rewriting links in the web browser is bad? There are five web browsers that people care about: Chrome, FF, IE, Safari, Opera (mostly mobile). There are thousands of webapps and at least tens of thousands of moderately popular sites that would have to rewrite content in the database or webapp in order to go ssl-only.

runningdogx··on How Hackers Stole 200,000 Citi Accounts Just By Changing Numbers In The URL
Sadly, other financial services companies do not seem to understand, either.

Etrade's password system requires 6 to 32 characters with at least one number according to the password change instructions. They don't mention punctuation, but if you try to include some, they're deemed to be "invalid characters". Go figure.

This stuff makes me sad that I signed up for etrade:

"Thank you for your message regarding enabling HTTP Strict Transport Security. I have sent a request to our Product Development Team to have this feature added. Due to the high volume of requests, there is no guarantee that this will be implemented." -- etrade representative

"ETRADE does not allow certain characters to be used when establishing a password online. There is no specific ETRADE publication providing information on why certain characters are not allowed, such as special characters used for punctuation, etc. We appreciate your feedback concerning the online passwords. I forwarded this suggestion to the Product Development Team for future consideration and implementation. I can not guarantee when or if this change will be able to be made..." -- another etrade representative

runningdogx··on A molecule that switches on and off - ultra-dense digital memory & nanomotors?
Using molecule structure to store bits is a lot harder than it might seem.

For instance, I distinctly recall, in the early 90's, reading about some organic molecule that changed structure when hit with particular wavelengths of light. It was in one of the popular scientific magazines at the time. The theory was that you could construct a lattice and have lasers on each axis that could change state of arbitrary molecules in the lattice. (If I remember correctly it was a three state change molecule, or at least there were three wavelengths that did different things to the molecule depending on its state.)

Obviously that doesn't work so well, because using a laser to target only a single molecule in a lattice is hard, and because of statistical thermodynamics some bits are going to get corrupted so you'd need error correction and periodic rescans of the entire data array to fix corrupted bits. Nevertheless, they were speculating it would see commercialization in 4-8 years. Well, here we are approaching 2 decades later. I see no organic molecule storage on the market.

Finding a molecule that can change state is nothing new. Getting it into an array or lattice where it's practical for dense, long-term data storage is the tricky part.

runningdogx··on Reflections on Trusting Trust (Ken Thompson's Turing Award speech)
What about trusted hardware? Without that, your somewhat trusted compiler cannot be trusted, even if it's something like tcc whose code you might realistically be able to formally verify by yourself.

There are at least two approaches, but neither are really satisfactory... first, using an electron microscope and validating the finished design gate by gate. That won't work if the software and hardware toolchain used to generate the gate design is compromised. So you'd have to validate the gates without the help of the software toolchain used for design and layout.

I suppose you could also attempt to validate silicon (sans microcode) by attaching a clever external apparatus and external clock source and current meter (which you trust?), and timing every single instruction that hits the processor, validating the time it takes for the instruction to return results and the power consumed, and comparing that against the design specs. That suffers from the same general problem though: unless you're deriving timing and power consumption from basic principles, the tools used to generate expected timings and power consumption could have been compromised.

For processors with few gates, though, it could work.

The worst situations are where the malware is so subtle -- one changed gate or instruction for instance, with a specific application in mind that can be subverted through that change -- or where from an external POV the malware is non-deterministic, for instance if a processor randomly and rarely injects malware into a running system, using a hardware rng for randomness, on average once every million years of cpu time. Would chip makers be able to detect those sorts of attacks if they tried?

runningdogx··on How to replace 30 laptops (and $10,000) with 150 sheets of paper
The alarms would start going off again when empty boxes reached the precision scales. Behold, an $8mil broken-fan detector.
runningdogx··on Supreme Court rules against Microsoft in major patent case
So non-obvious (by the PTO's standards, or even by some wise person's standards) mathematics should be patentable?
runningdogx··on Newegg stores your password in plaintext
If you're implying that Newegg or any public-facing authentication system could be implemented with a non-exposed password vault, then I don't see how.

If user passwords are encrypted with symmetric encryption, either the application must have access to the key in order to compare your provided password to the password stored in the DB, or the DB has to have a stored procedure with access to the key.

Therefore it's not -only- customer support that has access, but anyone who hacks the production system also gains access to the key. Of course there are ways to obscure that if it's kept in memory and required as input before the server starts up, but theoretically it's available to anyone who compromises the relevant server.

(And even if there were a system that were implemented as you describe, with access allowed only when a support staffer and a supervisor provides an asymmetric key each, who supervises the supervisors?)

(edit: The offered suggestions show that I wasn't considering non-symmetric or non-software approaches; in any case, at best those seem like complicated band-aids for a situation that should not exist.)

runningdogx··on Newegg stores your password in plaintext
Passwords that can be decrypted reliably are isomorphic to passwords stored in plaintext. That is very undesirable. Even assuming they are encrypted with a strong key, anyone with access to the decryption key and the database can trivially recover any password stored by such a system.

Password hashes are not isomorphic to plaintext passwords, because there can be collisions. That means there is no function to reliably take the stored passwords and turn them back into plaintext passwords; therefore, while reversing some/most passwords through complex iterative functions (e.g. brute force) is possible, the amount of work (particularly for bcrypt, pbkdf2, or other many-round hash+salt schemes) on average is many, many orders of magnitude greater than applying a decryption key. For good passwords, the amount of work is not quite unbounded, but might as well be.

It's true that plaintext passwords and singly-hashed md5 passwords are both quite bad, but at least with the md5 hash nobody can (I think?) currently crack a 32 character random string since there are no rainbow tables for such long passwords. (Assuming the password is vetted for common weaknesses, since "random" alone does not guarantee anything. "aaaaaaaa" is a random 8 char password, after all.) Functions that attempt to reverse non-isomorphic mappings end up being complex and, even in the case of rainbow tables for (unsalted) md5, are much slower than not having to recover the unencrypted password at all, or having some master key that decrypts all passwords.

runningdogx··on Linux kernel 2.6.39 released
commit 442b9635, increasing the initial tcp window (initcwnd) to 10(segments), first landed somewhere around 2.6.39-rc2. I consider that a significant thing for servers. The initial receive window was increased first, several months ago. (I think the receive window change was in .37 but I don't know for sure.)
runningdogx··on Dropbox Lied to Users about Data Security, Complaint to FTC Alleges
That would prevent de-duplication in the non-shared case, which as I understand it is important to Dropbox for increasing storage efficiency. I suspect that is the primary reason why they don't have an option to "store encryption key locally only" in the client. The excuse that users expect to be able to share and access files on the web can be overcome with a big scary warning indicating that is not possible when using client-side-only encryption keys.

Dropbox advocates TrueCrypt in one breath, but refuses to integrate client-only encryption keys in the client with the next breath. Obviously they know what we all know: TrueCrypt presents a poor UX for non-technical users, and so most people won't use it even if it's recommended. Then DropBox gets to be the hero for advocating TrueCrypt while they get de-dup efficiency because they know few people actually bother using TrueCrypt.

Any transfer of keys to dropbox, even temporally, means your data on dropbox should be considered insecure. You have no way to know what's going on on the dropbox side. The same issue arises for CAs that let customers generate SSL keys within a web interface, to avoid the nuisance of having to generate a key/cert/csr themselves and uploading that. It doesn't matter if the CA promises never to store the private key. It's insecure and it's bad practice.

runningdogx··on An Argument Against JavaScript Minification
Minification still offers a substantial savings. The following file size stats are from jquery 1.6.0 just now pulled from google's urls:

Uncompressed (normal:minified) 232651:90518

Gzip -6 (normal:minified) 66778:31610

Gzip -1 (normal:minified): 81986:36633

gzip -9 only saves a few hundred bytes on each vs default compression level of 6.

bzip2 and xz do not achieve dramatic reductions of the readable jquery code either; the normal:minified ratio is slightly improved, but is still approx. 2:1.

← PreviousPage 2 of 5Next →