797 karma · joined May 29, 2010
Solutions like display:table enforces source order, which to me is pretty undesirable (e.g. when A/B testing layouts by switching a CSS file).
In the mean time, the slightly older Template Layout Module is available as a jQuery plugin that would make an excellent basis for a truly semantic, separate, and A/B test-able grid system: http://code.google.com/p/css-template-layout/
The HTML5 spec is such a mess when it comes to the new "semantic" elements that they are nigh on useless for anything. They're so broad (e.g. check out how broad the definition of "article" is) it's like calling a knife, a spoon, a plate, and table all a "fork" -- too broad to be useful. Div's are fine. They're all ignored equally by machines.
Screen readers also pay zero to no attention to these tags -- heading tags are, however, very much used, but HTML5 intends to supersede that in a completely botched way too. It's not pretty. Also, you would imagine strong and em were useful as opposed to b and i. It turns out most screen readers ignore all of them entirely.
Re the parent comment - as for the usefulness of true separation between presentation and content, I find it very promising for A/B testing layout, and therefore a worthy goal, but CSS really needs an actual layout system.
> And, quite frankly, the first victims of A/B testing are beauty, elegance, charm, and grace.
Right, and the first casualty of a designer's-opinion-only approach is "beautiful" designs that users hate. (Of course, neither premise is necessarily true, this is just a baseless appeal to emotion. Who could be against "beauty, elegance, charm, and grace"?) Can the author demonstrate from, say, ABtests.com or Whichtestwon.com that this is the case? Or did he simply have a bad experiences at highly data-driven companies? Is Netflix, for example, guilty of sacrificing "beauty, elegance, charm, and grace"?
> Instead we get a unsightly pastiche of uneven incrementalism lacking any kind of holistic cohesiveness or suggestive of a bold, vivid, nuanced vision that inspires users.
Such as? Is the author suggesting we go with poor-performing designs because in one or a few people's opinion, they "inspire" users? How do you know users are inspired if their performance suffers?
> It is the implicit charter of a high-quality design team (armed with user researchers and content strategists!) to propose something a user may not be able to imagine, that is significantly better, since they are so conditioned by mediocre design in the mainstream.
IMO, it's implicit that a high quality design team could come up with 2 or more such designs, or design variations, to test. Not only that, that they would come up with such design variations to test that are cohesive, elegant, and beautiful.
> A/B testing may only be as effective as the designs being tested, which may or may not be high quality solutions. Users are not always the best judge of high quality design. That’s why you hire expert designers of seasoned skills, experience, judgment, and yes the conviction to make a call as to what’s better overall.
Make a call based on what? Gut feel? The quality of designs being tested is not a problem of A/B testing -- if you only want users to choose from "high quality design", only test "high quality design". I don't understand the objection here. If a designer is truly an expert with seasoned skills, they should be the first to push for A/B testing of their best ideas, not retreating into vague notions of what they feel is best. There is no "One True Solution". Designers should own and drive the A/B testing process, not feel like they are victims of it.
> As is true with any usability test, you gotta question the motives behind the participants’ answers/reactions. Instead, biz/tech folks look at A/B test results as “the truth” rather than a data point to be debated. Healthy skepticism is alwayswarranted in any testing. Uncovering the rationale for a metric is vital.
People buy or they don't. They click or they go. You can't "question the motives" or hundreds or thousands of users, that doesn't make sense. You can't wish away the data.
> A/B testing is typically used for tightly focused comparisons of granular elements of an interface, resulting in poor pastiches with results drawn from different tests.
Poor A/B testing practice may be a problem, fine, but let's not throw out the baby with the bathwater.
> How do you A/B test novel interaction models, conceptual paradigms, visual styles (by the way, visuals & interactions have a two-way rapport, they inform each other, can’t separate them–see Mike Kruzeniski’s talks) which may vary wildly from before?
Uh, you run an A/B test and measure what you're doing.
> Would you A/B test the Wii or Dyson or Prius or iPhone? Against what???
Category error. We're talking about graphic/interaction/web design. All those companies absolutely should A/B test their web sites, for example.
> A/B testing locks you into just two comparative options, an exclusively binary (and thus limited) way of thinking. What about C or D or Z or some other alternatives? What if there are elements of A & B that could blend together to form another option? Avenues for generative design options are shut down by looking at only A and only B.
Seriously? Even basic free tools like Google Website Optimizer let you A/B/n or multivariate test.
> Finally A/B testing can undermine a strong, unified, cohesive design vision by just “picking what the user says”. A designer (and team) should have an opinion at the table and be willing to defend it, not simply cave into a simplistic math test for interfaces.
Sure, and not A/B testing can undermine a business by costing them thousands of dollars by not "picking what the user says". Pitting design teams against a "simplistic math test" is a silly way to polarize the issue. The designer or design team should be thrilled they have such a wealth of data at their disposal to come up with the best design possible. It's not about fighting, defending, or caving, it's about using the data to know -- and not blindly guess about -- what's going on.
> A/B test results perpetuate a falsely comforting myth that designs can be graded like a math test, in which there’s a single right answer.
The author asserts this several times throughout the piece but never provide any evidence. Will they shed some light on real case studies where this happens?
Indeed, if there is any "falsely comforting myth" doing the rounds, it's that designers can guess and know what thousands of disparate users will prefer within a single percentage point of accuracy. If you can guess that well, I'd like you to pick my stocks.
> And you risk dissuading top quality design talent from joining the team’s cause for good, beautiful, useful designs that improve the human condition.
Who's "human condition" are you improving by potentially harming client's businesses by blindly rolling out designs which may in fact cost them thousands (or hundreds of thousands, or millions) of dollars, and therefore jobs, and so on...? This is another silly, polarizing argument.
tl;dr: If you want "good, beautiful, useful designs", then A/B test "good, beautiful, useful designs". It's not us vs them. We can have our cake and eat it too!
--
Surely it's also not simply being pedantic to insist that things like CSS3 (!) and SVG don't get lumped into the "HTML5" catch-all. Pity the poor designer or developer just getting to grips with this stuff who has to figure out what someone means when they say "HTML5".
Nevertheless, as Mark Pilgrim said: "HTML5 will continue to be popular, because anything popular will get labeled "HTML5"." (http://diveintomark.org/archives/2011/01/09/dive-into-2010)
However, what I think there is a bit of a market for is web-app-as-a-commodity -- well, not actually the web app itself, but the billing, membership, marketing site integration stuff that's all _generally_ (and I stress _generally_) pretty common. Or, at least, developers who market themselves that way -- "I'll take care of all the x, y and z, and focus on making your app idea as good as it can be (and here's how many times I've done it before)". Boom, great, take my money, get it done.
The actual web app part though is an absolute wild card, and that's where you should be aiming to make the $$$ -- provide everything around it as a done deal, and focus on getting the MVP/app idea right (which is 90% communication & email with the client, and 10% actual coding).
Remember, people want their hand held. There's a million different things to worry about. Thinking is hard. It's time consuming. Make it as easy as possible, hold their hand, allocate way more time than you think is reasonable (and you'll still get burned), and charge a fair rate for it. After all, that's what you're doing - it's still time-for-cash, it's just a question of how you market those services.
Finally, don't forget:
> The napkin crowd believe they have specs. They do not have specs; they have napkins.
Good luck! ;)
Their 'reverse freemium' approach is really interesting -- rather than de-emphasising the free and focusing on the [pre]mium side of their biz, they've been slowly expanding their free options and growing rapidly -- in terms of profit -- because of it.
Generally speaking, tech publishers have a sausage factory-like production line books get pushed through. There's editors, designers, marketing, and acquisitions people involved in taking the author's manuscript through to a finished product, so that's where the money goes.
It also gets spent on all the dreck tech publishers (again, in general) pump out -- they work on a hits model, so they publish 20 books and hope one of them takes off. Their main function is to nag authors to get their work done.
I've heard "[book] publishing is broken" over and again, and it's true. Tech publishers only pay peanuts, so authors get pushed to finish the book quickly (if they start at all), so the quality isn't very good, so people don't buy many books, so tech publishers can only afford to pay peanuts...
Sadly, not much has changed for the major tech publishers since the late 90s at least, when Philip Greenspun wrote about his experience in 1997: http://philip.greenspun.com/wtr/dead-trees/story
That said, there are publishers which give their authors a better deal -- Pragmatic Programmers for one (which Peter Cooper discusses -- see link in his comment), who pay 50% as discussed here: http://pragdave.blogs.pragprog.com/pragdave/2009/10/pragmati... . They may not do the same volume as the majors, but I doubt it's 5x less. Anyway, I agree it would be more healthy if those buying books bought from publishers who gave their authors a fairer deal; however I'd really like to see authors support businesses that support them -- they're the ones with the product after all. The more we can support highly efficient indie publishers, the more the book reading community will grow around them, the healthier the ecosystem on the whole will be, imo.
Failing that, I'd like to see tech giants like Google just give employees who write a book a $5k raise, cover editing/production costs for $5k, and then use the self-publishing infrastructure (with the same $1/book for the author) & free electronic distribution to get the books out there as far and wide as possible - $9 print, $0.99/free PDF/app, free web version, say.
14k sales in 6 weeks is great for a tech book (~5k lifetime being the norm), but on the back of 2 million visitors to the web site, and Mark Pilgrim's reputation, it illustrates why publishing is a tough biz.
O'Reilly pays authors 10% of net income, which is usually 50% of retail. Given Mark's book is going for $30 retail and $17.72 at Amazon, that's about ~$0.88 - $1.50 per book for print (for the sake of the argument, let's assume digital is in the same ballpark). Mark's royalty check for $9225, + whatever advance Mark got (say $5k?) equals roughly $1/book sale for the author (assuming that payment was for 14k sales to date). Given it takes 100s of hours to write a book (and hundreds more to write a good one), making $1 per sale, on average sales of 5k lifetime, or 14k sales in 6 weeks for a top tier author like Mark... let's just say technical writing is not exactly something you do for profit :)
Also of note is the 75%/25% print/digital sales ratio. This would suggest that the free online version is cannibalizing digital sales, given O'Reilly usually sells ebooks 2:1 (http://radar.oreilly.com/2009/08/does-digital-cannibalize-pr...), but I guess you could make a case that the loss in digital sales is made up in extra print sales from the book's exposure online.
I understand the sentiment here, but I don't think it's true. There was a recent quote from El Jobso (can't find off hand, sorry) saying that Apple actually sat on the top end of the market with the Mac (in his absence), got greedy, failed to innovate, and suffered.
Instead, look at what they did with the iPod. They didn't just remain as a high-end mp3 player, they developed products which covered nearly the spectrum of the marketplace, while still remaining the high-end brand.
I think past will be prologue, but it should be the iPod, not the Mac, that will be the best example. That said, when you're taking 50% of the profit in the market with just a 4% share, there's not a lot of reason to be particularly unhappy with your current position!
Wave took something that worked and made it worse -- fewer contacts, bloated app, confusing purpose.
FB is taking something that works and making it better (for its users) -- more streamlined (like chat), less hassle (existing friends list), less noise ('Other' bin).
What politician is going to risk canning a security protocol when there's a not-unreasonable chance of a civilian airplane being blown up in the next few years? Who wants to be the politician that has to explain, after several hundred people died, why they canned any device or protocol which may have helped stopped the attack, however remote the likelihood?
I'm not saying it's rational or appropriate, I'm saying it's politically untenable, regardless of the horror stories like the one posted.
Perhaps airlines should offer 'enhanced' or 'standard' security flights, and people can choose whether they fly on a plane where everyone has pat downs or scans, or whether they don't. At least that way there would be a choice...
Surely the iOS phenomenon (and Android & WP7) -- particularly the iPod touch -- shows that for handheld gaming a chunky controller isn't necessary.
As Gruber often points out, where _is_ the iPod touch competitor? This is likely to be too expensive to compete with the iPod touch, and too chunky/niche to compete with iPhones and other smartphones as a platform in its own right. And that's what Sony needs -- an iOS-like platform, not this strange bit of hardware.
It's pretty sad that for a company that has built some of the most popular consoles of all time, all they can come up with in 2010 is the mutant child of their controller and a generic smartphone.
There are a few small problems though:
- Page 'load' time is a bit ambiguous, I'd prefer page 'rendering' time, perhaps. (The author does note the limitation of this approach.)
- Time to window.onload may be a _very_ long time in some environments, especially when 3rd party scripts are present, especially (if memory serves) in IE.
- From eyeballing the "Bounce rate by page load time" the linear trend placed on top seems dubious -- if anything bounce rate seems to drop until the 3700ms mark. (The author notes in the text however that users seem tolerant up to the 5000ms mark).
Quibbles aside, having more page rendering time data correlated to bounce rate is an excellent idea, as I've seen some posts in web design land get pretty carried away with n-th degree optimization because they read speed matters for Google and Amazon.
In my experience, I optimized a site's page rendering time dramatically, expecting a bump in average pages per visit, and saw absolutely no change whatsoever. We need more data :)
But Windows Live Mesh is not just a way to synchronize files. That's just the sample app. It's a whole goddamned architecture, with an API and developer tools and in insane diagram showing all the nifty layers of acronyms, and it seems like the chief astronauts at Microsoft literally expect this to be their gigantic platform in the sky which will take over when Windows becomes irrelevant on the desktop. And synchronizing files is supposed to be, like, the equivalent of Microsoft Write on Windows 1.0.
It's Groove, rewritten from scratch, one more time. Ray Ozzie just can't stop rewriting this damn app, again and again and again, and taking 5-7 years each time.
And the fact that customers never asked for this feature and none of the earlier versions really took off as huge platforms doesn't stop him.
If they just want Flash, I bet they're bummed they didn't buy Macromedia a few years ago.
Don't know about all these books recs. All good books, but reading books wont make you a better designer. Reading books about fine art wont make you a better painter; and reading books about soccer wont make you a better soccer player. You'll learn the rules, sure, but at the end of the day you need to develop the skills by training/being coached (if I can run with the metaphor), so I suggest taking a short course on graphic design fundamentals and learn by doing.
Getting drilled on the fundamentals will stay with you much longer than anything you read in a book about ux/design.
Nice bit of marketing for IE9, though.
From the first piece: "So why do people listen to this guy? Part of it is doubtless the “famous for being famous” effect. Taleb is witty and clever, and ruthlessly promoted himself as a public figure after his trading days were over. Not a bad strat for success in the modern media culture: grab the megaphone and declare yourself a genius. Everyone wants to think they’re more of a mental bigshot than the eggheads who run things. Plus, it’s not like Jim Simons has time to write Financial Times articles. He’s too busy making money more or less proving people like Taleb wrong."