Comment by pyre | Hacker News Reader

Comment by pyre | Hacker News Reader

Parent Full thread

> One claimed a 78% larger sign-up rate and the other was something like a 65% larger sign-up rate.

Important, but unanswered questions (because they are hard, so let's ignore them!):

(1) How many of these users just mark all of the newsletters as spam?

(2) How many users don't realize that they can dismiss the dialog, and enter their email because they think that they are forced to? How many of these are really "engaged" users vs. just users that are willing to jump through a couple of hoops before they disappear in the ether?

(3) If those extra sign-ups are due to people thinking that they have to enter their email, how is this any different (ethically or practically) than actually forcing them to enter their email? Are you only making this distinction in the smallest way possible so that you don't run afowl of anti-spam laws?

(4) Did the A/B test include a "No thanks, I want to read the article instead" instead of just a "No thanks" button? How much does that affect the number of sign-ups?

(5) How many of those users will actually open their newsletters vs. getting lost in the clutter of other newsletters that they automatically sign-up for?

I've run A/B tests on this and seen meaningful higher conversion rates.

To answer your questions from my experience (dealing with a non-tech audience):

> (1) How many of these users just mark all of the newsletters as spam?

The deliverability and negative feedback for users who come in through the popover is not statistically different from users who sign up in other positions (bottom of the page, for example).

> (2) How many users don't realize that they can dismiss the dialog, and enter their email because they think that they are forced to?

We did some basic user testing and the vast majority of people had no difficulty with finding the close dialog. If this were the reason, I think it would be born out in (5).

> (3) Are you only making this distinction in the smallest way possible so that you don't run afowl of anti-spam laws?

IANAL, but I don't think there's actually any law preventing an email wall. We even tested one at some point but it did have a meaningful effect on bounce rate, and it did tend to get more undeliverable emails than the dismissable version.

> (4) Did the A/B test include a "No thanks, I want to read the article instead" instead of just a "No thanks" button? How much does that affect the number of sign-ups?

Why would a business do this? What business goal would including the additional copy fulfill?

> (5) How many of those users will actually open their newsletters vs. getting lost in the clutter of other newsletters that they automatically sign-up for?

In my experience, ~20% open the emails and then 40-50% click a link within the email. These rates were not statistically different for people who subscribed through popovers vs. less intrusive forms.

That being said, I still personally dislike popovers and never subscribe through them.

> In my experience, ~20% open the emails and then 40-50% click a link within the email. These rates were not statistically different for people who subscribed through popovers vs. less intrusive forms.

That is, by far, the most ridiculous click through rate I have ever seen for a newsletter.

> That is, by far, the most ridiculous click through rate I have ever seen for a newsletter.

We have a pretty great fanbase. :)

I get those kind of rates on welcome e-mails and product specific follow-up e-mails, but not for general blasts. That said, there are some segments of super fans that will consistently click through at a 40% rate (especially in Canada for whatever reason).

I recently received an email from someone because because it was too difficult to sign up for my newsletter. Mailchimp uses double-opt in and the steps ended up being 1) fill out email 2) confirm email address 3) click 'I am not a robot' button.

My newsletter open rates are around ~45% and click rates are ~15% which both seem to be pretty good and I attribute this to the fact that you have to really want to subscribe in order to do so. Given that I pay based on the number of subscribers I have, and that the purpose of my emails is to point people to the website, I actually periodically unsubscribe people that never click/open them.

Reply on news.ycombinator.com