hhw

Almost everyone new I meet moved to Vancouver from elsewhere. I often get the comment that I'm the first person they've met that grew up here.

Rain is preferable to freezing snow in most other parts of the country, and true Vancouverites don't mind it because it means fresh powder on the mountains for skiing/snowboarding, conveniently located 30 minutes from downtown and open at night with artificial lighting. And there's no place better in the summer, when there's no rain, extreme heat, humidity, or mosquitoes. Just lush green surroundings, and ocean and mountain views everywhere

I've lived in an older but expensive (even by Vancouver standards) neighbourhood with larger houses and lots most of my life, and many of our neighbours have been replaced by mainland Chinese with absentee husband/father.

I'd say the effects are overestimated by locals, but they are definitely around and easily observable by those who live here, and since they buy housing at the top end of the market, they skew the average home sales prices significantly upward. Realtors also espouse a "buy now or be priced out forever" which scares locals into borrowing way beyond their means, and banks in Canada are happy to loan them the money when taxpayers cover the mortgage for them (but not the home owners) through CMHC. Household debt in Canada is at record highs, worse than the US in 2008, and Vancouver is by far the most extreme example especially given that our average incomes are significantly lower than the other major Canadian cities.

Yes, we pay 7% PST on sales tax, which doesn't apply to groceries or restaurants, where presumably a lot of their spending would go. That's a far cry from the income taxes that should be paid if they're claiming resident status:

5.06% on the first $38,210 of taxable income, + 7.7% on the next $38,211, + 10.5% on the next $11,320, + 12.29% on the next $18,802, + 14.7% on the amount over $106,543

Generally, the ones who come over here would fit in that top tax bracket. They immigrate into the country based on a loophole in the immigrant investor program into Quebec, but then end up living here in Vancouver. As soon as they get their citizenship, the husband goes back to China and the wife and children stay here and no income is declared.

Property taxes are municipal, which does not contribute to healthcare and education. They may not be paying nothing into the system, but they certainly take more out of the system than they put in.

What makes this the most egregious is that many of them claim additional low income benefits, like discounted pharma drugs and even collect welfare once they've received citizenship.

There's also several pregnant mainland Chinese women coming into the country before they show, and having their babies here so that they get Canadian citizenship and all the benefits that entails.

Yes, it's fairly easy to get a student visa in Canada. There are tons of ESL (English as a second language) schools around downtown, with tons of latin american, middle eastern, Japanese, Korean, and mainland Chinese young people. Some of those schools are designed so that they don't actually have to show up, but still show up as enrolled for student visa requirements.

The worst part is they collect social benefits like education and healthcare without paying any income taxes into the system.

There's no such tax on 'passive income', it's just taxed as business income, which is exactly what it is and taxed no higher than employment income. It's just not taxed as capital gains nor has dividend tax credits, which it shouldn't be/have.

The only tax that discourages renting out is that the portion of your principal residence used for renting out no longer qualifies for tax-free capital gains.

Considering how almost nobody in BC ever declares this, and barely anything is done to curb it, it's kind of a moot point.

> Whoa, nope[1].

Perhaps fewer issues in the last month, but your own status page shows a litany of issues in September and earlier months. And I am fairly certain your status page has not covered every instance of attacks / packet loss on your network(s).

> We aggressively de-peer with networks that regularly originate attack traffic, allowing us to size our ports according to utilization rather than worst case attack sizes.

The Internet is far from perfect, and every network of sufficient scale is going to have a number of compromised machines on it, especially eyeball networks, who should be your most desirable peers. Rather than blanket de-peering such networks, you should be looking at PNI's with them.

True, there are a number of 'bulletproof' networks, but they are few and far between.

> Multilateral peering is actually a bit more expensive than transit these days - much more expensive if you intend to significantly overprovision capacity.

That is simply not true either. Even if taking the worst case of leasing waves across 3 major long-haul routes across the continent, you're looking at about $0.20/Mb. 100Gb waves have gotten cheap in the last year, almost cut in half of what they were not too long ago. Unlike transit though, you get full use of both directions, so your effective rate as a content heavy network is going to be closer to $0.10/Mb. And this is just a small minority of your traffic; most peering traffic is local and much of it would travel much shorter distances as the majority of non-local destinations aren't going to be at extreme ends of the continent. And at the end of the day, you can simply limit not accepting peering routes in a given city from other cities that are too far away, and push those out through local transit instead if you prefer.

Some exchanges are even one-time fee only, and most others are priced quite reasonably. You peer with networks available on those exchanges first, and only peer with networks at the more expensive exchanges if they're exclusively there. Believe it or not, other networks like to save money too, and those worth peering with are usually at the more cost effective exchanges.

> That's fair, but missing context. We're figuring out who works best for our traffic profile. We will scale back/remove the underperformers and scale up those that prove their worth.

That's a very expensive way to do things, considering minimum 1 year terms for most providers. Would've been significantly cheaper for you to hire some consultants with extensive experience working with the networks you were considering. Heck, you could have just sent some of the more outgoing members of your network team to a NANOG and gathered feedback for free over a few vendor sponsored beers.

It's also not exactly rocket science to figure out which Tier 1's are strongest in which corner of the world. At the end of the day, you're not dependent on their support if you're multi-homing (and shouldn't be because most of them are terrible).

> All of this is dramatically incorrect.

See points above.

Moving to suggests you're not actually there yet. What are your aggregate capacities like in each location at present as of this announcement?

Why the heck would you send traffic out transit links using GRE tunnels, when it's more cost effective and you have more control over your own private backbone links? At the scale I speculated 1/10th of what you're suggesting, it would have already been cost effective to operate a backbone. If they're anywhere near the scale you're suggesting, then it should be a no brainer for them to operate their own backbone of 100Gb waves.

I'm quite skeptical of the numbers you're citing though. Their PeeringDB profile suggests they only have 10-20Gbps of peering per city. Considering their profile was updated just a few days ago, I would be inclined to consider those listed capacities accurate. Although they mention 'hundreds of Gbps' of capacity per city, they also mention sending up to 50% of traffic through peering in London, where they only have 50Gbps of total peering capacity. Perhaps you're right on that 300Gbps of capacity per location, in which case they would run much lower utilization rates on transits than peers. But that would be even worse allocation of spending than in my initial assessment, considering the cost of a port at an exchange is much cheaper than a transit link with CDR. It also leaves them highly vulnerable to DDoS attacks through exchanges.

For a content network, public peering at exchanges in North America just with route servers and networks that have open policies would result in 30-40% of traffic going through the exchange. They would easily do more traffic at the exchange than any one transit in a mix of 3-4, let alone in a mix of 5-7.

With any significant private peering, easily 60% of traffic could be settlement free. And guess what, most significant peers require peering at multiple locations with a full set of prefixes, which requires you have a backbone. With the traffic levels you're suggesting they run, they should be able to negotiate settlement free peering with many major regional Tier 2's, making it even less sensible to be purchasing from multiple ones. Considering most Tier 2's within a given region will all peer with each other, there are very few improvements to be had by turning up additional ones. Where there's the most room for improvement is being on the right long haul fiber paths. In which case, given their North American focus, they should be buying from Level3 and they probably could at comparable rates to their current agreements by concentrating more of their commits at fewer providers. If they had their own transport, they could also determine which fiber paths they take across their backbone to ensure optimal latency. Beyond that, given that Tier 1's all peer with each other by definition, it's just a matter of dumping traffic out any one of them for local traffic without traversing a congested peering link. The microseconds it takes to go an extra AS hop within a city has indistinguishable impact on performance.

I'm not sure what best practices you're referring to. Who else can you name that utilizes up to 7 transit providers in a given city, without operating their own backbone? The only one that I can personally think of Internap, when they abandoned building their own backbone halfway through turning it up. Ask their former network engineers, from their golden years when they had their highest market share, how that worked for their network and for their business.

Are you a current Linode customer in one or more locations? If you were, you'd probably have experienced packet loss issues on a regular basis due to DDoS attacks. There's a reason why they're performing these network upgrades; they've had near daily network interruptions due to DDoS attacks since Christmas of last year, with some outages lasting up to almost a day. Smart network engineering would have never let their network become that unreliable in the first place. And if you were going to blame a lack of budget for that, my suggestions would be even more appropriate for them as they would them to scale their network in a much more cost effective way. An external scrubbing service makes sense, when they've been ineffective at mitigating attacks to date. Your network is only as resilient towards DDoS attacks as your weakest links, and spreading out capacity to a larger number of providers instead of concentrating higher capacities with fewer ones makes it much easier to saturate connectivity to one of them.

The only way Linode's current network strategy makes sense, assuming that it's not due to technical oversight, is if it's marketing driven. That's a fair reason, but it should also be fair to call them out on it. I'm not sure why you feel that strategy is in any way optimal, when it's the opposite of the models of hosting companies most renowned for their networks. Take for example SoftLayer, who went to great lengths to build out their own backbone fairly early on. I may halfheartedly agree that Linode's network upgrade strategy might be smart marketing, but I would wholeheartedly disagree that it's smart network engineering. It's not cost effective, is sub-optimal for resiliency against attacks, and fails to leverage peering effectively.

"we now manage our own true service provider network, allowing us to deliver robust and reliable connectivity."

What's needed to combat DDoS attacks is distributed defense. Without their own backbone / private transport links between all of their locations, their network is just a disparate set of data centres and there is no advantage to their having multiple locations, so far as protection from DDoS attacks are concerned.

They also fail to mention what capacity each of the links are. They could be anywhere from 1Gbps to 100Gbps, but I presume they'd mention as a selling point anything 40Gbps and up, so let's assume they're using all 10Gbps links and not 1Gbps to give them the benefit of the doubt. So, they range from 50Gbps (Singapore) to 100Gbps (London) per location.

It's an impressive list to look at in aggregate, but not really that much for any one location in 2016, especially given a company of their size and visibility, when you can rent shared access to a 200Gbps+ botnet for $19.99. https://www.nanog.org/sites/default/files/20161015_Winward_T...

Instead of buying transit from up to 7 carriers per location, when there are starkly diminishing returns after 3 or 4 so far as routing performance is concerned, they should have instead bought higher capacity to each provider (to ensure at least 10Gbps of unused capacity per provider outside of regular legitimate traffic), external DDoS mitigation, or domestic backbone links and turned up more capacity at the LA Any2 (for Asia) and NYIIX (for Europe) to absorb the majority of DDoS traffic which comes from those regions. With up to 7 carriers, they simply have 7x different points of failure each at only 10Gbps, while getting worse deals on transit pricing due to lower volumes with each provider.

Other people's tragedies can be cathartic, or at least so I recall being taught in high school English class from nearly two decades ago. Knowing that someone has it worse than you do can make you feel better about your own situation, not due to schadenfreude but just by helping to put things in perspective.

I'm not sure what was shared with me should be public knowledge or not, but based on how the numbers were come up with, I think most employees would've received high 5's to 6's. Not sure about early employees, or if there really are any. Marcus had already made it quite big already (millions per month in ad revenue) when he was just a one man show.

This would count as capital gains, so (33% federal + 14.7% provincial) / 2 would be closer to 23.85%.

That's not counting the portions that fall under lower tax rates, or the lifetime capital gains exemption of $750,000.

If he did structure it under a holding company, he could better spread out the payments over a number of years so that more of it could fall under lower tax rates.

Way to assume the worst of someone without knowing anything about him, or any details of the situation.

I have good friends who work there, and they received generous bonuses when the acquisition happened last year, as well as having been treated well all along previous to that.

How many networks still run SONET at this point? Most global carriers would run OTN over DWDM at 25GHz or 50GHz spacing, with up to 200Gbps per channel. There is of course OTN encryption though

His regrets after Malcom X was dead were warranted, and doesn't excuse his betrayal. What matters is how you treat someone when they're alive, not what your sentiments are after their death.

It's been several years since I read Malcolm X's autobiography, but he did specifically talk about his relationship with Ali and what happened, which was the real basis of my comment. Malcolm's depiction of the events is much more negative from my recollection. I just provided the above link instead, since I wasn't able to find an excerpt from the book.

Not sure why my previous comment was voted down, given that it was both true and on topic. I can only assume someone would prefer to keep their hero on a pedestal rather than admit everyone has their flaws.

Ali also turned his back on Malcom X:

http://www.npr.org/sections/codeswitch/2016/02/25/467247668/...

It's the GPL that's incompatible with the CDDL and not the other way around, which is an important distinction. Other than other GPL variants and BSD and MIT style licenses, the GPL has restrictions which make it pretty much incompatible with everything.

Using dedicated ZIL significantly reduces fragmentation:

http://www.racktopsystems.com/dedicated-zfs-intent-log-aka-s...

Anyone using ZFS in a serious capacity would have both dedicated ARC and ZIL.

Property crime is commonplace in Vancouver, given that we've always been the heroin capital of North America. Violent crime like muggings, however, are very rare. The addicts are not confrontational.

Itanium was EPIC, pretty much the opposite of RISC. Intel bet on compilers being good enough to tell the CPU which branches it could execute in parallel, and lost that bet.

It's not trading with China that's resulted in Hong Kong's success, but that they were the exclusive intermediary for trade with China for a very long time. North Korea doesn't trade with the West so there's no point in comparing them. There was also a large influx of fortune and human capital that came out of China after the war.

Singapore is more a gateway with Southeast Asia, which certainly has provided it with a great deal of opportunity also, but not nearly to the same extent. I think it would be fair to say to that much of their success is owing to their policy, on a number of fronts.

I also do not mean to say anything about whether Hong Kong has good policy or not. I'm just saying that they would have still been successful regardless, so long as it didn't preclude their role as a trade intermediary. So, even if you were to perfectly recreate all of Hong Kong's policy, you should not expect to get anywhere near the same results. What's generally attributed to the 'genius' of Hong Kong would be much more aptly attributed to dumb luck.

Hong Kong's prosperity in the 80's and 90's had more to do with being the gateway between China and the West as trade started to open up, and less to do with policy or entrepreneurship, as much as some people would like to take credit for such.

If you need something US side, one of my ventures, microservers.io offers C2750 using Supermicro MicroBlades starting at $25/mo (with coupon code LAUNCH) out of Seattle. Includes full KVM over IP access so you can install any OS you like. Most popular distros are available via local samba.

We soft launched over a year ago and have a number of hosting industry insiders for customers, but are still working on the main, public facing website (domain just goes directly to an ordering page) as well as automation on the back end for near-instant provisioning. We'll be doing our official launch after that.

$0.05/GB may be cheaper than Amazon, but is rather expensive for bandwidth these days (would work out to approximately $10/Mb 95th percentile using a 200GB:1Mbps conversion ratio). With us, 2TB of bandwidth is included, and additional bandwidth is $0.002/GB or $0.40/Mb using a GTT/Tinet, Hurricane Electric, and Cogent bandwidth blend provided by my main hosting company's secondary/value network (AS63213).

If a single point of failure took down your entire network, then your solution wasn't engineered properly. Just because on provider messed up big time doesn't mean every other provider would. The hardwdare design/architecture can also be provided to you for free as part of the sales process, no extra service required.

As for hardware failures in general, you can afford to have some complete extra servers on hand doing nothing most of the time with the amount of money you'd save on renting bare metal instead of paying for cloud. Or if engineered properly, you'd be using all the machines at once but be able to afford to lose half of them, and failover automatically.

Yes, a cloud (bunch of VMs on a cluster) is going to be more reliable than a single machine, as you get hardware redundancy. But a cluster of machines with hardware redundancy is going to be much more reliable than a bunch of VMs on a cluster, as there's less complexity involved without the cloud stack, and thus fewer points of failure.

The CDDL does not prevent ZFS from being used in Linux. It's the GPL that prevents using CDDL code in the kernel. Oracle doesn't have any grounds to sue based on their IP. They would only be able to sue on behalf of Linux, and violation of the GPL. Although I wouldn't rule that out, it's somewhat less likely to happen.

On the other hand, I suspect RMS isn't too happy with this turn of events. The sfconservancy may be the more likely party to bring a lawsuit. I'm curious to see either of them comment on the situation.

It's possible to only pay the lower end of that range for a 2Gb 90th percentile commit on a 10Gb connection out of most major markets. Cogent pricing fluctuates wildly depending on your ability to negotiate.

Ignore the whole song and dance when they have to bring in a VP into the call to specially approve the pricing they're trying to pitch to you at. Those are standard transit sales tactics. Or avoid Cogent's sales entirely by working with a reseller.

This is a false dichotomy. There's no need to flip to colo, when dedicated hosting exists. The provider takes care of the network, hardware, and potentially much more depending on the service agreement. You can even have the provider setup a cloud environment for you, if you want to be able to spin up and down VMs. This eliminates the capex costs, while having much lower opex costs than colo at smaller scale.

Considering that you can easily do dedicated for 1/7th the cost of Amazon for the equivalent amount of resources, it doesn't take anywhere near 150 machines to realize cost savings, it's more on the order of one complete physical machine as a dedicated server. You can purchase a 2nd one for redundancy and/or for always available additional capacity, so that you can burst to double your usual peak performance and still save 5/7th the costs of Amazon. If you're expecting to require the ability to scale much higher than 2x, you can also purchase a bunch of servers just for 1 month, or even less time depending on the provider, or take a hybrid approach if you really need an extreme amount of dynamic scalability. The reality though is that such extreme scaling is in the minority of cases.

I'm not sure how you arrived at the 150 machines number, but that sounds like 3-4 full cabinets and about right to have a reasonable scale for colo.

The real package manager itself comes upstream from Debian; Ubuntu just normally puts fragile GUIs around it. Most of the positives for Ubuntu come from Debian. The Ubuntu specific stuff on the other hand sometimes breaks core functionality like booting off software RAID being broken in 12.x, which has worked fine in Debian for at least a decade.

Generally speaking, a server environment is not the right place to run bleeding edge software. Debian's conservatism is quite sensible when it comes to production server environments, and traditionally (before the whole systemd debacle) they were pretty good at prioritizing stability over the feature of the week demanded by more desktop-centric folks. There's Ubuntu and other derivatives that can cater to their specific needs without compromising the integrity of the distribution in general. And if you have a specific application that requires a more recent version, you can always make use of a 3rd party repository (that you'd of course have carefully vetted first), a binary package, or just rolling your own from source. It makes much more sense to take those extra steps for specific applications than take a blanket approach to rolling out new code all over your production environment.

There's still a whack of services that should be disabled on a server like rpc/nfs, bluetooth, avahi, etc (depending on 5.x/6.x/7.x) even if you go with a minimal install. Bloat aside, they can be security risks.

From the op: "I enjoyed its minimal install to create a light environment, intuitive installation process, and it’s package manager."

CentOS is neither minimal nor light. Ubuntu also isn't either of these things. Both of these distributions are more targeted towards convenience and ease of use, which means a lot of features/services that are generally unnecessary are enabled by default. The main reasons to use CentOS is compatibility with proprietary software made for RHEL, Ubuntu for people already familiar with its desktop version, or if needing to buy enterprise support for either.

If the op is primarily looking for minimal and light, he should look at pretty much any other major popular Linux distribution like Debian proper, Slackware, or Gentoo before CentOS or Ubuntu.