DigitalOcean VPC
blog.digitalocean.com
blog.digitalocean.com
[1] https://blog.digitalocean.com/its-all-about-the-bandwidth-wh...
https://www.lastweekinaws.com/blog/why-zoom-chose-oracle-clo...
As long as Larry lives and breathes, Oracle gonna Oracle.
Oracle really should remove its logo from the footers of all those collapsing web sites. It's embarrassing.
Dislike Oracle all you want, even many cloud architected applications would fall over with a 25x traffic increase in a single week (remember Fail Whales?).
Nothing to do with Oracle Cloud (although Oracle Cloud won't be on my list unless it's marginally cheaper than other cloud service providers).
BTW: Talking about Oracle Cloud, my free tier trial ended miserably, 2 weeks after provisioning the VMs, Cockpit (I run it on my home NAS - managing/monitoring a small group of cloud VPS using the web UI) reported connection failed, only to find that my account has been terminated without any warning or notification along with my 2 free VMs based in Phoenix, lucky that I didn't actually put any workload on it (left them running only - feeling something's gonna happen...), contacted support and was told account deleted, no reason, redirected me to customer support (my oracle support, I couldn't figure out how that works, so give up...). I still don't understand how Oracle Cloud login works...
I've been a long time DO customer and overall happy for the past 7 years. I've write my personal experience [1] with DO in another post.
Why remove the logo if you can't see the website? ;)
Or maybe it's palliative care.
Is there a cloud or dedicated server farm with even cheaper outbound bandwidth?
Edit: as much as I hate Oracle, their first 10TB is free, and each GB after that is $0.0085/GB. Better...
I'm currently sitting at 4.2 TB of outbound traffic for the last 30 days, so I still have plenty of room to scale up my outbound traffic before I hit any limits. But most importantly my costs are fixed.
One problem is also that apparently some Americans have really bad peering to my server. As an European, I can't really confirm if this is the case, but it's what I've heard.
Not unlimited to 10gbps - but free up to 20tb:
> Traffic usage is unlimited and free of charge. Please note that our unlimited traffic policy does not apply to servers that have the 10G uplink addon. In this special case, we will charge the usage over 20TB with € 1.00/TB. (The basis for calculation is for outgoing traffic only. Incoming and internal traffic is not calculated.) There is no bandwidth limitation.
I've moved several people off AWS into Hetzner exactly because of their egress costs, in one case cutting their total hosting cost by 90% for that reason.
Even for people who stick with AWS and don't want to deal with any added complexity, even something as simple as putting a caching proxy in Hetzner and routing European customers to it can sometimes produce significant cost reductions.
That was two years ago though, maybe it has improved.
External outgoing traffic
First step up to 75 GB: 0 € per GB
From 75GB to 499TB: 0.01 € per GB/month
The servers themselves come with unlimited transfer.
---
On the technical side, we make use of an apps ability to direct where it stores its settings if it has one, and also move settings into/out of the registry and/or APPDATA on the local machine when needed. Our open source 'launcher' acts as a helper app to handle this for each app so it doesn't mess up a local version that's already there and so it adjusts paths if you move around between PCs and the paths to your apps or documents change.
1000GB Data consumption is 10$ (1.000 GB @ $0.01 / GB) but adding a Droplet for 5$ adds 1000GB to you Datapool.
So with adding cheap Droplets you could lower the Bandwith Cost 50%?
But it only gets you so far, its a cost optimization at the low end, but eventually it isn't "worth it".
You are checking your bandwidth use against the pool limit, then spinning up a new VM when you get close to the limit, but then back down the next month so you aren't paying for unneeded bandwidth, paying the extra $5 and not worrying about some custom price hack is a lot easier and less stressful.
- VPC network ranges cannot overlap with the ranges of other networks in the same account. (Edit: Does this mean each VPC in the account has to have a non overlapping subnet?)
-Resources do not currently support multiple private network interfaces and cannot be placed in multiple VPC networks.
- Not being able to change the VPC connected to stuff without taking a snapshot
- You can do this, but it's highly discouraged since it means no VPC peering if you ever need that.
- Can't do this at all with network interfaces, it all is via VPC peering.
- Can't change the VPC after an instance has been created, you have to take a snapshot and relaunch it.
I thought one would have a node two VPC, e.g. app and database, so it can speak to both, but the load balancers can't.
With peering one would have an app VPC and a database VPC and peer them?
Not sure best practice for DO since I haven't tried their VPC setup but it doesn't appear to have a way to let two VPCs interact yet.
- You can do it, but it's probably not a great idea if you need to do VPC peering (or attach multiple VPCs to one VM, see next).
- Does actually work, but it does not work if the VPCs you're trying to attach to a single VM have overlapping CIDRs.
- Same deal, almost. You cannot add or remove network interfaces from an existing VM.
If you're launching ephemeral networks for testing VMs / virtual appliances in their own, isolated networks, it can be totally feasible to have lots of them using the same addresses. You can only create 5 VPCs (at all) by default per AWS account, but they'll raise that limit for you if you request it.
Overlapping subnets tend to be a mistake and will bite you in the behind whenever you want to peer them. What'd be your reason to want overlapping subnets in the first place?
Simple pricing, nothing hidden, not the most feature rich ecosystem, but I get no billing surprises.
Source: customer for 3 years.
My main wish at this point is cross data center load balancers.
We manage them using Ansible.
[0]: https://www.digitalocean.com/docs/databases/redis/#redis-lim...
Is there a chance you could poke someone into looking into this?
I'm the tech lead for Kubernetes at DO. Just wanted to jump in and provide some clarification around the security issues you brought up.
The blog post you're referring to came out in December 2018, shortly after we released DOKS as a Limited Availability offering. By the time we announced our General Availability release in May 2019, we had done the following:
1. Changed our node bootstrapping process so that etcd information is no longer necessary in the metadata API, and removed said etcd information from metadata. 2. Firewalled off etcd so that it's accessible only inside the cluster. 3. Shifted how we run the CSI controller component so that a DO API token no longer needs to be stored as a secret in the cluster. 4. Switched from Flannel to Cilium as the CNI plugin, which allows users to configure network policies. We don't configure any network policies by default, but the option is there for users who want to use them.
These changes fix the vulnerabilities explained in the blog post. We do have further hardening measures planned, including limiting the scope of API tokens (one of the suggestions from the blog post, and also an often-requested feature from DO customers), but that's a big project so we can't provide a firm timeline for it at this point.
Hope this clarifies the current situation. If you or anyone else finds new security issues with DOKS (or other DO products) we would love to know about it. Our security team is always accepting vulnerability reports via their disclosure program: https://www.digitalocean.com/legal/contact-security/
Corporate prefers paying yearly to paying monthly, and for that reason work uses Linode. (Which is not bad either, IMO)
(Over in AWS land, they wrote a CNI for their own VPC networking. It turns out to have many strange limitations. For example, you can only run 17 pods on a certain type of node, because that node is only allowed to have 19 VPC addresses. I was quite surprised when pods stopped scheduling even though CPU and memory were available. Turns out internal IP addresses are a resource, too. DigitalOcean has the advantage of starting fresh, so might be able to use something open source that can be played with in a dev environment and extended with open source projects.)
The EC2 k8s network driver they wrote essentially will attach/detach extra ENI's on the fly and pre-allocate IP addresses to your EC2 host to allow for fast pod spin up/down.
I found this article pretty helpful to explain some of the AWS differences: https://www.contino.io/insights/kubernetes-is-hard-why-eks-m...
That's not what is happening in AWS. IP address are resources (duh), but that's no the issue. With their CNI plugin each pod gets its own Elastic Network Interface. ENIs aren't just virtio's virtual network, it could be ENA (100Gbps) or Intel VF (10Gbs). It's a hardware limitation of amazon virtualization stack starting with previous generation instances.
> I was quite surprised when pods stopped scheduling even though CPU and memory were available.
This is well documented here: https://github.com/aws/amazon-vpc-cni-k8s
10/8 should go very far if you do it correctly, hence why it's in use in almost all internal networks.
Until you start trying to connect to enterprise networks...
This is seriously ridiculous. It's 2020, and Google and now DO have no IPv6 in their cloud networks.
Couldn't tell you why they've chosen not to use IPv6 for VPC networks, though. Probably just for management simplicity.
(My guess, honestly, is that nobody asks for it. IPv6 is a problem for Some Other Day.)
After that, I had a whole native /56 at Red by SFR.
Currently I'm at Sosh by Orange and I have a native /56.
[0]: https://lafibre.info/remplacer-freebox/freebox-erl-et-ipv6/?...
It is also an administration problem - you've given out an internal IP network to someone else -- and now your ability to move it to a different IP address for whatever reason depends on the change processes at the other enterprise, which can take months -- or on the ability of your own network infrastructure to NAT it to the new location.
It tends to work much better all around if you have a well defined "external" interface, that gets properly monitored and filtered with a network/app firewall, and gets routed to the right server -- and then internal change for your organization are decoupled from change processes in the others.
And for that, v4 exhaustion really doesn't matter.
If knowing internal IP addresses is a security risk, then IMHO you have serious security problems. I used to do netsec too and the cornerstone was internal scans for unpatched or rogue systems and services and keeping systems patched and locked down. A network is only as secure as what is connected to it! We also had smart switches and APs where you could lock port to MAC and IP and thus could prevent rogues.
My personal rule was: any system that would not be safe to directly connect to the Internet without a firewall is insecure and needs to be fixed. The only exception is backplanes for things like internal databases/services or testing/dev, and those were separate networks for that purpose only. Separation was either physical or virtual/cryptographic. Back then we didn't have stuff like ZeroTier so we did that with IPSec and it was ugly, but we did it. Those nets could sometimes access the Internet (with restrictions) but could not even see the controlled internal LAN. They accessed the net via a port to outside the DMZ.
Next up was auditing software installed on internal systems. Next up was monitoring network traffic to detect anomalous activity. Firewalls are always the last line of defense. NAT is not a security feature at all.
I never once worried about keeping internal IPs secret (why?) and we ran IPv6 internally without NAT because IPv6 NAT is dumb.
We had two incidents when I was there. Both were the result of phishing to get malware onto personal PCs or phones.
My very strong personal opinion is that security people worry about the wrong things. They worry about network security and firewalls when what should really terrify them is phishing, auto-updating software made by who-knows-who, popular apps and SaaS services that are invisible security dumpster fires (Zoom anyone?), and of course barbarous demonic evocations like "npm install ...". Your firewall will do very little to save you from any of that, and NAT won't do crap because once again NAT is not a security feature.
Obscurity is NOT security. But obscurity as one layer in a larger defense-in-depth setup IS helpful.
Do note that scanning IPv4 through a fishing page is still about a million times harder (literally) than targeting a known address.
And NAT is not security, but in some context is still helpful as one layer in a defense-in-depth setup - you can’t directly attack something that’s not routable.
Security is not binary; there are costs and there are benefits to various setups. My point was that the benefits provided by being able to provide an internal IPv6 address to an external entity are dwarfed by both Netsec and netadmin costs.
Also, if you can so easily scan my internal network with malicious web pages, you can probably passively listen for the v6 addresses. On the networks I managed, browsing happened through VNC to a browser on tightly controlled host that could only connect outside and only through a proxy. How do your fishing pages counter this?
I am not opposed to network firewalls and such, but they're just defense in depth. If the whole network wouldn't remain secure if it were connected to the Internet with no firewall, it's not secure.
Given that these things are afterthoughts, I am not willing to prioritize them much over efficiency, complexity reduction, and user experience. Afterthoughts should be sacrificed to complexity reduction because complexity negatively impacts security a lot more. Inefficiency and poor UI/UX also have security implications. They increase the amount of "shadow IT" type activity and also seem to make phishing easier. If you secure something in ways that prevent people from getting their work done, they will get their work done insecurely.
Treating NAT as a must-have or should-have rather than the ugly hack you don't want to have increases complexity and negatively harms UI/UX by making P2P stuff not work and making people have to work harder to do simple things. If removing NAT makes you insecure, you were insecure to begin with.
Needless to say I am a fan of the BeyondCorp/deperimeterization approach. Ideally physical networks should be dumb pipes and everything should be virtual. The LAN itself is legacy baggage.
If you look at my original message, I was not suggesting NAT was useful - on the contrary, I was cautioning against relying on your internal NAT as a mitigation of the other enterprise's change processes. My whole post was about complexity reduction (as it relates to inter-enterprise conections)...
> Needless to say I am a fan of the BeyondCorp/deperimeterization approach. Ideally physical networks should be dumb pipes and everything should be virtual. The LAN itself is legacy baggage.
I also like it. But the post I was originally replying to implied a server->server connection between two enterprises, which is afaik not at all addressed by BeyondCorp or any of the projects it inspired - specifically, you need to treat the other corp like a Google user at home, rather than a Google employee in a hotel because you cannot enforce trusted hardware, inventory tracking, or any of the other things that make BeyondCorp as useful as it is.
I was wondering if DO publish some kind of roadmap? I'd really like to know what else they plan on delivering over the next year or so?
VPC support on DigitalOcean was soft-launched almost a month ago:
https://www.digitalocean.com/docs/networking/vpc/quickstart/
https://www.reddit.com/r/digital_ocean/comments/g1hkhu/digit...
The introduction of VPC just means you can isolate within the same account.
They also will automatically enable a private network interface for you if you use their Floating IP feature. This caught me by surprise when I found out the hard way :)
The worst part is here in Canada we have to pay for incoming texts.
Some plans will claim it's an "unlimited texting plan" but really the charges are still there hidden. Maybe the provider has an agreement with other mobile phone providers to reimburse them for texts their customers send.
My last provider refunded the cost if I forwarded the message to their spam department text number.
We have pretty terrible mobile phone plans and rules here in Canada.
I've seen plans that you choose friends or pay more but in some way you are paying more for the free part a bit like insurance. Maybe plans with unlimited texting versus regular plans are more common so the extra cost is seen as normal?
Hopefully it's changing or maybe texting is becoming less of a thing due to more mobile Internet access.
One thing I want to do is setup a VPN tunnel from my home network and lock everything else down. Wasn't possible before but it is now with this.
Given that now “Security and customer trust are at the core of what we do”, it would be nice if they could fix the massive oversight in their Spaces offering where every API key has full access to all spaces/buckets.
I also wish they'd add the feature to turn DO Spaces into a static server, like most other cloud providers.
Could anyone here share some other benefits to using DO? Or any particular Must Have's on a particular cloud provider?
Personal experience with DO: I've been a happy DO customer for the past [7 years](1). Linux VM [uptime](2) record has been amazing for personal use case.
This week I migrate the droplet hosting my personal website (5/m) from DigitalOcean to Amazon Lightsail (3.5/m plan) this week. Trigger being Ubuntu LTS upgrade to 20.04 again failed to boot on first few attempts again (wasted quite sometime chroot trying to fix to no avail without access to the hypervisor - IaaS...), mainly because of the way DO's flavour of KVM (hypervisor) works (I am not the only one), my other VPS (e.g. 123Systems - KVM) worked well and never had the same problem, let alone Xen powered VMs (EC2, self-hosted XenServer, etc. - I know hypervisor well because I've worked for XenSource/Citrix on XenServer for several years).
Customer (technical) support quality has dropped over the last few years, I can tell the difference by comparing the last 2 support tickets, I don't want to guess the root cause, sigh...
Finally I have had enough (4th time down with upgrade), it's time to move on to something better without paying more, migration is made easy due to the way workloads are deployed (most containerized, thanks to Docker/Docker Compose). With Lightsail, in addition to the AWS name/brand, has the advantage to move the Lightsail VMs into AWS EC2 instances so as to leverage full-fledged AWS infra (e.g. VPC, etc.) seamlessly.
Over the years, low end VPS competition has becoming much tougher (DO, Linode, Vultr, Amazon Lightsail late to the game but powerful strike, etc.) DO has lots its key competencies for bang for the buck, without offering 2.5~3.5/m plan on par with competitors.
Last but not least, I'll definitely consider DO as an option when Cloud Infrastructure is need, still ;-)
BTW: On Oracle, my Oracle Cloud free tier trial ended miserably, 2 weeks after provisioning the VMs, Cockpit (I run it on my home NAS - managing/monitoring a small group of cloud VPS using the web UI) reported connection failed, only to find that my account has been terminated without any warning or notification along with my 2 free VMs based in Phoenix, lucky that I didn't actually put any workload on it (left them running only - feeling something's gonna happen...), contacted support and was told account deleted, no reason, redirected me to customer support (my oracle support, I couldn't figure out how that works, so give up...). I still don't understand how Oracle Cloud login works...
Vpc isn’t simple.