1,079 karma · joined October 14, 2014
If you are looking to add Accounts Payable to your B2B product, drop me a line at sandeep at merc○a d○t c○m
Prev: Eng @ Stripe
Founder @ Kbee.app / Simplemnt.com
Developer Advocate @ Google Cloud
https://twitter.com/SandeepDinesh https://medium.com/@SandeepDinesh
[ my public key: https://keybase.io/thesandlord; my proof: https://keybase.io/thesandlord/sigs/f2WVOvjTgL_9f-Z701fGoqiRtlfAavM5RzczPZjZem8 ]
I normally see 1-2 minute deployment times depending on the application.
(I work for GCP)
For example: https://medium.com/google-cloud/kubernetes-configmaps-and-se...
In this example, I have a single gist with multiple files, but Medium can't display just a single file from that gist at a time (maybe things have changed, but I don't think so).
Medium's gist and code support in general is sub-par compared to other platforms IMO, but as a platform it does make it easy to create and share content that is also easily discoverable.
I'm probably going to start hosting the canonical post on my own domain and just use Medium to syndicate it.
It looks something like this:
your container -> Compute Engine VM (GKE Node) -> container -> Borg
The container on top of Borg is used for scheduling and management. Joe's talk has a slide on this. As a GCP customer, you never have to worry about this or care about it, as it is an implementation detail.
>"The isolation is not perfect, though: containers cannot prevent interference in resources that the operating-system kernel doesn't manage, such as level 3 processor caches and memory bandwidth, and containers need to be supported by an additional security layer (such as virtual machines) to protect against the kinds of malicious actors found in the cloud."
As a GCP customer using GKE, your applications are separated from other GCP customer using VMs.
However, if you want to run your OWN untrusted workloads, then in the past you would have to spin up a separate VM for untrusted workload A and a one VM for untrusted workload B.
This sucks in terms of resource utilization. It would be better in many cases if you could run workload A and B on the same VM. That's where gVisor comes into play.
your untrusted container -> gVisor -> Compute Engine VM (GKE Node) -> container -> Borg
I hope this makes sense!
gVisor lets you run multiple untrusted workloads on the same VM, in this case a GKE node.
A public example of this is Cloud Run [1, 2]
[1] https://news.ycombinator.com/item?id=19616832 [2] https://cloud.google.com/run/docs/reference/container-contra...
HN Discussion: https://news.ycombinator.com/item?id=16319248
Basically the same idea, but it does refresh when you submit. The CSS buttons are quite a clever workaround to not refresh the page.
https://devblogs.microsoft.com/commandline/announcing-wsl-2/
> WSL 2 uses the latest and greatest in virtualization technology to run its Linux kernel inside of a lightweight utility virtual machine (VM). However, WSL 2 will NOT be a traditional VM experience.
Also, K80s are pretty old these days, though this makes them much cheaper than P100s and V100s. Comparing all of them from a price/perf standpoint (especially with TPUs in the mix) can get tricky because it's different for every use case.
I don't know what the roadmap is for removing these restrictions (or even if it is on the roadmap) but I'll update this post if I learn more.
Where do you feel it is shoehorned in? Firestore is easy to access from App Engine and Cloud Functions (and GKE/GCE as well) using the built-in service accounts, it has managed export into BigQuery. Would love to know where you found friction when using it with other GCP services.
> Why is Google so bullish on this thing?
While I can't speak for Google as a whole, I'm personally bullish on Firestore for a few reasons:
1) Gives you the same "client-side only" model that Firebase popularized. This makes creating applications much faster as you remove the whole server side component and basically all your ops work.
2) True "serverless" pricing and ops. Because you only pay for reads/writes and not instance time, your costs scale linearly with your app's usage. And you don't have to worry about sharding or other operationally complicated things to scale your database as you grow. The caveat here is you have to structure your data and queries well, otherwise your costs will skyrocket [0].
3) Gives you the same "server side" functionality as Cloud Datastore (which Firestore is basically the next generation of [1]). This means you can use Firestore in place of a traditional NoSQL database like MongoDB.
4) Strong consistency. One of the biggest problem with Cloud Datastore (and most NoSQL databases) is eventual consistency. Datastore worked around this with "Entity Groups" which in my opinion were super clunky to work with and very limiting. Firestory is strongly consistent so you don't have to worry about this even at scale, which is super nice.
5) GCF Triggers. The fact that you can trigger a cloud function when something is written to the database is very powerful, it's basically like a traditional database trigger or stored procedure but you can do anything.
The biggest feature gap for Firestore is the querying abilities. While it is way better than the original Firebase DB, it's nowhere close to a relational database or something like Cloud Spanner. The team is working on it though.
The "real time" stuff is interesting but not really super relevant to the things I like to build.
[0] https://hackernoon.com/how-we-spent-30k-usd-in-firebase-in-l... [1] https://cloud.google.com/datastore/docs/upgrade-to-firestore
https://firebase.google.com/docs/firestore/manage-data/expor...
Edit: As wsh91 says below, you are still responsible for the scheduling.
(I work for GCP)
https://chromeunboxed.com/news/chromecast-video-and-smart-di...
Not all of them, but in my experience, yes.
Many various GCP databases (most notably Firestore[0], and Datastore[1] before it) do this today. No provisioning, pre-warming, or capacity planning needed.
[0] https://firebase.google.com/docs/firestore/ [1] https://cloud.google.com/datastore/
You can run a container on a f1-micro VM for free forever: https://cloud.google.com/compute/docs/containers/deploying-c...
If you want something more similar to Hyper.sh, serverless containers are coming out publicly soon, and you can sign up for the alpha here: https://g.co/serverlesscontainers
Are there any examples of public gRPC APIs that provide their .proto files? I know Google does (https://github.com/googleapis/googleapis), does anyone else?
If you look at an image of a Cloud TPU board [0], I'm not sure how it would even work with off the shelf servers. It's 100% custom for a Google Datacenter.
Nvidia also has the full supply chain set up, as well as things like driver support and all the things required to sell this kind of complex physical hardware.
Google also has a program to give research scientists free access to TPUs: https://www.tensorflow.org/tfrc/
[0] https://storage.googleapis.com/gweb-uniblog-publish-prod/ima...
(I work on GCP, but not on ML and TPU stuff)
https://news.ycombinator.com/item?id=17795626
(I work for Google Cloud)
Both the Echo and HomePod have multi-microphone arrays vs the two on the Home, and this really shows in far-field settings (especially with loud music)
(I _do_ work for GCP)
(I work for GCP)
I actually think most of the server SDKs don't even expose many of the realtime APIs. Maybe they will in the future, but it shows that you can use Firestore like a normal database just fine.
(I work for GCP)