190 karma · joined February 16, 2012
With transcription I would recommend trying out "gpt-4o-transcribe" or "gpt-4o-mini-transcribe" models, which will be more accurate than "whisper-1". On any model you can set the language parameter, see docs here: https://platform.openai.com/docs/api-reference/realtime-clie.... This doesn't guarantee ordering relative to the rest of the response, but the idea is to optimize for conversational-feeling latency. Hope this is helpful.
This also enables shell-aware LLM prompting!
It's much more bare-bones than Fig but perhaps useful if you're looking for an alternative! Send me feedback!
I've been experimenting with CLI/LLM tools and found my favorite approach is to make the LLM constantly accessible in my shell. The way I do this is to add a transparent wrapper around whatever your shell is (bash,zsh,etc), send commands that start with capital letters to ChatGPT, and manage a history of local commands and GPT responses. This means you can ask questions about a command's output and autocomplete based on ChatGPT suggestions, etc.
You can see this approach here, I hope it proves useful to other folks! https://github.com/bakks/butterfish
- An unmentioned alternative to this pricing is that GCP has a deal with Cloudflare that gives you a 50% discount to what is now called Premium pricing for traffic that egresses GCP through Cloudflare. This is cheaper for Google because GCP and Cloudflare have a peering arrangement. Of course, you also have to pay Cloudflare for bandwidth.
- This announcement is actually a small price cut compared to existing network egress prices for the 1-10 TiB/month and 150+ TiB/month buckets.
- The biggest advantage of using private networks is often client latency, since packets avoid points of congestion on the open internet. They don't really highlight this, instead showing a chart of throughput to a single client, which only matters for a subset of GCP customers. The throughput chart is also a little bit deceptive because of the y-axis they've chosen.
- Other important things to consider if you're optimizing a website for latency are CDN and where SSL negotiation takes place. For a single small HTTPS request doing SSL negotiation on the network edge can make a pretty big latency difference.
- Interesting number: Google capex (excluding other Alphabet capex) in both 2015 and 2016 was around $10B, at least part of that going to the networking tech discussed in the post. I expect they're continuing to invest in this space.
- A common trend with GCP products is moving away from flat-rate pricing models to models which incentivize users in ways that reflect underlying costs. For example, BigQuery users are priced per-query, which is uncommon for analytical databases. It's possible that network pricing could reflect that in the future. For example, there is probably more slack network capacity at 3am than 8am.
Rough patches are:
- Live migration is sometimes not seamless.
- Pub/sub is missing some core features.
Yes I know Azure runs Linux, let me unpack that point: We had run previously on a cloud that wasn't focused on Linux hosting as their flagship OS. The effect we observed was that Linux was a second-class citizen in terms of features and performance. Perhaps its unfair to project that onto Azure, but I think its true that AWS and GCP think about Linux first, and Azure doesn't. Running a company on the cloud means relying on the compute product (GCE/EC2) as the foundation for your infrastructure, so we think this makes a difference.
It would be valuable for a lot of people to see more comprehensive stats across all clouds - I would love to see this personally and I think it would help people make better decisions about cloud infrastructure.
I work at an ed-tech company in the Bay Area - We're a for-profit company but we've found success in building study tools targeted at students and teachers rather than top-down school district procurement. If you're interested in working in this space, or just chatting, feel free to message me. Same to others!
There is going to be some huge number of new developers every year for the foreseeable future, and Mongo is the lowest barrier to entry database available. These people don't know SQL and don't care that "Postgres is better". Mongo does a few useful things really well, and everything else (like sharding) is a moat around that core value proposition, which is why Mongo gets away with being bad at a lot of stuff.
Mongo is good at:
* Starting up (when you download it, you literally just get a mongod binary).
* Inserting JSON documents.
* Indexing those JSON documents.
* Composing queries programmatically (because they're JSON documents).
* Replication (oddly).
Mongo is bad at:
* Analytical queries. I've literally written a book about Mongo and still can't remember most query syntax.
* Efficient data storage - documents take a huge amount of space.
* Scaling.
* Sharding - this feature has always been half-baked.
* A lot of other stuff.
Your immediate inclination is probably to say something like "but look at (Couchbase|MySQL|Cassandra|Rethink|Postgres), its so much better written and it does (document storage|relational querying|scaling|sharding|json documents now in version 9.4!!!!!) so much better". Again, thats not the point, the lesson here is that MongoDB is a thing because barrier to entry is a better feature for many users.
The irony of RMS spreading his own vision of "Freedom" is that the GPL restricts things which he doesn't agree with, which seems to violate the most basic principle there. I've been involved with several organizations which try to avoid GPL code, by policy, in favor of MIT/BSD because it raises too many issues. Not saying GPL hasn't done good things (my understanding is that much of Apple's compiler work is public bc they started with GPL code), but the wind seems to be blowing towards less restrictive OSS licenses.
The idea is to enable real SQL queries directly on data sets, so you don't need to worry about accessibility or how you'll query it. Not much data in it now, and it doesn't solve your immediate problem, but perhaps a better model for storing and representing this data.
This is the 3rd or 4th time I've seen this article in the past few days so I've decided to post my take. I work with Mongo in a production scenario but I'm hesitant to post because these things tend to turn into a pointless argument. So let me stress: this is not an attack on the blog post, I'm hoping to improve the discussion here. Mongo has some real problems, these mentioned are not the problems. Here goes, written along with the article's segments.
- "It lies." Mongo used to have a default where a driver would fire off a write and not check that it succeeded with the server. This was very obviously a decision made to improve benchmark performance, though I imagine a benchmark with only default settings would be rather naive. Regardless, yes, the default was a stupid corporate decision but it is well known and should be apparent if you're deploying a Mongo cluster. Additionally, as the author notes, this default has changed and this entire point is no longer a concern.
- "Its Slow." A real point he raises is that its a little wonky you need to send a separate message for getLastError. I suspect this is an artifact of Mongo's historical internal structure. http://docs.mongodb.org/meta-driver/latest/legacy/mongodb-wi.... . If you look at the wire protocol, I think it is designed such that only the OP_QUERY and OP_GET_MORE message types get an OP_REPLY back. getLastError is a command, which are run through an OP_QUERY message. He notes that using this check affects performance. It does, but lets dig into this: "Using this call requires doubling the cost of every write operation." If the author benchmarked this, I suspect he would find that it vastly more than doubles the latency of a single write operation from the client's perspective. My understanding is that when performing this kind of safe write, the driver sends an OP_INSERT, for which it doesn't have to wait for a reply, then immediately sends an OP_QUERY message (getLastError), on which it hangs waiting for the OP_REPLY. In other words, this is now slower because we've created a synchronous operation immediately after just firing off the insert command. Again, its a little wonky that we send Mongo two messages, but one is immediately after the other and that is vastly overshadowed by the fact that we now have to wait for a reply. I believe 1 synchronous send and receive is unavoidable to ensure a safe write in ANY system, and the argument about sending 2 messages really boils down to sending about 200 bytes over a socket vs 400 bytes, I personally don't worry about it.
- "It doesn't work pipelined / it doesn't work multithreaded." He is also missing the real complexity here, and this is where I have a problem with this blog post because this is not a theoretical discussion, if its content is true then it should just be proven rather than FUD launched into the world. As noted in the docs ( http://docs.mongodb.org/manual/reference/command/getLastErro.... ), getLastError applies for the socket over which the command was run, so its up to the driver to execute the getLastError on the same socket as the write, which is an implementation detail and a solved problem. The way the drivers do this in practice is you set the write concern and the driver takes care of the rest. If you run getLastError manually then it depends on the driver, but for Java the correct procedure is addressed at http://docs.mongodb.org/ecosystem/drivers/java-concurrency/ . So for the fastest possible safe performance, you multithread (which is effectively pipelining from the server's perspective) and run operations with the driver's thread-safe connection pool. Suffice to say people actually use these drivers in a multithreaded context in the real world, and they work.
- "WriteConcerns are broken." There are several relevant settings here, including write concern acknowledgement and fsync, the author is confused about how these map to the Java driver WriteConcern enum values. The acknowledgement setting (elegantly stored in a variable named "w", thanks 10gen) is the number of replicas that must confirm the write before the driver believes it has succeeded. I personally set this to 1, but you could potentially wait for the entire replica set to acknowledge. The fsync setting is whether or not this acknowledgement means that the server on these machines has completed the write in memory, or actually synced the data to disk. I set this to false for performance. There is an excellent StackOverflow answer on Java Mongo driver configuration at http://stackoverflow.com/questions/6520439/how-to-configure-.... . The author also spends time noting that if you only ensure the write succeeds on a single machine, and you irreparably lose that machine before replication, then the data is lost. This is obviously true for every distributed system.
I've used a 4-shard Mongo in a production multithreaded environment that handled several hundred million writes. For part of this period our cluster was extremely unstable because of a serious data corrupting Mongo bug (more on this in a second). I haven't done a full audit but based on our logs, in about 6 months I've seen exactly 1 write go missing (which I believe was in the rollback log), so I'm personally not concerned about the things mentioned in the blog post. I've also been happy with performance as long as the data size comes under the memory limit. If your data exceeds memory, Mongo essentially falls on its face, though this is hard to avoid when a query requires disk accesses.
Mongo is not without its problems, however. As I mentioned, QA is a real concern, we hit a subtle bug when we upgraded to v2.2 that caused data corruption when profiling was turned on and the cluster was under high load. It was very difficult to debug, and basically should have been caught by 10gen before their release.
Another serious problem is that sharding configuration is still somewhat immature, and it seems like every new release is described as "well it used to be bad, we finally fixed it". Here is an example: you pick a shard key that can't be split into small enough chunks, and now shard balancing silently fails. Ok, so you pick a better shard key, but you can't migrate shard keys, so you have to drop the collection and start again. Except dropping a collection distributed across a cluster is buggy, so you can't recreate the collection with the same name and a different shard key. So you just pick a new name for your collection, and you have this weird broken state for the original one sitting around forever unless you completely blast your cluster and start from scratch. This sort of thing is not fun!
Mongo has many pros and cons, personally I think its real advantage is simplicity for developers which makes it worth putting up with the other stuff. Sorry for being long winded, hopefully this has been useful.
- The most important part of the Dvorak keyboard is that the most frequent English letters are on the home row with the vowels pushed to the left. This means that you're less likely to move your fingers from the home row and more likely to switch between hands for each letter. Thats the theory of why Dvorak is faster and more comfortable. It's a little crazy that E isn't on the Qwerty home row.
- It was much harder to switch to a new layout than I expected. It took about a month of typing with Dvorak every day and destroyed my ability to type on a qwerty keyboard. For a period during this transition I couldn't really type well on either keyboard, not fun, and not easy to explain to my boss why I couldn't fucking type. Interestingly I can still type fine on my iphone qwerty keyboard, so it is apparently a separate process in my brain.
- I think the article is similar to saying "there's no statistical proof that using the metric system is faster". Using the metric system is the kind of thing that makes sense intuitively, but if you took 100 scientists using imperial measurements and retrained them to use the metric system, it would be hard to conclusively prove that it is _BETTER_.
- To extrapolate a little bit, there I think there is a broader point here that statistics are usually used to make an argument, often deceptively. It is incredibly hard to create a clean sample in the real world, and even then its difficult to really extract meaning from those numbers. Remember, the average person has less than 2 legs.
- My personal feeling is that Dvorak is a LOT more comfortable and I type maybe 10% faster with it, though I can't really back that up. Maybe its just the fact I learned it second, or maybe I'm just fooling myself, who knows. But I spend like 10 hours a day typing, so if I type 10% faster over my lifetime then I've... turned a profit? Maybe I'll spend those extra days at the end of my life doing a better study about how much faster sailors can type on Dvorak.
Even if you don't use Dvorak, I HIGHLY recommend switching your Escape and Caps-lock keys, especially if you use Vim. Think about how much more you use Escape than Caps-lock. I use something called PCKeyboard Hack to configure that feature and Dvorak on OS X. On Windows, theres something called Auto Hotkey.
Some background mixed with some opinion:
This is an area still without a major commercial competitor and there aren't yet any such public product efforts as far as I know (IBM has a couple of researchers publishing on the topic). The difficulty is that really exploiting the GPU requires rethinking architecture from the ground up, including how your query plan handles parallelism, how your data is stored, and most significantly, how to efficiently move data to the GPU for processing. Most databases aren't partitioned nearly well enough to shoehorn GPU processing in on the backend, and it seems like the vast majority of recent database development has been around highly distributed systems and commodity hardware, eg Hadoop. A German company called ParStream has a DB that uses GPUs for certain indexing operations, though last time I spoke to them it sounded like there is much much more that can be done with GPU hardware.
In general, speedup is highly variable, and depends on the data set, indexing characteristics, data type (fixed vs variable length), individual query, etc, but its fairly common to see 10x speedups over optimized multicore query execution. Though the extreme parallelism of the GPU is responsible for some of this, in my opinion the big win there is just absolute memory throughput. Newer Tesla GPUs have a memory throughput of > 100 GB/s without breaking a sweat, while most CPUs are around 30 GB/s. If you can push data through your hardware that quickly, its easy to see how the GPU can have such an advantage.
The difficulty, however, is actually moving the data into GPU memory (1-8 GB generally) to exploit this throughput, which is a major bottleneck (some of my research was around doing this better). I understand that GPU direct disk access is just becoming a reality, which could make this easier. The holy grail, however, is a full-powered integrated server CPU/GPU. If both of these components had equivalent main-memory access and this architecture becomes cheap and commoditized, then GPU databases will immediately become extremely relevant.
I expect that there will be a serious commercial competitor in this area within 3 years, either in the relational data warehousing space (think Vertica) or closely integrated into MapReduce type execution. My guess as to the reason it doesn't exist is that 1) production DB development is really hard and 2) GPU processing is only a really big win for certain workloads, namely heavy analytical/research workloads over fixed-size data. As I said, if GPUs become commoditized in server hardware or fully integrated with CPUs, however, I expect this domain will quickly become extremely important. This is a very cool area and its common to see 10x speedups even with optimized CPU implementations.
Always happy to chat about this stuff, feel free to shoot me a message.
- "It lies." Mongo used to have a default where a driver would fire off a write and not check that it succeeded with the server. This was very obviously a decision made to improve benchmark performance, though I imagine a benchmark with only default settings would be rather naive. Regardless, yes, the default was a stupid corporate decision but it is well known and should be apparent if you're deploying a Mongo cluster. Additionally, as the author notes, this default has changed and this entire point is no longer a concern.
- "Its Slow." A real point he raises is that its a little wonky you need to send a separate message for getLastError. I suspect this is an artifact of Mongo's historical internal structure. http://docs.mongodb.org/meta-driver/latest/legacy/mongodb-wi... . If you look at the wire protocol, I think it is designed such that only the OP_QUERY and OP_GET_MORE message types get an OP_REPLY back. getLastError is a command, which are run through an OP_QUERY message. He notes that using this check affects performance. It does, but lets dig into this:
"Using this call requires doubling the cost of every write operation." If the author benchmarked this, I suspect he would find that it vastly more than doubles the latency of a single write operation from the client's perspective. My understanding is that when performing this kind of safe write, the driver sends an OP_INSERT, for which it doesn't have to wait for a reply, then immediately sends an OP_QUERY message (getLastError), on which it hangs waiting for the OP_REPLY. In other words, this is now slower because we've created a synchronous operation immediately after just firing off the insert command. Again, its a little wonky that we send Mongo two messages, but one is immediately after the other and that is vastly overshadowed by the fact that we now have to wait for a reply. I believe 1 synchronous send and receive is unavoidable to ensure a safe write in ANY system, and the argument about sending 2 messages really boils down to sending about 200 bytes over a socket vs 400 bytes, I personally don't worry about it.
- "It doesn't work pipelined / it doesn't work multithreaded." He is also missing the real complexity here, and this is where I have a problem with this blog post because this is not a theoretical discussion, if its content is true then it should just be proven rather than FUD launched into the world. As noted in the docs ( http://docs.mongodb.org/manual/reference/command/getLastErro... ), getLastError applies for the socket over which the command was run, so its up to the driver to execute the getLastError on the same socket as the write, which is an implementation detail and a solved problem. The way the drivers do this in practice is you set the write concern and the driver takes care of the rest. If you run getLastError manually then it depends on the driver, but for Java the correct procedure is addressed at http://docs.mongodb.org/ecosystem/drivers/java-concurrency/ . So for the fastest possible safe performance, you multithread (which is effectively pipelining from the server's perspective) and run operations with the driver's thread-safe connection pool. Suffice to say people actually use these drivers in a multithreaded context in the real world, and they work.
- "WriteConcerns are broken." There are several relevant settings here, including write concern acknowledgement and fsync, the author is confused about how these map to the Java driver WriteConcern enum values. The acknowledgement setting (elegantly stored in a variable named "w", thanks 10gen) is the number of replicas that must confirm the write before the driver believes it has succeeded. I personally set this to 1, but you could potentially wait for the entire replica set to acknowledge. The fsync setting is whether or not this acknowledgement means that the server on these machines has completed the write in memory, or actually synced the data to disk. I set this to false for performance. There is an excellent StackOverflow answer on Java Mongo driver configuration at http://stackoverflow.com/questions/6520439/how-to-configure-... .
The author also spends time noting that if you only ensure the write succeeds on a single machine, and you irreparably lose that machine before replication, then the data is lost. This is obviously true for every distributed system.
I've used a 4-shard Mongo in a production multithreaded environment that handled several hundred million writes. For part of this period our cluster was extremely unstable because of a serious data corrupting Mongo bug (more on this in a second). I haven't done a full audit but based on our logs, in about 6 months I've seen exactly 1 write go missing, so I'm personally not concerned about the things mentioned in the blog post. I've also been happy with performance as long as the data size comes under the memory limit. If your data exceeds memory, Mongo essentially falls on its face, though this is hard to avoid when a query requires disk accesses.
Mongo is not without its problems, however. As I mentioned, QA is a real concern, we hit a subtle bug when we upgraded to v2.2 that caused data corruption when profiling was turned on and the cluster was under high load. It was very difficult to debug, and basically should have been caught by 10gen before their release.
Another serious problem is that sharding configuration is still somewhat immature, and it seems like every new release is described as "well it used to be bad, we finally fixed it". Here is an example: you pick a shard key that can't be split into small enough chunks, and now shard balancing silently fails. Ok, so you pick a better shard key, but you can't migrate shard keys, so you have to drop the collection and start again. Except dropping a collection distributed across a cluster is buggy, so you can't recreate the collection with the same name and a different shard key. So you just pick a new name for your collection, and you have this weird broken state for the original one sitting around forever unless you completely blast your cluster and start from scratch. This sort of thing is not fun!
Mongo has many pros and cons, personally I think its real advantage is simplicity for developers which makes it worth putting up with the other stuff. Sorry for being long winded, hopefully this has been useful.
I don't quite understand this, that architecture was basically deprecated a while back, can you elaborate?