You aren't getting what I'm saying. logN is already a huge optimization for a hot path. Look at that pic: https://dev-to-uploads.s3.amazonaws.com/i/ya7xk4n9ch50xgwibt...
Seriously look at O(logN) and look at NlogN. Databases everywhere already take logN as acceptable. You are optimizing past that and introducing an NlogN sort step that will block all hot path calls because redis is single threaded. I don't think it's a to each their own thing here, i believe the user experience metrics will definitively play out towards my methodology but we can't know until we actually have metrics so we're at an impasse here.
>This assertion is so wide that it cannot be true. No matter what language you're in, you can absolutely do sorting in the API layer.
No this is absolutely true. Especially if you're a nodejs programmer. Nodejs is single threaded, sorting imposes NlogN penalty on a single thread.
Backend web development for the api layer is all about transmitting as much compute as possible to the database layer. You can get away with small sorts and things can still work but in general the practice of backend optimization is indeed to let the database handle as much as possible.
We can debate it but from my experience I know I'm pretty much categorically right on this. It's a fundamental principle. And it has special application to things that use Async IO like NodeJS. That's why there's huge emphasis in nodejs in writing non-blocking code. A sort is a potential block.
There are very few cases where you would handle compute on the api server rather then the database. The first instance is obvious, if N is guaranteed to be small which is not the case in the takehome. In other instances of sort with large N, the server ideally needs to launch that sort in a separate non-blocking thread so that the server can continue handling oncoming requests. In nodejs, to my knowledge since I last used it, you can't easily spawn a thread at will, you just have to pray you have available workers that aren't blocked by a sorting operation. In python the GIL makes this more complicated, but slightly better then nodejs.
For node you probably have 8 workers on 8 CPU cores. If you have 100 requests/s, all you need is 8 of them calling a sort on large N and all threads become blocked.
Don't believe me? Read the official docs of nodejs: https://nodejs.org/en/docs/guides/dont-block-the-event-loop#...
NlogN stands at the cusp here. It could work for small N, but for large N it will fail. Anything above NlogN is a huge risk.
>You don't need to pipeline if the operation is O(1) -- it's not a question of whether transactions are good/bad, I don't know why you're assuming people don't know about them.
This is categorically false. The pipeline operation as I said HAS NOTHING to do with O(1) or O(N).
This is essentially the operation:
set["key"] += 1
# another thread can update the value here leading to print displaying something unexpected.
print(set["key"])
>Again, this is a matter of what people think is hard and what isn't. Integration testing does not require that you spin up docker compose -- I literally linked to a libraries that do this in a reusable (importable) way w/ docker.
You linked? Where? I'd like to know about any library that will do this. Tell me of any library that does integration tests that spins up infrastructure for you. The only one closest I can think of that you can run locally is anything that would use docker-compose or some other IAC language that controls containers. I honestly don't think any popular ones exist.
>If docker was the problem, you can open subprocesses and assume that redis is installed in the test environment (i.e. your laptop, or CI environment), or take a binary to the redis binary path from ENV.
No way I'm going to assume the user has redis installed on his local machine. Many devs don't. It's all remote development for them or everything lives in containers.
Docker is not the problem. Docker or virtual machines makes this problem more amenable to a solution, but even using docker here with testing is overkill and hacky. A take home should not expect a user to build excessive infrastructure locally just to run tests.
>You said you don't do this kind of testing regularly, so it is understandable that it's not easy for you.
I'd like to know your perspective. How is spinning up infrastructure for integration tests easy? You have to write code that launches databases, launches servers and views the entire infra externally. Do-able? Yes? Easy? depends on what you call easy. As easy as unit tests? Definitely no. Easy enough for a take home? In my opinion, no.
Maybe for this take home project you could be right. I could do some integration tests by spinning up docker-compose from within python. Hacky but doable. But in general this solution is not production scalable as production involves more things then what can be placed inside a docker-compose.
>You can even remove the burden of running redis from the test suite and just make sure it's running manually or in test suite invocation/setup. There are many options, and it's just not that hard in 2023.
So I should expect the reviewer to have redis running on his local machine? I would expect in 2023, I should have the entire project be segregated from the context it's running in. That's the cutting edge I believe.
Yeah it's 2023, you tell me how integration testing should be done as easily as unit testing on a takehome. I had one takehome project involving S3 and aws lambdas. They expected me to get an AWS account and literally set up infrastructure because there's no way around even testing what I wrote without actual infrastructure. That entire project was just integration test nightmare. Much rather run asserts locally.
>I referred to the context of the project -- you did* have full control.
Well I mean 99% of hires get into a context where they aren't in full control. So why is it logical to test what a developer will do if he/she had full control? Isn't it better to test the developers ability to code and to adapt to different contexts? Your methodology sort of just tests the programmers personal philosophy and whether it matches your own. That was my point.