11 karma · joined January 23, 2020
I think I read something similar in Sapiens.
About this app, (django,react,tfjs) It takes a twitter handle as input and lists the most recent tweets along with buttons for each to predict toxicity using tensorflowjs's toxicity model (pretrained) I plan to make a twitter bot with my learning so far. Also this project initially had a much larger scope and I had to cut down/scrap most of it as i'm using only freely available services to host this and running ml on such services did not seem fair to me.
followed by further reading on this https://www.ciosummits.com/media/solution_spotlight/TDWI_Che... And possibly sql query optimizations. In most cases, efficient indexes should be enough to maintain reasonable performance.
But in cases where scale gets exponentially bigger at a rate at which you cannot possibly keep adding computing resources, breaking the best followed practices is the only way to get more performance without adding hardware resources at the same rate as increased scale, this at times includes but is not limited to, getting rid of foreign keys, coupling your app logic along with your db to to breakdown larger tables periodically/automatically/ runtime creation. This though requires skills across the entire stack and has it's fair share of downfalls and is never a recommended practice unless you know what you're doing and are okay with compromising on some of ACID properties. since maintaining something like this becomes unsustainable without contingencies. Take all of this with some grain of salt as the topic is highly subjective.
If you have to break it down in terms of raw basics, what approach you choose largely depends on what you want to achieve. Performance or security. wanting both requires large amount of human as well as computing resources. As a general example, consider node or apache, node gives you a performance improvement in terms of raw latency and concurrency by a factor of ~20-100 (based on a few simple tests i've done using apache bench) in most scenarios compared to apache but in terms of reliability, if your code for any reason doesn't handle exceptions, it takes down your entire server/service. With apache, that wont happen, your perfomance degrades, some users might get affected but the service as a whole would still be operational. This is just an example, most of the technology out there is about making the choice on what you're okay with comprimising with the available resources (human and computational) that you have. This is just my view on it.