krishadi

Gotcha, do you also provide functions to build and store docker images from a code repo?

AWS Copilot has been great for deploying `services`. But I wasn't sure if the `jobs` were what I was looking for, since they are event triggered. I'll test it our now.

Essentially I'd like to build a docker image of code from a repository, and deploy and run it, and manage its lifecycle. Perhaps I could copy the CF template from the Copilot to do the same.

I am looking for a service that helps me orchestrate the deployments of docker containers in AWS. These containers run Python workloads, but are not web-api endpoints. I want to be able to deploy them programmatically and, pause them if they've been idle (something like what supabase does). Is this something you are looking to provide?

From a quick glance at your docs, it seems that you are mainly focussing on web facing applications.

Author of PDX here: I've built quite a few applications powered by LLMs. This is the mental model I use to build and manage prompts to interface with the LLMs. It greatly helps me and my partner work an iterative on the prompts, and the chain/tree of thoughts methods of prompting. At the same time, keeping track of how these prompts performed.

Thanks.

https://krishadi.com/ : most of the stuff here are a knowledge base. I am starting to write more thoughts and observations now.

archive.is worked for the articles I was looking to read. This would at the least help me evaluate which publishers I like more.

Oops, I meant to ask the author but realised that the author is not the same as the op. Hah!

I've been recently reading up on multithreading and multiprocessing in python. You mention that you've taken a multi threaded approach since the processes are i/o bound. Is this the same as running the script with asyncio as async/await?

How does one here manage container deployments to AWS?

I have been using AWS Copilot CLI, and it's awesome! They've just made it to so easy deploy and maintain applications online. To think of it, it's quite a complex orchestration, and yet it feels like child's play.

Second that!

Claude isn't super fast either. It's about a third or so.

The ease of funding opportunities these days. Startups start to please their source of money. And, it rather seems that those are investors/grants rather than customers. My assumption is that, it's easy for the MBA-types to please VCs/Grant-agencies. And, so you have the outcome...

Exactly! The audience is not a standing army but a moving parade. Rehashing something Ogilvy said.

They are not all computationally expensive. The rate limiting step here is the LLM call itself over the API. So, async is definitely needed. The other aspects would be loading the template from filesystem. I would assume this could be something that's needs to be optimized in the application.

Exactly. The most important part was working with Jinja templating. So, openai + jinja2.

This is exciting! Does that mean that I can load a sqllite db or a duckdb onto the browser through wasm and then connect it with the python env?

Great! Thanks...

Simple answer: don't build anything yourself if you can find something that fits 80% of your use case.

The use-cases and tooling around language models is very premature. So, any framework you build now will either look like bloatware or will remain close to just calling an API.

The dust around language models needs to settle a bit, for a useful framework to emerge from it.

For our own use-cases, I built a framework from scratch, and it was the best decision we made.

Have you been using promptlayer? How does it fare for you?

Just some thoughts on it. The op initally talks about data warehouses, but then in the comments they talk about production db. So, I am not sure if I completely understand the problem statement:

1. Why would you want to do data analytics on a production database? There is a reason why data warehouses exists. And you don't want to block your production databases for someone from the business team to run a query. 2. If you'd want to change something in your prod-db, I wouldn't give that permission to a non-technical person. Do they know about the schema thats implemented? data types? 3. If it's just analytics on data warehouses, there are plenty of db visualization tools out there, the so called BI tools. There are plenty, I mean too many out there. In different flavours. What makes this different?

Compared to Code + Docker + AWS Copilot CLI, am I going to see a 10x improvement in my workflow?

Completely agree! If I want to add TailwindCSS to NextJs, which I would assume a lot devs are doing, I have to run many setup scripts and edit files, I think this is redundant. Could be easily automated, since most of it is standardised.

If you are a web-developer, building application that store username, email, password and all that stuff, where you'd have to do multiple row level fetch of the data with a lot of parallel connections then you'd use postgres or the like.

If you are a data analyst, analysing a lot of data, which isn't updated in real time, where you'd have to do joins, aggregates which are usually column wise functions, you'd use something like a OLAP db, where duckdb is great for that!

Is there a roadmap on handling geographic data?

I wonder how My Maps is going to evolve, or not for that matter. The ability to customize the maps, and edit existing data is really painful. That makes me wonder, what was the goal of it all? Started out as a competitor to ArcGIS, but has now evolved to be something in between of a lot of existing software, but not a complete replacement of any.

Most new things I learn, are somehow related to solving problems that have piled up in the years. So, I just pick out that problem, try solving it with this new thing I've learnt. I start with watching a video on its simple usage or, read some sort of documentation on it. I wouldn't per say call them rules, but more of a guideline.

Is duckdb-wasm also going to get this update soon?

Is there a reason you are learning it now?