siscia

Why?

Whenever using a model to be more effective as a developer I don't particularly care if the model is open source or closed source.

I would love to use open source models as well, but the convenience to just plug an API against some endpoints in unbeatable.

Commercial service have a context of 64k tokens, which I find quite limiting.

https://api-docs.deepseek.com/quick_start/pricing

Running it locally is quite a bit beyond the scope of being productive while coding with AI.

Beside that 128k is still significantly less than Claude

DeepSeek is an amazing product but has few issues:

1. Data is used for training

2. Context window is rather small and doesn't fit as well large codebase

I keep saying this over and over in all the content I create, the valu of coding with AI will come from working on big, complex, legacy codebases. Not from flashy demo where you create a to-do app.

For that you need solid models with big context and private inference.

I am getting quite deep into coding with AI and cost of tokens is a bit of an issue indeed.

Trivial issue because it saves me A LOT of time, but it could be an issue for new people testing it.

I would love to test this approach. Are you guys fine tuning for each codebase?

As engineers we often aim to perfection, but oftentimes it is not really needed. And this is such case.

Knowledge is organised into topic, and each topic has a title and a goal. Topics are made of markdown chunks.

I see the model being able to generate insightful questions about what is missing to the chunks. As well as synthesise good answer for specific queries.

Of course, but at least in my personal case is more about the lack of tooling.

Most of our white collar jobs are about knowledge sharing and synchronization between people.

And surprisingly this is an aspect in which I see very very little progress.

The most we have are tools like confluence or Jira that are actually quite bad in my opinion.

The bad part is how knowledge is shared. At the moment is just formatted text with a questionable search.

LLMs I believe can help in synthesize what knowledge is there and what is missing.

Moreover it would be possible to ask what is missing or what could be improved. And it would be possible to continuously test the knowledge base, asking the model question about the topic and checking the answer.

I am working on a prototype and it is looking great. If someone is interested, please let me know.

Wouldn't be simpler to just use a linter then?

How do you handle wrote concurrency?

If you different processes write on the same file at the same time, what do I read after?

Congrats on the launch!

A bit unrelated question, but what is the fastest way to obtain a similar looks and feel for the UI? Is there a framework?

I am the author of https://getgabrielai.com

Basically LLM to filter and manage emails.

I am not sure I understand why you are using a multimodal model. Why rendering and reading back the email?

How the visual aspect helps?

Hummmm while there might be some similarities the concept are not that close.

Cells are usually used to reduce blast radius and to force design that can scale.

It is a bit like a multi-AZ/multi datacenter architecture where however cells can share the same AS/datacenter. But nothing else.

So cells can share physical hardware but not logical components.

If you were to create a cell based architecture, you might have 9 cells over 3 AZ each using a different S3 bucket and DynamoDB table. For 9 S3 buckets and 9 DDB table in total.

I honestly wanted to try to use it, but their pricing was quite off-putting.

I am not commenting on the specific framework, as I just skimmed the readme.

But I find this approach working well overall.

Moreover it is easily debuggable and testable in isolation which is one of the biggest selling point.

(If anyone is building ai products feel free to hit me.)

In general I agree, but it also true that we are not using the LLM as well as we should.

The example in the article: https://chatgpt.com/share/6709a02d-b7cc-800c-882b-430bf019a0...

I barely understand it, but it looks very interesting!

I created a similar bot for telegram.

The focus was on searchable audio. So you send either a text or an audio. It passes through Open ai whisper and replies with your message transcribed.

https://t.me/Simple_DIary_bot

Just yesterday I sketch a solver for a board game: "Search for planet X".

The objective of the game is to figure out what kind of planets are hidden in sector of the boards using clues like: "There are 2 comets between secotr 3 and 7" or "No comet is next to an asteroid ".

Sketching the solver was incredibly fun and rewarding!

The article mentions that some primitive of GCP are nicer (than AWS?).

I wonder if you guys have any example of those nicer primitives.

We have been discussing a similar idea with friends.

The topic of knowledge synthesis is fascinating, especially in big organisations.

Moving away from fragmented documents into a set of facts from which LLM synthetize documents from, tailored for the reader.

There are few tricks that would be interesting to have working.

For instance the agent keep evaluating itself against a set of questions. Or user adding questions to see if the agent is able to understand the nuances of the topic and so if it can be trusted.

(Not dissimilar to what would be regression testing in classical software engineering)

Then the "homework" sections, when we ask human experts to evaluate that the facts stored by the agents are still relevant and up to date.

All these can then be enhanced with actions usable by the agent.

Think about fetching the PoC for a particular piece of software. It is the employer Foo.

If we write this down in a document, it will definitely get outdated when Foo move, or get promoted.

If we put this inside a knowledge synthesis system, the system itself may keep asking every 6 months to Foo if it is still the PoC for the software project.

Or it could daily talk with the LDPA system and ask the same question as soon as it notices that Foo has changed its position or reporting structure.

This can be expanded for processes to follow. Report to create, etc...

The more I think about the less I am completely against this approach.

Instead of applying an obscure set of heuristic by hand, let the LM figure out the best way starting from a lot of data.

The model is bound to be less debuggable and much more difficult to update, for experts.

But in the general case it will work well enough.

Can I get a pricing quote?

I was trying to express exactly what you say!

Sorry if it wasn't clear.

Not only the intention, but also the reason behind it.

My issue is not explaining how the code work. My issue is understanding why the code exists at all.

The default workflow is to follow the commit history until I don't get to where and when the code in it's current shape was introduced. Then trying reading the commit message that generally link to a ticket and then acquire from tribal knowledge of the team why it was done like that. If it is still necessary, what can we do today instead, etc...

And similarly when designing new code that needs to integrate on existing piece of code... Why there are such constraints in place? Why was it done like that? Who in the team know best?

The main issue with these set of tools is that I mostly read and understand code more than writing it myself.

Not enough attention is been given to this imbalance.

It is impressive having an AI that can write code for you, but an AI that helps me understand which code we (as a team) should write would be much more useful.

Thanks for the clarification, this makes more sense.

The thing I don't understand about all of this is another, much less technical and much more important.

Why the blas radius was so huge?

I have deployed much less important services much more slowly with automatic monitoring and rollback in place.

You first deploy to beta, where you don't get customers traffic, if everything goes right to a small part of your fleet, and slowly increase the percentage of hosts that receives the updates.

This would have stopped the issue immediately, and I somehow I thought it was common practices...

I have been building [GabrielAI](https://getgabrielai.com) also for address the too much spam in Gmail use case.

Specifically smart filter to remove SPAM in a smarter way.

Most people get a lot of spam from sales agents, SEO services, start-up accelerator, etc...

With GabrielAI you can say stuff like:

"If the email is from a SEO agency or it is trying to sell me SEO service"

Then move it to SPAM.

Similarly for all other type of spam or emails.

You can also move stuff to different labels in Gmail to organise your inbox.

In Java it would be something like JBMC?

I read until the appendix but I find the intro wrong.

From my experience big tech interviews are not for making sure that candidates don't write O(n^2) loops. Code reviews are for that, some of the times.

Those interviews are there because we haven't figured out a better way to interview yet and this seems somehow correlated with the job. I don't think it is correlated, but it looks like it is!

Those interviews have also the hidden characteristics (your call of it is an plus or a minus) to select for quite confident engineers that usually can talk and defend their opinion. Which is quite useful in big tech.

For the reason why this does not get fixed.

It is not because no-one looked. People in the team know. We all know.

But we are being told not to focus on them.

With scales comes cost of running the whole machinery AND opportunity cost of not having the next product launched. With the scale of big tech, the opportunity cost easily dominate the cost of 1% increase in performance.

People don't realize it, but it is working well and as designed.

It is wasteful? Absolutely.

It makes a shit tons of money? Definitely!