didgetmaster

>My only caution is that good writers and LLMs look very similar, because LLMs were trained on a corpus of good writers.

While that might be ideal, is that really the case with most LLM training data? Does the curation process weed out all the slop from bad writers?

The article mentions music and movies as following a similar route; but why no mention of software?

Once upon a time, a small startup could build something and get off the ground by selling a few thousand licenses at $20 apiece. Nowadays, it seems like version 1.0 has to be a bestseller or no one will touch it.

If I knew that a company had paid a refundable fee (e.g $1000), that was only paid back if the job was filled by someone outside the company within 6 months; then I would take it much more seriously.

I always thought one of the best promises of AI was to be able to feed it a 'jargon heavy' document (business contract, corporate policy, congressional bill, etc.), and have it spit out the real meaning in plain language.

"Don't sign this because you are screwed if x, y, or z happens!" would be a good summation in many cases.

Has anyone had much success with this kind of AI interaction?

The article seems to imply that only the ones are real data. In fact every zero is just as important as a one.

I have a project that has been in 'perpetual beta' for years so it is a free download (25 MB zip file).

It runs completely on the user's computer so there is no service to maintain.

It is a new kind of data management system that was originally an object store to replace conventional file systems; but the tagging features I designed made it useful for creating, querying, and analyzing relational tables.

It is a hobby, so I like seeing how much faster I can perform operations than regular RDBMSs. It is extremely flexible, so lately I have been testing it out using large data sets. Creating tables with 100,000 columns or doing a pivot table in a 227M row table is fun for me.

See my profile for links.

We have dealt with this in the political realm for decades. Vague word salads are being passed off as coherent policy arguments every where we turn.

After spewing some vague policy position, a politician's worst fear is that a reporter might ask "What do you mean by that?" Or "How would that work in real life?"

>It feels closer to how social networks used to work originally. I hope it stays that way.

Is there something about it (it's architecture or the company behind it) that is fundamentally different than other social networks? If not, it is doomed to follow them all eventually.

It's not just the tools, it is your tests. Most times you encounter and fix a bug, your first question should be 'Why didn't my tests catch this?'

If you asked that question to 100 random people on the street, I wonder how many would respond with 'walk'. Proper reasoning is not just a problem for LLMs.

For those who maintain FOSS projects: How often when a user requests/demands your attention to support/fix/enhance the project to suit their particular needs, do they actually pay you something once you explain that what they are asking for, comes with a price?

But also be aware of 'analysis paralysis', where you spend so much time trying to plan for every contingency that you never get around to making the actual decision.

Like many technical people, my personality naturally gravitates towards introverted behavior. I am very comfortable being alone for long periods of time, and in spending my day writing code or fixing bugs with just me and my computer.

But I also enjoy interacting with other people. It just takes the right 'community' to draw me out of my shell. There have been periods of my life when I was outgoing, because I was in the right environment (college, certain jobs, sports, etc.). Other periods allowed me to retreat into my own isolated world.

There just isn't a magic formula that produces the right kind of community that we want on demand.

Every major life choice (career, marriage, buying a house, moving to a new city/state, etc.), comes with a set of pros and cons. Having children is no different. No matter which choices you make in life, you will always wonder if the other choice might have been better.

Raising a family is hard, but also has many rewards. I have 4 children (now grown) and never regretted it, but I try not to judge others who have made other choices.

You should not have children for your own benefit. Those who expect children to take care of them in their old age, might be disappointed. If you are expecting to get out of them more than you are willing to put into them, you are doing it wrong.

Obviously. I was just wondering how it does with huge numbers of files.

How does it perform if you have one of those 30 TB hard drives filled with files (100 million+)?

Are you saying that you think that YouTube was fast, easy, and cheap to build?

Anything (software or physical things) that is fast, easy, and cheap to build; will never be a financial success for a single company. The minute you get some market traction, your competitors will come in and take away all your customers.

I hate websites that hide charges until you have spent significant time entering information. There should be a law!

Everyone assumes that they collect and store personal information (to use or sell later), even when the user abandons the process after finding the true cost. There should be another law!

Crops are a commodity where you can't instantly ramp up or down the supply to meet demand. Most require the better part of a year from seed to harvest. If it grows on trees, it can take years before they produce.

Forecasting crop output can also be tricky. Weather conditions, pests, or other things can lead to failed crops or bumper crops.

The life of a farmer can literally and figuratively be 'feast or famine'.

The wheel seems like a pretty big one.

This is why I rarely join those 'customer loyalty' programs that almost every store offers.

I say 'no thanks' when the cashier wants to know my phone number, even if I am paying cash.

It is almost impossible to remain anonymous in the consumer space, even if you are really trying.

Interesting. Are you willing to try out some 'experimental' software?

As I indicated in my previous post, I have a unique kind of data management system that I have built over the years as a hobby project.

It was originally designed to be a replacement for conventional file systems. It is an object store where you could store millions or billions of files in a single container and attach metadata tags to each one. Searches for data could be based on these tags. I had to design a whole new kind of metadata manager to handle these tags.

Since thousands or millions of different kinds of tags could be defined, each with thousands or millions of unique values within them; the whole system started to look like a very wide, sparse relational table.

I found that I could use the individual 'columnar stores' that I built, to also build conventional database tables. I was actually surprised at how well it worked when I started benchmarking it against popular database engines.

I would test my code by downloading and importing various public datasets and then doing analytics against that data. My system does both analytic and transactional operations pretty well.

Most of the datasets only had a few dozen columns and many had millions of rows; but I didn't find any with over a thousand columns.

As I said before, I had previously only tested it out to 10,000 columns. But since reading your original question, I started to play with large numbers of columns.

After tweaking the code, I got it to create tables with up to a million columns and add some random test data to them. A 'SELECT *' query against such a table can take a long time, but doing some queries where only a few dozen of the columns were returned, worked very fast.

How many patients were represented in your dataset? I assume that most rows did not have a value in every column.

Is there really a market for these kinds of relational tables?

I created a system to support my custom object store where the metadata tags are stored within key-value stores. I can use them to create relational tables and query them just like conventional row stores used by many popular database engines.

My 'columnar store database' can handle many thousands of columns within a single table. So far, I have only tested it out to 10,000 columns, but it should handle many more.

I can get sub-second query times against it running on a single desktop. I haven't promoted this feature since everyone I have talked to about it, never had a compelling use for it.

Once upon a time, you could enjoy the works of a creative person at face value; mainly because you didn't know much of anything about their personal life.

That seems to have all changed in this age of the Internet; where every aspect of your life is exposed for all the world to judge (at least if you are famous). All your words (written or spoken) are presented as proof positive that you and your works are not to be tolerated; even if they are from your teenage years.

It seems like you cannot say anything these days without offending a large number of people; some of whom will try to lead a boycott against you.

I generally like to enjoy a good book, movie, blog, or comic strip without letting politics get in the way.

When I was single with no kids, I felt pretty comfortable leaving a good job to join a startup. I took a 50% pay cut to join when the risk seemed high, but the reward also seemed high.

It paid off for me, but who knows if I would have taken that leap later in life.

Although almost every company issues a 'we care about your privacy' statement, but there is often very little 'money where your mouth is' resources to back that up.

This is why I am almost always very reluctant to give out any information that is not absolutely necessary to provide me the service that I need. If they don't know it, they can't leak it.

Every company wants you to fill out their standard form that tries to get you to volunteer way more info than they really need.

So the law requires a fridge be provided, but doesn't state what kind. So now renters can expect to move into an apartment with a tiny, old fridge that does not suit their needs. Then they have to find a different fridge that will hold more than a half gallon of milk, as well as find a place to store the provided, unusable fridge until they move out.

Is it just books that you expect to be free? Or does your advice apply to every song, painting, movie, and program?

You expect every artist, actor, painter, and programmer to make zero $ from anything they create?

So your fist attempt was an online coding course where you would train users and they agreed that once they got a job, they would pay you a portion of their salary. But no one would honor their agreement once they were trained and employed.

Did I read that right?