didgetmaster

As much as people want to believe that there is a tried and true formula for success, there isn't one. Many self-help books and videos want their customers to believe there is one, but if you get 100 determined and talented individuals to try their formula; you may get 100 different outcomes.

There are certainly traits and tactics that can enhance your chances for a good outcome, but nothing is guaranteed. Every story of success or failure is just another anecdote.

So...somehow communists and socialists are better stewards of the Earth's resources???

What if AI can greatly reduce the amount of money (i.e. tax receipts) that the government needs to function properly? What if AI can quickly determine who really qualifies for government assistance, and cuts off all the fraudsters without needing an army of bureaucrats?

We could even cut out a ton of middle-men and return congress to a part-time job. One can only dream, right?

Surprise, Surprise. LLMs will respond according to the set of data that their model was trained on!

While just about every LLM is trained on data that far surpasses the output of just one person, or even a decent sized group; it will still reflect the average sentiment of the corpus of data fed into it.

If the bulk of the training data was scraped from websites created in 'WEIRD' countries, then it's responses will largely mimic their culture.

I developed a new database management system and I needed a GUI application to use as an admin tool for it.

I decided to build it using Qt (Qt Widgets in c++) mainly because my whole data engine is also in c++. Since it just uses standard windows and dialog boxes; I haven't felt the need to keep up with the latest Qt version. I am still using Qt 5 (I think revision 13 or 15).

I have been contemplating moving to Qt 6. Have users noticed a big difference (e.g. performance) between Qt 5 and Qt 6?

Reminds me of the old saying: 'If you have just one watch/clock, then you always know what time it is; but if you have two of them, then you are never sure!'

Makes you wonder if the reason why some trivial bug in a closed source project goes unfixed for years; is because all the engineers are afraid to touch the code in some obscure library and instantly become its new 'owner'.

I wonder how effective it is for an OSS maintainer to try to prevent someone from 'stealing their project' when <corporation> doing the fork is huge with plenty of resources (engineering, marketing, and legal) vs just some startup that is trying to gain some traction.

I have a side project that I have worked on for years, mainly because I enjoy writing software. I get an adrenaline rush when I can make my data management system do things better and/or faster than other things on the market.

I have written articles about it and made the binaries freely available on my website under an 'open beta'. People keep telling me that if I really want it to take off, I should open source it.

So far, I have resisted doing that, for many of the reasons that you cited.

If you are bilingual, with one language being stronger than the other one; try reading something in the language you are least comfortable with. I do this (e.g. reading LoTR in Dutch) and it forces me to pay closer attention to each sentence.

There are words you don't know or know how to use them properly. It will help you learn the second language better, while also helping you to not gloss over whole passages.

Back in college (for me the 80s), I learned that storing table data in rows would greatly increase performance due to high seek times on hard disks. SELECT * FROM table WHERE ... could read in the entire row in a single seek. This was very valuable when your table has 100 columns.

However; a different query (e.g. SELECT name, phone_number FROM table) might result in fewer seeks if the data is stored by column instead of by row.

The article only seems to address data structures with respect to indexes, and not for the actual table data itself.

The lack of needing the last name might have allowed a hacker to brute force the whole list; but it seems that even with a last name, it could expose a lot of PII. Just pass codes along with popular last names (Smith, Jones, Nelson, etc.) and it seems like it could spit out a bunch of reservations.

I still have a 25 year old HP LaserJet printer that has a built in FAX function. Unfortunately, I haven't been able to use it since we gave up our landline about 20 years ago. Still prints well, however.

It is very much like crime in general. The vast majority of crimes committed each year are by a tiny minority of people. Criminals often have a rap sheet as long as your arm; while a huge percentage of the population has never had a run-in with the law except for a few traffic or parking tickets.

While crime is definitely a major problem, especially in big cities; it only takes a few news stories to convince some people that almost everyone is out to get them.

There is an old saying that you are as happy as you choose to be. While some people have experienced very challenging circumstances and have trouble feeling happy; most miserable people are that way because they let little things get in the way of their happiness.

There are rich, healthy, popular people who feel awful. They might feel like a failure because they are constantly comparing themselves with more successful people (or at least believe all the wonderful posts on social media). They might immerse themselves in negative thoughts about the world and their own immediate surroundings.

But if you are always counting your blessings and trying to serve people who are less fortunate; you might realize that 'It's a Wonderful Life'.

If you re-read my comment, you will notice that I said nothing about how much taxes are paid, or what those tax dollars are used for. I was talking about how much time, energy, and capital is needed just to figure out how much you owe.

How much effect has the regulatory environment had on overall productivity? How much of a company's (or individual's) resources are wasted trying to jump through hoops that do nothing to increase output?

Tax compliance. Defending against frivolous lawsuits. Chasing permits. Settling labor disputes. Sensitivity training. Wading through government red tape.

Each of these and dozens of others drain resources (time and money), but contribute little to productivity.

Sometimes it feels like there is an awful lot of software out there that shipped without much review. This was happening long before AI arrived on the scene.

Hard to tell if anyone was 'comfortable' with that.

The open source world is littered with all kinds of specialized licenses that came about because a developer once said "Here is my source code. Do anything you like with it! ... wait! Except for that!".

I don't know if there is, but I wouldn't be surprised if there is a license available that restricts you from porting the code to MS Windows or any other proprietary OS.

Several of your assumptions are correct.

My project Didgets (short for Data Widgets), started out as a file system replacement. I wanted to create an object store that would store traditional file data, but also make file searches much faster and more powerful than other file systems allow; especially on systems with hundreds of millions of files on them. To enhance this, I wanted to be able to attach contextual tags to each Didget that would make searches much more meaningful without needing to analyze file content during the search.

To facilitate the file operations, I needed data structures to support them. I decided that these data structures (used/free bitmaps, file records, tags, etc.) should be stored and managed within other Didgets that had special handling. Each tag was basically a key-value pair that mapped the Didget ID (key) to a string, number, or other data type (value).

Rather than rely on some external process like Redis to handle tags, I decided to build my own. Each defined tag has a data type and all values for that tag are stored together (like column values in a columnar store). I split the tag handling into two distinct pieces. All the values are deduplicated and reference counted and stored within a 'Values Didget'. The keys (along with pointers to the values) are stored within a "Links Didget'.

This makes analytic functions fast (each unique value is stored only once) and allows for various mapping strategies (one-to-one, one-to-many, many-to-one, or many-to-many). The values and the links are stored within individual blocks that are arranged using hashes and other meta-data constraints. For any given query, usually only a small number of blocks need to be inspected.

I expected analytic operations to be very fast, like with other OLAP systems; but I was pleasantly surprised at how fast I could make traditional OLTP operations run on it.

I have some short demo videos that show not only what it can do, but also benchmark many operations against other databases. Links to the videos are in my user profile.

Although I have done many more benchmark testing against other databases for query speeds; I haven't noticed any significant speed degradation on writes.

Could you clarify what you mean by 'this limits your write capabilities'?

I am not an expert on all the other columnar stores out there; but it is my understanding that they are used almost exclusively for OLAP workloads. By 'regular database tables', I meant those that handle transaction processing (inserts, updates, deletes) along with queries.

My system does analytics well, but it is also very fast with changing data.

I also think that some of those systems (e.g. Duckdb) also use indexes.

This article points out some of the pain associated with index maintenance. It should also point out that ALL indexes on a table suffer from the same issue. If your 20 column table has 7 indexes, then the suggestions should be applied 7x.

It is conventional wisdom that indexes are absolutely essential for any relational table of at least reasonable size (e.g. thousands of rows) and is accessed more often than daily. Indexes can be a pain to create and maintain; but can greatly speed up queries and primary key validations. The pain mostly comes from having to figure out what indexes to create and how often to maintain them, rather than doing the actual thing.

Indexes also have a performance penalty for any table updates. Creating new rows, updating existing rows, or deleting rows all require updates to each index.

But are indexes really required? I am creating a new kind of general purpose data management system (a kind of object store) called Didgets. The tagging mechanism that I invented to allow tags to be attached to each data object, are key-value stores that essentially form a set of columnar stores.

I found that these columnar stores could also be used to create regular relational database tables. The data is structured such that indexes are not needed. All the tests that I have run (up to a thousand columns with over 100 million rows), show that query speeds are equal to, or better than other database systems that are well indexed.

The system is still under development, so it is still missing some key features that would make it a drop-in replacement for other databases; but it proves that it is possible to structure relational data such that query speeds can be optimal without needing separate indexing structures that have to be maintained.

But where do you draw the line? Not buying a game that was created by the Nazi party to promote antisemitism is one thing. Shunning a text editor because one of its creators once said something you disagreed with politically, is another.

I said I don't care about the backgrounds of the creators of tech that I use.

I do care when other people in tech insist that I should care, and that I should make decisions based on the identity of certain individuals.

This guy lost me when he started talking about diversity issues instead of tech. I couldn't care less about the race, gender, or sexual orientation of the person(s) who created the hardware or software that I use. Does it work? Is it easy to understand and use? These are the things I am interested in.

I am reminded of an early cartoon of a dog sitting at a computer saying 'On the Internet, no one knows you are a dog!'

It seems to me that there are many people in almost every field who are more interested in what you can do for them, than in working with you collaboratively to build something meaningful.

I don't know that there are more or less of them in tech, than in other fields.

I upgraded my pc about this time last year (new CPU, RAM, and motherboard).

I was originally going to just get 64GB of DDR5-6000, with the option of adding another 64GB later, thinking the price might drop even further. At the last minute, I decided to get the whole 128GB instead. Glad I did.

How many great opportunities are passed up by people waiting for the 'perfect' one to come along?

Job offers passed on because it wasn't the perfect fit. Marriage chances missed because the perfect soul mate must still be around the corner. Good schools rejected because it wasn't their dream school.

Luck often accompanies those willing to accept less than perfect in many cases.

Where is the Earth in its orbit around the sun when this event happens? For half of each year, the sun is closer to Voyager 1 than is the Earth.

Also, does anyone know how long communication with the probe is disrupted when the sun is directly between them?