diamondap

I read an interview a few years ago where a reporter asked a writer what was the difference between authors who made it and authors who didn't. The writer said that the ones who made it stuck with it. They just kept on writing despite years and years without strong sales or recognition.

If you're going to stick with it, there has to be something there beyond a desire for sales or widespread praise.

Success, too, has its downsides. My wife is a painter--and if you think it's hard to make a living at writing, try painting. She met another painter who makes a huge income, but that painter complained, "My first sales were paintings of boats. And for years now, everyone comes to me wanting a boat. I'm bored to death of them. I want to paint other subjects, but this is what pays, so I keep doing it, as frustrating as it is."

Look into fiction writing, and you'll find few writers whose publishers will let them write outside their genres. Those writers may want to explore, but the publishers know what has sold in the past, and they want more of that. The only major author I can think of who sells big in multiple genres in Stephen King.

Self-publishing has been a godsend for those who like to explore. You can write anything you want and push it out at will. You might not make money at it, but you can do it.

At bottom, you have to choose your definition of success and failure. If you define success as being accepted by editors and critics, receiving praise and prizes from the publishing community and hitting the bestseller lists, all of that is beyond your control. It's like fishing for likes on Instagram.

If you define success as getting better at your craft, writing something that you and the few people you really respect like, then you can find fulfillment. I often think of gardeners who pour hours into their roses, not to get rich or to receive recognition, but just because it pleases them and fulfills a deep inner appreciation.

jasode's answer is spot on. At work, we run an digital preservation system that ingests large file sets into S3, Glacier and Wasabi. The files have through a pipeline of microservices to verify integrity, identify file formats, and extract other metadata.

We use AWS Elastic Container Service (ECS) to ensure all of the services are running and to scale when necessary. We use NSQ to make sure tasks are sent and re-sent to workers until they complete the entire pipeline. And we use Redis to store interim processing data from each worker.

Keeping that interim data (state) in a place where all workers can access it is key. ECS kills workers when it wants, and workers occasionally die due to out-of-memory exceptions. With the state info stored in Redis, a new worker can pick up any failed task where the last worker left off.

This system has been running well in production under heavy load. It replaces an older server-based system that used different technologies to handle the same responsibilities: supervisord to keep processes running and BoltDB to store interim processing data. That system worked, but could not scale horizontally because BoltDB is a local, disk-based store. Distributed workers need a shared, network-accessible store to share state info.

You'll find a detailed overview with diagrams of the new system at https://aptrust.github.io/preserv-docs/overview/

There's a shorter write-up of the goals and how we achieved them at https://sheprador.com/2022/12/architecting-for-the-cloud/

This stuff isn't too hard, as long as you get the pieces right. Try to stick to fredley's advice. Keep things simple and you'll save yourself many headaches going forward. Also, be sure your workers handle SIGTERM and SIGKILL explicitly, cleaning up their work as much as possible before they die.

I once had a job like that, though my situation wasn't quite as bad as yours. The job started out well and then went sour when they hired a new manager. After years of 70-hour weeks, I couldn't remember what a decent job was like, nor could I imagine a job being better elsewhere.

When I finally did make the jump to a new position, I landed in a great company, with a great manager, and everything turned around.

One thing I'd advise, now that you're in a better place, is to try to work past the resentment. As long as you carry that in your heart, the old boss still has a hold on you. It'll take time to get it out of your system, but do try to get it out. You'll find that a more positive and productive attitude creeps in to the space that resentment once occupied, and your level of happiness goes up another notch.

In case you're interested, I posted about that job gone sour at https://sheprador.com/2022/04/worst-manager-ever/. The guy wasn't mean. He just didn't care. And writing the post made me laugh.

In the old days (1990s), it was a "bad idea" to use CGI for heavily trafficked sites because there was a lot of system overhead involved in spawning a new CGI process for every request, firing up an interpreter (usually Perl), parsing the code, then executing it.

As one of the previous comments mentions, operating systems do a better job of caching now, so there's less overhead there. Fast CGI also gets rid of that problem by running your CGI script as a single persistent process.

CGI may not be the best solution for really huge applications that have hundreds of routes or very complex business logic.

That said, it's still incredibly useful. I use it on some personal sites, and it's nice to know that I can drop it into any new site without having to worry about dependencies and without having to do much configuration. It's simple and it works. Those are two big pluses in a world where things have gotten so complex.

That's actually pretty good for a first book. Congrats on that.

Blogging is good practice for clarifying and sharpening your thoughts. And going into it with the idea of collecting the works into a second book is good motivation to keep going.

I've published eight novels. It's hard to earn money on fiction, unless you're good at marketing and you're writing a series in a genre that has a lot of binge readers (like romance, mystery, thriller). I do it mainly to satisfy an itch. I get an idea, and I want to get it out on paper.

Non-fiction is easier to sell, if it provides value. You can charge higher prices for programming books than for mysteries, and people will pay because the knowledge they gain from the book will be worth the money.

If your goal is to earn money from writing, please know that even if you produce excellent work, marketing is key. Identify your audience and find ways to connect to them. Have at least a minimal marketing plan in place before you publish.

The post mentioned by surprisetalk, https://news.ycombinator.com/item?id=34250688, was posted about the same time as yours. That guy had a (very) minimal marketing plan, and it worked out pretty well for a first book.

What's the website?

How does Storj ensure long-term availability of data? If a number of nodes drop out, or their hardware goes bad, you may not be able to reconstitute your files.

Or maybe long-term storage isn't what Storj is for?

Yes. Password manager with all essential entries shared between spouses. Plus a written document describing how to get into the password manager should we both go. We shared the location of the document with key people.

Still, there's probably more we could do, and a number of bases left uncovered. For example, we each have a number of monthly subscriptions that are auto-drafted but won't need to continue after death. We should identify those and have cancellation plans.

Plus we both have lots of crap, and possibly some important in various online/cloud storage services. Even with password access, it would be hard for survivors to know what to look at and why.

And then there are the accounts with two-factor auth. What if one of us goes with our phone? Oy!

I started by reading the source code of Amazon.com's homegrown internal customer service tools when I worked there in the 1990s. Those were written in Perl. Then bought a book on C and started fiddling with that.

I wrote CGI scripts in Perl for paying customers in the late 1990s, then was lucky enough to move into a small company with some top-notch programmers who taught me to write more reliable, maintainable code.

The best lessons I learned early in my career came from having to maintain my own code in production for years on end. You learn not to cut corners and to avoid convenient hacks when you have to maintain the code. You take the long view in your design, aiming to prevent problems you now know how to anticipate. You seek and implement best practices, because dealing with problems in live systems is no fun at all.

There's an old saying: Always code as if the person who will maintain your code is a violent psychopath who knows where you live. When that person is you, you learn your lessons.

The systems I wrote early on helped researches run studies on human subjects. Later systems helped digital archivists preserve materials at multi-terabyte scale.

Maybe https://www.producthunt.com/

I've noticed they have gotten worse. After three bad MacBook Pros in a row, I bought a used 2015 MacBook Air, which has never given me a problem.

The pros all had battery/power issues, shutting down randomly several times a day. The keyboards were also horrible.

I've been a Thinkpad fan for years, and I use a Carbon X1 with Linux outside of work. No complaints about that, though I do wonder if it's as physically durable as the older, bulkier models.

I've noticed Lenovo is putting out more consumer grade laptops now. I don't know about their quality, but it doesn't seem as high as their professional line.

My son has a Dell XPS 15, and had to replace a loud, failing fan after a year.

I suppose manufacturers face a lot of pressure selling to consumers who don't know what to look for in terms of quality and don't understand why two visually similar machines with similar specs would be priced so differently. The low-end machines in Walmart look just like the high-end machines at Best Buy.

The business grade machines from Lenovo, Dell, and HP have incentive to be higher quality because businesses often buy them with high level service plans. Lenovo, Dell and HP really don't want to send techs into the field to service laptops if they can avoid it, and the best way to avoid that is to build a good product.

> We used to have a tech-debt ticket every couple of sprints to review and update various packages in order to take care of any potential vulnerabilities.

We should probably start doing that.

We have a pretty thorough set of tests, so we do catch breaking changes from Dependabot in the CI pipeline.

Yes, the Electron app has a huge number of dependencies, many more than our Go apps, which actually implement more features. On subsequent JS projects (after this Electron experience), I've become extremely strict about dependencies. Like you, I tend to write the functions I need rather than importing packages that bloat the codebase and make the app hard to debug.

Yes, we switched to triaging after a while. Still, there are times when we're forced to update, such as when Electron stopped accepting Let's Encrypt SSL certs, breaking a key feature.

I do think I'm going to move on to another technology with lower maintenance costs.

There are some technologies like this. I'm going to look into Wails: https://github.com/wailsapp/wails

I'm actually weighing the cost of doing just that. For our particular use case, it may be the better option.

Funny. I wrote a similar article, with almost the same title, seventeen years into my programming career, after writing my first novel.

https://adiamond.me/2015/01/writing-and-programming/

The huge help that programming gives to a novelist is the practice of breaking down huge, sprawling problems into smaller, manageable, interconnected pieces. As a programmer/writer, you also learn to keep the big picture in mind, so the pieces naturally assemble back into a whole.

Thanks. My author site is https://adiamond.me

I spent much of the first 15 years of my career at companies that would rather ship crappy software tomorrow than good software next week. The stress level was high because we were always in a rush, and management always was always pushing to get more out of every developer, like wringing the last drop of juice from an orange before they threw it out.

We often worked overtime so we could write the code we were assigned to write. We couldn't do that during office hours because office hours were for meetings, answering email, and fixing bugs in the crappy code we pushed out last week. Because last week's code was counted as "done," fixing it didn't count as "progress." We weren't getting anywhere unless we produced something new, and that only happened after hours.

A few years ago, I started working at a university in the US, where things move much more slowly. I started having the time to properly architect software, to rewrite chunks of code when necessary, and to write much more thorough tests so I'd have fewer surprises when we got to production. The saner pace and the ability to be thorough made the work enjoyable again.

The academic world in general tends to move more slowly and cautiously than the start-up world, and that academic mindset creeps into even the non-academic parts of the university. The go-slow-and-be-right attitude is good for designing and building complex systems.

That said, universities are a mixed bag. Get into the wrong department, and you might find yourself maintaining crappy software that no for-profit enterprise would ever run. You might find yourself in even more meetings, with more politics, about things that matter even less.

But if you manage to find a decent project with a decent manager, the university can give you back your work-life balance, along with some of your sanity. You'll even find a little extra energy to put into your side projects, because the job hasn't wrung you dry.

It's been worth it for me, though not yet in the way I had expected. I write novels when I'm not coding, and I maintain a self-hosted blog about reading and writing mysteries and thrillers. I had hoped to draw a loyal audience of readers who would buy my books, though I never actually implemented a coherent readership-building strategy. I just wrote about whatever I felt like.

Turns out, a handful of like-minded readers, authors, and editors have gotten in touch through my contact page. Those contacts have opened up a number of opportunities and helped me develop a network.

I had a larger following on my Facebook author page years ago, but I didn't like the platform or the company. I stayed on solely for the author page. Judging by the number of followers and post engagements, I seemed to have a broader reach there. But I can say the actual engagement of those followers was much shallower.

People have contacted me through my blog for film rights and European publishing rights (neither of which panned out). They've given me free copies of books by other authors to review. They've invited me to write articles for their sites, and more.

I wouldn't say my blog has helped develop a brand, but it has definitely been useful. And it's never a chore if you just write when you please about what you please.

Amazon kind of gave up on discovery, at least for books, when they realized they could make more money running ads. I sell books on Amazon. Years ago, their book pages had two carousels of "similar titles" on each book detail page. Similar titles were selected by some pretty helpful algorithms based on what purchasers of the current title had also purchased, as well as some other secret sauce.

At the time, ads weren't very prominent, and they were profitable for authors. I could buy them for 5 cents a click and even if only one in fifty clicks resulted in a purchase, I made money.

Now, most of the algorithmic recommendations are gone, and for many ads, Amazon suggests bids of $2.50 or more per click. Those clicks bring in money whether Amazon makes a sale or not.

To some extent, they've given up on matching customers to the best or most suitable products. Why should they work that hard if ads are more profitable?

They've also had problems with piracy and counterfeits, as mentioned in this article. I heard a radio interview with a Target executive a couple years ago where a journalist said, "Amazon is underpricing you on everything. How do you plan to stay in business?"

She said, "We vet every product on our shelves. We know who makes it and where, whether it's legit, and whether it contains lead paint and parts kids can choke on. Try to find that info on Amazon."

The reporter didn't seem too impressed at the time, but the Target woman clearly saw where things were going. As the quality, authenticity, and reliability of Amazon's merchandise has declined, people are starting to notice.

I'm not sure this is a problem they will ever have the will to fix. While they break even or lose money on genuine goods they have to ship for free, profits from third party sellers and Chinese drop shippers keep the retail operation afloat.

I think part of the issue is that developers and engineers tend to think in concrete terms because to make things work, they must be concrete. People in sales, marketing, and management often speak in jargon and metaphors that cover for a lack of substance... though, to be fair, many of them don't know they're doing it because they don't really implement things themselves.

Years ago, I worked at a company where the head of marketing gave a 40-minute presentation to management about the new marketing strategy to drive growth. Her presentation was such a success they told her to present it to engineering, thinking it would whip up enthusiasm about what great vision the company had and how fast we were poised to grow.

The woman presentated to a sea of bewildered faces. None of the software devs could figure out what she was saying, and that's a problem, since marketing's job is to convey a clear message.

Finally, we started peppering her with questions. Not in a rude way. We were genuinely trying to pin down some concrete meaning behind the flood of jargon. After ten minutes of questions, one dev finally summed it up: "So we're going to run more ads and let people know we offer a broad selection and good prices?"

"Yes," the woman said. She seemed frustrated and defeated, though all the devs were happy to finally understand a message that made sense.

Then she blurted out, "God, I hate talking to developers!"

Yes. And that's the root of this problem:

> Nobody can subtract from the system; everyone just adds.

I've been using this for a couple of years, and it's really nice. It works with SQLite, which makes it very lightweight. I think the default DB is MySQL/MariaDB, for multiuser installations.

Well, it's a cool project overall. Definitely something you can fiddle with. Finding and recommending similar books seems to be a difficult problem. Amazon does a decent job, sometimes, though they tend to push top-sellers, making other titles invisible.

Goodreads does an OK job, some of the time, though most of the books I discover there come from reading my friends' good reviews.

RiffleBooks once looked promising, but they seem to have abandoned their site. It's been down for weeks. Like Goodreads, the main path to discover was following the reviews and recommendation of other readers whose tastes were similar to yours.

TheStoryGraph.com takes a unique approach. Readers tag titles with a handful of attributes, such as mood, pace, length, etc. If you like a book, the site can recommend others that have a similar mood, pace, length, etc. BookBub uses a similar tag-based approach.

If your site starts to pick up subscribers, you may want to consider the tag-based approach. It makes it easy for readers to contribute curatorial knowledge. It may also combine well with Google's recommendations.

That may be more than you want to think about for a hobby project, but I wanted to contribute my two cents.

The ability to read the opening pages of the book is really cool. The results were pretty good for the searches I did, like a search for Stephen King's Mr. Mercedes bringing up a number of Dean Koontz books. It would be nice to have a little blurb with each title, so we don't have to judge by the cover alone.

Did you have to create the previewer yourself, or does that come with the Google books API? And how do you determine which titles are similar to a specified title?

I worked at Hotelicopter, which was a hotel meta-search engine written in Ruby on Rails. We were bought out by the six major hotel chains in the US and re-branded as Roomkey.com. To get us started, they agreed to push a lot of traffic at us. We knew from load testing that it would be very difficult, with our practically non-existent ops team, to scale Rails to handle 1000x our current load.

We knew we had to rewrite the system in a more scalable language, and we had to be live on a fixed date. This was late 2010, early 2011. Our CTO evaluated a number of languages and chose Clojure. A team of five or so devs was able to write the entire system in a few months and launch without problems. I don't think that would have been possible in a non-lisp language.

The Clojure learning curve was steep, especially since stack traces for anonymous functions in Clojure 1.1 often began with "nil pointer exception, no source file, line 0."

Once our dev team "got" Clojure, we were hooked. The verbosity of other languages started to look appalling. Clojure let us iterate quickly without sacrificing performance, which was suddenly a big issue.

Writing Lisp code permanently changed the way I think, and the way I code in other languages. I strongly prefer composing short, deterministic functions, which are testable and easy to reason about.

Roomkey became a victim of the pandemic, when hotel bookings declined sharply.

I'm just now starting to pick up Common Lisp, and it's a joy: the simplicity and consistency of the language, the REPL, the emacs integration. I also like that it compiles to native code. That's good for performance and for deployment, since it doesn't require the target machine to have a JVM.

Bessel van der Kolk's book The Body Keeps the Score is a brilliant and enlightening study of how trauma and all kinds of negative experiences are literally stored in the body as tension, which, when chronic, can become debilitating inflammation and pain.

Van der Kolk noted in his decades of practice that trauma victims, and those who are chronically angry, anxious, wary, etc. are deeply out of touch with their bodies. They were often clumsy and visibly tense, though when asked how their bodies felt, they either couldn't say, or they were just "normal."

He had to first get patients to recognize how their bodies felt, then taught them to relax and change those feelings through various exercises, meditations and practices. When the body started to loosen up, the mind and emotions followed.

https://www.goodreads.com/book/show/18693771-the-body-keeps-...

Edit Eger, a holocaust survivor turned therapist, found the same things in her decades of practice. Both books spend a lot of time describing the symptoms and problems, and the address healing practices toward the end.

Eger's book is at https://www.goodreads.com/book/show/30753738-the-choice

In general, both Buddhist meditation and Yoga address these mind-body issues well, in forms that you can practice as part of your daily life.

Thanks for mentioning this. Since I turned on Verizon's spam blocking feature, all texts from Verizon are marked as junk. (And I didn't even configure that, Verizon just blocked itself out of the box.) Anyway, I found the message in spam, but it's impossible to opt out right now because the site is having problems.