Ask HN: Where do I go from here?

If the suggestion of solving other people's problems or entrepreneuring doesn't (directly) interest you:

Find a second skill to cultivate that complements programming. (Prime examples would come from the arts and science.) Seek a novel interdisciplinary approach to using both the new skill and your current ones. You don't become passionate about hard work overnight, yet you've somehow reached that point with your programming abilities. So lifestyle changes notwithstanding, if you can do it once, you can do it again with a new skill.

Doing this will bring you out of the "meta" realm where it looks like everything in software is one of (hugely complex, painful to maintain, already built, a pointless hack). Instead, you will have the benefit of perspective and can start to apply your technical armaments to some other field where computer-oriented problems are obvious, commonplace, never-ending, and usually - at first - easy to solve.

This is an excellent response. Diversification can do nothing but assist you, even if you deviate only by a small degree. Example: I used to have a similar issue with "idea paralysis" and it wasn't until I created a focus by recognizing my desire to write (but not replace programming, mind you) that I was able to come up with a solid idea and follow through with it (a decentralized content API is the app I've built and that atleast two of my sites run their content off of).

Follow the advice of above, it is very good.

"Hey, we need someone to develop an infrastructure for scheduling and running thousands of jobs on hunreds of different boxes and to keep the status of said jobs, etc." But that kind of makes me an architecture astronaut which, supposedly, is A Bad Thing (tm)

Just doing it to do it makes you an architecture astronaut. Finding somebody with that problem and solving it, especially in a different way, is a desirable thing. You could either out-compete an existing solution by doing it better, or find a chunk of problem space they don't really address (or don't address well).

Then you're an entrepreneur, which around here is certainly a desirable thing :-)

That is to say, figure out what it's useful for and keep that in mind.

Yes, this general "solution template" arises in many different applications, each with their own domain-specific constraints or specialized technology, requiring a customized solution.

What do you do for inspiration? How do you come up with your personal projects? When do you find time to do them outside of work?

I talk to people who have problems. (I recommend meatspace for this since if you just go to HN or Twitter you'll tend to select the problems that people on HN or Twitter have, and they'll say something like "I am really involved in too many social networks and need like a meta-social network, with RSS feeds and Twitter integration", and you'll go off and build something that will never be used by someone without a Twitter account which, by last count, excludes 99.9% of the population of earth.) For my personal project, I solved that problem. Finding time was a matter of looking at what I did between leaving from work and going to sleep and deciding what mattered to me less than being a successful businessman. Bye, WoW, it was nice while it lasted.

totally agreed about the meatspace thing! talk to your relatives, family friends, acquaintances, etc. about their jobs and hobbies whenever you're at a social occasion, and if you can see some inefficiency that they're currently forced to manually trudge through, start thinking about possible software solutions to their problems.

Funny you should mention this, as I meant to put this in my original post.

The one thing I did recently that made me happy and that I genuinely enjoyed, was writing a 20-or-so line script in Python that checked some file my wife was using at work. Basically it involved reading an excel spreadsheet and checking to make sure all the data was sane - a tediously manual process that was easily automated.

I talk to people who have problems

I'll second this, but add that you can reach more people online than in meatspace. Change your focus: instead of only reading blogs frequented by programmers, read car enthusiast forums, gardening blogs, dry cleaning business forums... you get the idea. Spend time listening to people whose problems are completely alien to you.

And get another hobby :-)

If your dream job is "thousand of jobs on hundreds of boxes", why not start hacking on something like Hadoop/Hive (or any one of the other new/popular distributed data stores) and, after proving yourself to the community, try to get a job for one of the companies that sponsors/offers support for the project?

E.g. cloudera, engineyard, something like that.

That way you're programming on a project that is technically challenging, more mainstream, and also a core part of the company employing you (instead of just being an enterprise IT cost center), all while leaving the business stuff to the company's sales/execs/etc.

My hunch is that those types of companies are more open to telecommuting as well, which might be nice given you mentioned having a family.

You are stuck with ideas but you have a potential. Try something simple, if you are not working for money so you'll really not care if 10 or 20K users use it every day.

Look around, what can help people? Something simple, small and easily maintainable. Finding it is a little hard. Don't forget that no one will give you the idea; but you can find it yourself.

Do a little search on forums, what are people looking for? what causes them headaches? May be the solution is a little Firefox plugin, but it can help kill your free time... if you finished it, go to the next one. One of those projects may turn big and take to what you have never thought of...

While the work is somewhat interesting and challenging, I'd hardly say it's something I'm passionate about.

The reason you are not passionate about your work is because something is missing. Identifying what is missing is your first step in determining where to go from here.

I have been in a similar situation to you. Always working. Important stuff. Sometimes cool, often not. But something was always missing.

Architecture not rigorous enough. Inadequate data base design. Insufficient requirements definition. Lousy code base. Unable to scale. Unable to expand or handle completely new features.

But I always managed to make it work anyway.

Then it occurred to me, if such mediocre systems were able to produce adequate results in commercial environments, what would be possible with great systems?

So now I'm building a framework/architecture/environment that beautifully handles everything I thought was missing before. The passion is built-in. Instead of, "Look at me, ma!" now it's "Look at this, everybody!"

Where do you go from here? Fill in the gaps that should have been providing passion all along. That oughta keep you busy for a while.

Easy: keep your day job. Work night and weekends on something you are passionate about.

Where to find cool stuff? As a programmer, the world is your oyster: there is a 100:1 ratio of people with ideas vs. people who can implement them. You probably don't know that I run a website to help people find such ideas: http://fairsoftware.net/publicProjects

If nothing else works, there got to be some open source project which you think could use some serious rewriting. In my case, I wish I had the time to design a better shell (bash just isn't smart enough for the way I work - if you ever used the Mac MPW in the late 90s, you know what I mean).

For others, it's a better, smarter GUI framework. I also think you could build something really cool to display Google search results: coverflow is just one starting point, but you can easily beat Google at the display part because of their fanaticism with providing simple page results. There is a better way for sure.

"My problem is - or, rather, one of my many problems (because I surely have more than one problem) - is that my hobby, my passion is programming."

I know its been said over and over again, but I think the advice still holds: join an open source project. Contribute to a piece of software you and others enjoy using. Software only gets better when people are willing to work on it.

Start a company. Just pick some interesting idea you're actually capable of executing and just start building it.

Don't worry so much about whether you will actually make money, to keep it a hobby, but make it in such a way that you might, to keep alive hopes of actually making a living out of it.

Doing it that way will allow you to make it however you please and to make it as beautiful as you want. The fun part is that along the way you will encounter critical areas you never thought about, but which are quite interesting to do once (setting up a build infrastructure, migration etc.)

Just make sure your code stays a thing of beauty, something so solidly engineered that you're truly proud of it. Don't rush things but take the time to make them 'just right'.

That way you'll have had a great time writing it, and if your company ever gets off the ground you'll have an amazing code base to start off with.

Start a company. Just pick some interesting idea you're actually capable of executing and just start building it.

Ideally, I'd do just this. I'd leave my job, take a few months and do my own thing. The problem is that I'm not prepared to take that risk; I have a good job, it pays well, and it supports my family. If I walk away and succeed (where success >= the same amount I'm making now), awesome. If I fail, however, I've left a job I'm good at, and that I'm comfortable at.

Perhaps that's the crux of the issue right there. I don't want to take the risk of putting all my eggs in one basket, but I find it difficult to get inspired if I'm not fully invested. Thus, if given the option after a long day of work to either watch Top Chef with my wife or do something interesting in Python, Top Chef+Wife wins every time.

Somehow I forgot to mention "as a side project" in my original post.

Your job will most likely always be shitty, at least I know it is for me. No company I can find excites me and the ones that do generally don't stay exciting for more than a year, after which you'll be hunting for a great job for several years before you find another one.

If you don't want to work on it in your spare time, your only choice is to find a kick ass job that pays enough to keep your family happy. If you figure out how to do that; let me know... ;)

Could you do something for 15-30mins before or after watching Top Chef? If so, try it for a week and see how it goes. I think you can get surprisingly far with short bursts of activity. Also, how about taking some of your vacation time - maybe a week to jumpstart something?

Why is being an architecture astronaut supposedly a bad thing ?

There are plenty of places that are in dire need of people with that skill, it's taking system administration to a whole new level and not that many people are capable of doing that.

If you're up for a challenge find some fledgling open source project that tries to solve a real world problem that you have some affinity with and start contributing.

Sooner or later inspiration will strike. It usually does when you stop looking for it!

best of luck,

  Jacques

The reason why being an architecture astronaut is a bad thing is the tendency for building complex systems that solve all of the problems they thought about which then makes it virtually impossible to do some simple things that other people need to do which they didn't think about.

Oh, and the other problem I have encountered is the tendency for the astronaut to be unable to explain the complexities of their system in ways that people back on Earth can understand. (More than once I've been assigned such systems because I actually can figure them out. I don't enjoy it, but I can do it.)

Find a local meetup, and try to connect with people in your area who have cool ideas. Then put together a team and start building.

BTW, if you happen to be in the NYC or Philly area, email me.

Go to conferences. Make friends with like-minded people. Get another job.

Then you will be closer to your dream than you are now.

Read HackADay www.hackaday.com and improve on one that sounds fun.

start a blog, build a following, then once you decide to do your own startup, you'll have that first 1000-2000 early users

Can you recommend a guide?

I would like to start a blog, which is technically no problem. It's just, running into all the abandoned blogs on the internet has been discouraging.

having a blog successful enough to generate 1-2k users is a pretty big project in itself; sounds more like a way to put things off than a good first step.