Hacker News Reader

TLDR: There are still many unsolved problems. You have to go beyond your mental comfort zone and start exploring your "unknown unknowns". I jumped out of my mental well, found my "unknown unknowns", realized that there's an ocean of problems out there, and am now swimming in it very happily!

Long version: Same situation as you 6 years ago. Quit "safe", salaried job to work on freelance projects. But eventually realized that most freelance projects too are the CRUD type. Quit that too.

Then I found that fields like computer vision, speech recognition, space exploration and embedded systems still have many many unsolved problems, and require proficiency in theoretical subjects like linear algebra and algorithms, hands on challenges involving electronics and mechanical engineering, and practical software development problems like performance optimization. Every time I learn something in these fields, my mind bursts with new ideas and makes me explore and do more. It's an expanding spiral of unsolved problems. I don't think one lifetime is enough to solve my own ideas, let alone rest of the world's. It's a complete turnaround from how I felt 6 years ago.

Now I enjoy exploring and building my own projects and ideas, and have redeveloped my love for all kinds of programming - low level systems programming , desktop app programming, mobile app programming, even web programming.

I also learnt electronics on my own and thoroughly enjoy my hobby electronics projects.

That's actually an interesting reply. Do you actually get paid to do, say, computer vision? If so, what's the ratio of enjoyable to menial work (please be honest)?

Also how do you manage to find jobs like this? I found that hiring managers tend to, like, lie about the jobs. What I mean is there's no shortage of interestingly-sounding job descriptions out there that have nothing in common with what the hired person actually supposed to do. And you can't know the day-to-day reality of any job before you'll actually get it and then it's too late. How do you work around this situation?

I totally agree with you that most job descriptions don't quite match the day-to-day tasks we end up doing. It's been my experience as well. Not just in all my salaried jobs, but even in the freelance contracts I took up.

My solution to that might seem extreme to many, but here goes: I simply stopped applying for any salaried jobs and ignored almost all the freelance gigs that came my way! The answer to your question is that I'm not getting paid by anybody. I don't even want to get paid by anybody. Working for others on _their_ ideas is something I find downright unpleasant.

When I worked for others, many tasks seemed "menial" / "time consuming" / "boring" to my mind. In the past, I did not realize that this unpleasantness is simply because I'm working on others ideas, instead of my own. Instead, I blamed it on the field of software engineering. I even remember confessing to a friend that I'd started to find programming boring. That was some 4 years ago. As I said, things have totally turned around for me since then. Now when I work for myself, everything becomes a new adventure from which I learn something more. I feel that the only way to get work satisfaction and make any impact is to explore the frontiers in different fields, work on my own, indulge my hobbies and build my own innovative products. It's definitely a risk, but luckily I had built up a nice nest egg during my salaried and freelancing days.

If all this sounds too radical or impractical to you, then you can gently start off by offering to develop a full fledged software solution for a small business you know about via your connections (somebody mentioned veterinarians above...I agree with that kind of thinking). I'd avoid the freelancing sites though - they are a whole different set of unpleasantness. In small business freelancing projects, you are usually the only developer doing everything, and so you will make an impact without any kind of restraints on your technical creativity. Depending on the luck and nature of that business, that impact may even be widespread in some way.

Of course, like me, eventually you may get bored of this too, since most such freelance projects too start off as CRUD type applications. But even when implementing such boilerplate, I kept my mind open for alternate solutions that involved unconventional fields like computer vision and speech recognition.

Many business processes today involve humans in the chain only because vision, cognitive thinking, reasoning and speech do not have practical software solutions. Imagine implementing a computer vision based system that is able to keep accounts automatically from handwritten bills, without requiring users to enter details. Such problems are most definitely not solved. I'm ok if all I end up changing is making one local business richer and happier...it's still better than zero.

Maybe I missed it, but how do you get money to live on?

I'm kind of doing the same thing. I quit a corporate job a few months back to work on stuff I'm really interested in. Doing a bit of blogging and hoping that maybe working on things I'm interested in and then blogging about it will get me a gig getting paid to work on things I'm interested in. But I suspect it's going to be a long slog getting to that money part. I'm fortunate, in that I've been frugal over the years and saved up so that I could continue on this path for at least another six months to maybe a year (well, insurance for myself and my wife is almost the biggest expense, almost as much as the mortgage). I'm just afraid that after six months or a year or whenever I can't afford to keep doing this that in the end nobody will have taken notice and I'll just have to go back to a soul crushingly boring corporate job again.

Soul crushing is exactly what they are!

My earnings from previous jobs and contracts, and the returns on my financial investments, will keep me going for the next 5 years. I'm single, so it's relatively easy for me.

I'm currently writing an ebook and working on 2-3 software project ideas which I hope to sell to local businesses and earn revenue. I'm also preparing some training materials, since trainings pay a decent dime here.

My long term plan too is the same - software products, some electronics products, ebooks, and trainings.

In addition to waiting for a gig, perhaps you can also think of monetizing your ideas into a business to sell to customers.

I think this is fairly common, my programming friends and I are in a similar situation.

There are diminishing returns to programming skill. Instead of focusing on becoming a better programmer (learning more languages, frameworks etc) you should focus on developing domain knowledge. Most of the time, programming is only useful in conjunction with some other skill. Find some time to develop domain knowledge in a fertile area.

Random example to illustrate. How do vetrinarians work? Maybe their day-to-day productivity would be enhanced by better software. If you have worked for 12 years, chances are you are better set with cash and can take greater risk. Why not get an idiosyncratic job like working in a vet's office to learn a new domain, or work in an IT job that would at least be partially exposed to a new domain?

Get off the web/orm/whatever framework carousel, work on a farm and build a carrot picking robot, become a secretary at a law firm, get a truck driving license. Stop coding for coding's sake and get in touch with the real economy. Then start coding.

Hello, throwthrowaway I can relate to what you're saying as I started at age 21 and after 6 years, was pretty much disillusioned with the industry.

To be honest however, I never really bought in the whole "IT/software is eating the world" mantra and was tirelessly looking for an exit plan throughout my whole time working in corporate IT.

However what fixed me wasn't the glory of moving to an exciting project or getting more pay, but taking time off and finding joy in what I'm working on in the moment. I know it sounds dense but I was getting depressed because I was getting older and didn't see myself progressing along like the dreams that people advertise here, like becoming a financially independent entrepreneur who influences the blogosphere or an influential programmer who has famous open-source project who is revered here.

I equated professional success in the form of my salary, prestige associated with the project that I'm working on, the intellectual rigor associated with the tech stack that I'm working with - with my self-esteem. And I realized that life is too short to chase after these things instead of choosing to work what you want to work on but not being so hard on yourself to stop you from enjoying the process.

I know that every time I switched jobs, I was angry and frustrated with my current one and aimed for higher salary and title to compensate for what perceived slights I've experienced. But ultimately it was self-sabotaging as I was responding to what others deemed to be successful instead of what I wanted to do. And beforehand, I was too scared to do what I wanted to do because I always thought I needed more money, more time and have other parts of my life ready to tackle the problem; but really, I was just scared of failure.

But I think failure is the common lot of most folks, despite of what the VC promotion is; but if you can accept that and be happy with you just having the opportunity to work on your own thing and follow through, I personally found that more rewarding over money and arbitrary prestige.

I feel like you are talking up two different things in your post and comments:

1. Making an impact

2. Solving challenging technical problems

These seem like separate problems that have little to do with one another (there may actually be an inverse correlation, see below).

If you want to make an impact then create a product that solves a big problem for humanity. Programming has essentially nothing to do with this, other than the fact software happens to be at the core of most, but not all, products these days. You can make an impact by understanding people's needs, coming up with a great solution, marketing it, convincing others to join you, and creating it. Programming will likely be a part of the above, but not the most important. And its importance will decrease with your product/company getting bigger (as will the impact any single programmer makes).

If you want to solve difficult technical problems then you can find lots of those in Computer Science research, either academic or industrial research labs. These are seriously challenging and can be fun to work on. However, I believe there is an inverse correlation between things that make an impact and things are are challenging research projects. Research projects are often so narrow that not enough people care about the results, and implementation in the industry is usually delayed by 5-10 years (less for industry research within the company).

One programmer can make an impact to a degree. Yes, making an enterprise system for some industry requires a ton of coding for business rules. But it might be possible to make that coding simpler/easier so that people with domain knowledge could express it without having to code it, of course this is easier said than done.

Also taking a step back and seeing how things could work differently is a nice idea. One of the most recent examples is Bret Victor, just his demos gave a rise to a lot of new cool stuff. Or take LightTable, it has quite interesting ideas for interactive programming.

Regarding start-ups, take for example TransferWise, it solved a business problem, and it started as a two man band. ZeroTurnaround with it's JRebel, solved a technical problem, but was conceived by one person.

So, yes, one person can make an impact, but it takes solving unsolved problems or noticing problems needing to be solved. Of course one person usually can't be a business, which is somewhat different from making an impact.

As a start I would suggest watching Alan Kay and Bret Victor videos.

People of our age group entered this field assuming that developing new programming languages, software processes etc. would get us to be the next Turing, Dijkstra ... - however, that frontier has been long conquered, so as to say

Currently, enabling conventional processes through human network linkage via internet etc. is the big thing - think Amazon, Twitter, Facebook etc. Even this wave will end sooner or later

If I, you or anyone of our age group wants a real challenge, we need to create things which provide value above the already integrated human network(via electronic medium). I mean creating next Amazon etc. is not fun - the real deal lies in thinking what people will next need once they have such services available

PS: 1. Thanks for raising this question, I too have similar doubts - writing this response helped me frame things for myself better :-) 2. I don't think making impact through programming is viable anymore - my response hinges on that being assumed and understood

Have you considered making a switch to a data team? I switched to a BI/DW team 5 years ago because I did not enjoy building corporate CRUD apps all day. Working data in a large enterprise is really more of a mental challenge than a programming challenge.

The coding work is so much more enjoyable, only about 5% of the it is UI, the rest is all design and working the data. I absolutely hated how long it took to do the UI part of a crud app, so much that I steered my career around it.

Data work is so challenging because you truly have to understand every process you are taking data from and come up with creative ways to make that data useful to the business. The BI team at my work may not be the most advanced java guru's but they are some of the most creative solution designers you can find in our company. They are also the team with the most experience and the most knowledge about our super complex business processes because you have to understand them all to produce useful analytics.

I also can't stress how nice it is to have SQL as the most used tool in your toolbox. I scoffed at it as a simple data query tool before joining a data team. I now know it as the most efficient programming language I have ever used. Sure it can take a lot of mental effort to develop a complex query but it is so much faster than putting that logic in any other language. You simply ask a detailed question of the database and it returns an answer set to you. You do not need to tell it how to get he answer, the optimizer takes care of that for you and it does a much better job than you could ever do (99.9999% of the time). No more coding endless series of loops. A couple dozen lines of sql could take thousands of lines to code in java/c#/ruby/python etc.

There is also a whole new world of data science opening up right now and corporations are cashing in. I will be learning hadoop and map reduce very soon so we can position our team to offer a competitive advantage to our business through big data. I will have to learn R and python because they are used to gather and work all that data. The future is looking bright for data workers.

I can agree/sympathise/empathise with almost every single comment you've made, and I'm also mostly none the wiser on how to overcome it. I'd be quite happy to b!tch about it with you over IM or email tho!

I'm roughly your age, and started programming around 10, doing my first paid work around 18 on a part-time-while-studying basis. I truly miss the days where one person could know a lot about a lot. Nowadays, it's a struggle to either know a reasonable amount (not even a lot) about some niche area, or even just be barely competent in a wider spectrum of things.

Where are you based? I was starting to think the problem might be limited to areas or regions (since almost everyone I know elsewhere seems so content/fulfilled).

I'm in London. There are plenty of jobs around here and I don't feel that it's a place thing. It just feels like a state of the affairs in the whole industry. I always give example of VisiCalc being written by two people from scratch. On one hand, they didn't had the modern tooling, on the other hand, imagine doing a similar project today — no way it would be pulled off by just two guys.

VisiCalc was incredibly primitive by modern software standards. You could write a web-based version with equivalent functionality in a week or so (in fact, I think that's how Google Spreadsheets got started).

Small popular programs get extended, grow, are improved by large teams of paid programmers, and then eventually the category becomes so advanced that you need to have that large team to make progress. I can (and have) written Tetris in 3-4 hours, but there's no way I could write Starcraft 2. But that doesn't mean that small programs have become impossible to write; it just means that you have to start looking where other people are not. Look at the success of 2048 or Flappy Bird. There's probably another category-killer out there for small-business apps, but if someone's found it they aren't going to advertise it.

It's not so much the amount of jobs on offer...it's what the jobs themselves offer, I think.

A friend of mine used to refer to it as cowboy engineering - where a problem needs to be solved yesterday, and everyone does anything necessary to solve it. Nowadays, even on side projects, the overhead kills you - issue tracking, version management, module management, change tracking, support. Precious little time ends up being spent on the core problem in most jobs.

Go was created by three programmers, Ruby by one, D by one ... sure, those projects all grew as they took on a life of their own, but small-team or single founders can still accomplish a great deal.

> no way it would be pulled off by just two guys.

> it just feels like one programmer can't make any impact any more

Can you explain in more detail what makes you believe this?

Some of the most creative solutions redefine the problem. What you need to do to make an impact can often be less work than you imagine, like work that removes unnecessary layers.

If you can name a few project ideas where you feel it'd be impossible to make an impact, we can try to provide examples of how to attack them.

Perhaps you could re-focus? It's hard (but far from impossible) for a single dev to make a major impact by building something ... what about considering some of the unsolved problems in the field?

http://en.wikipedia.org/wiki/List_of_unsolved_problems_in_co...

Working on these problems could be done in parallel with a traditional coding job, and would give you no end of intellectual exercise, even if you didn't get very far with any of them.

Another option would be to pursue teaching. Introducing other people to coding & building things is incredibly rewarding, and will also possibly help your career.

The OP didn't mention what the rest of his life circumstance is tho. Married? Kids? Mortgage? All those things tend to drain time and energy away from doing meaningful side project work.

And depending on where he resides, I've found that barriers to entry into teaching jobs can actually be substantial (assuming you're doing more than teaching older people how to use a mouse).

I didn't necessarily mean teaching jobs. I'm currently a full-time dev with wife and children, and I'm (slowly) organising a free day course on Ruby.

Also, you can teach as part of many day jobs. Everything from mentoring new devs to running Rails Girls tutorials on company hack days.

Valid points, all of them. I volunteered for many years (up to reasonably high levels of responsibility) for a major technical society. The issue that became apparent though, is that your day job still consumes most of your time, and it takes a LOT of satisfaction elsewhere to make up for lack of motivation/passion/interest/recognition in it (especially if your employer has less than no interest in/support for your volunteer work.

If you're in a position where you can apply your skills to help others in your work environment (or at least with their blessing outside that environment), that is gold! Sadly, such opportunities do tend to be few and far between, from what I've seen :-/

Programming is not a profession.

"It would be great if programming were a proper profession, but it isn’t. A profession has a structured model for advancing through levels of skill and ability, be it studying for a law degree and articles (working for a legal practise) or the years of undergraduate and medical training a doctor undertakes before specialising. The latter has clearly-delineated ranks, from junior doctor, via a brutal regime of 80-hour weeks, to consultant." http://dannorth.net/2011/01/11/programming-is-not-a-craft/

I would venture to say 90+% of all useful software is some variation of a CRUD app with a shiny looking mobile or web frontend.

You could consider going back to graduate school, getting a PhD in CS and work on, say, algorithms or machine learning or something. That way you'd be discovering new technology. The odds of getting a job in academia are very low just because of overproduction of PhD's, but the degree is not likely to hurt your employment possibilities, and might even give you some new and interesting opportunities.

There are problems to solve, and there are Problems to solve. Are you looking to conquer a new programming challenge -- develop stuff nobody's done before from a coding standpoint? Or are you looking to make a big impact in a way that hasn't been done before?

Please read this blog post. Does it get at some of what you're talking about? http://erlebacher.org/2011/03/22/time-scales/

If so, I get it. Let's talk.

Many research projects, at universities and research centres, need talented, creative programmers to work on computer software that support some research project. For example, there is a lot of very high impact computational work being done in genomics research, astrophysics, etc.

You do not need an advanced degree to work in technical support roles.

I would imagine that the programmer who works on the analysis of protein synthesis that in turn provides a cancer cure would get a buzz out of their contribution.

For example, want to experiment with programming languages? You must plough your way through parsing (a solved problem) and maybe some variety of VM to emit code for before you can even start actual work.

Why? If you're just trying to experiment with language, why not write a simple syntax tree interpreter? Save the full, production-grade implementation for when you've experimented enough to know more or less what you want to implement.

Well, it sounds like it's time for someone to put on his big boy pants! Congratulations, son!

Yes, real-world software projects that make money for a business (and pay you a salary) are larger than one man can handle. There's a lot of grunt work, and maybe only a handful of new and interesting things to do. At some point, you're going to have to accept this, and just get on with it. If you work for a business, the goal is to make money, not to work on new, kewl, glamorous software projects all the time. Sometimes those goals do intersect, but it's rare. It's called 'work' for a good reason.

It also sounds like you're suffering from another common malady, and that is the idea that programming is all about writing code, and programming languages, and frameworks, and fast turnaround results. I think you're starting to realize that this is not really the case, but you can't figure out why, so you suppose that maybe it's due to the insurmountable grandioseness that seems to surround even the smallest programming project. I think you've grown up under the spell of "programming as an end in itself", and now the illusion is showing some cracks.

Look, programming is not an end in itself. It's a way to express the solution to a computational problem. Think about that for a minute, or, preferably, the next week.

Programming languages are basically the same idea as mathematical notation - it's a way to express an idea about how to solve a problem. Sure, there's also the coolness of having a machine take your expression of how to solve a problem and actually solve it for you, but programming is still just a way to express the solution to a problem.

Look at all the posts we see everyday on HN and proggit about programming languages and frameworks and whatever the latest hare-brained language idea that the functional programming circle jerk fest has come up with. Man, it's like going to a math conference and all everyone talks about is the latest ways to write down algebra! It's no wonder that there've been no real fundamental advancements in computer science in the last 30 years.

I think you need to rewire your brain about the philosophy of programming, what it is, what it's for, and what it's not. You are at a crucial point where you need to decide if you're ready to go up an abstraction layer, and start thinking more deeply about the various problems whose solutions may be expressible in a computation form, and stop worrying about programming languages, frameworks, and "impact". If you don't want to do that, then you might as well just be happy laying bricks and comparing trowels, and leave the hard thinking to more interested and capable people.

This reminds me of one of my favorite quotes:

"In programming, the hard part isn’t solving problems, but deciding what problems to solve." — Paul Graham

Look at meteor.js (I have been programming for around 20+ years – I haven’t seen a tool that is such a big step change for a long while – the tool allows an individual developer to achieve more than whole teams – realtime multiuser web applications with hot swappable code - shared client and server code - just some of the good things about it)

You need to work on projects that you genuinely feel passionate about, not just random companies to pay the bills. You can still be a part of a bigger system but feel great about the work / product that your company is putting out.

If you feel passionate about social networking, work on facebook or twitter. Hell, twitter's existence was almost solely responsible for enabling the Arab Spring uprisings in 2011 and 2012, resulting in some pretty widespread social changes.

If you feel passionate about Open Source, try working for RedHat. If you think AI is going to be the next big advancement towards human progress, try to get on at IBM or another research time devoted to it.

Are you really saying that there aren't any problems out there that you feel a need to solve, that there aren't any companies solving them that you'd like to work for?

No-no, this is not about passion. When one joins Facebook this does not mean he or she works on "social networking", it means he or she works on a tiny portion of a massive Facebook's codebase, implementing, say, a layout of a targeted ad for Chinese users.

I don't want to say anything is wrong with Facebook in this situation, it's just a company with a huge product and 7000+ employees. You are almost guaranteed to be re-inventing the wheel "Facebook way" there.

From personal experience, the bigger (or more structured) the company, the more likely it is that you will be actively discouraged from straying too far from your assigned bit of work, either by doing it differently, or taking on a tangential problem in the codebase. This essentially kills, over time, any desire or ability to 'make a problem your own', and you run the risk of becoming just another code monkey.

Becoming disillusioned in the programming profession – please help