This (empiricism) has driven philosophers to things like positivism (where they declare everything non-empirical to be worthless or even meaningless). It's a problematic tradition.
The problems are made worse because it turns out that the large majority of ideas are assessed in a non-empirical way (not due to people being idiots; this is correct). Using evidence is the less common case (though it is quite important when relevant).
In _The Fabric of Reality_ by David Deutsch, he gives an example of the theory that eating a kilogram of grass will cure the common cold. This, he points out, should not be empirically tested, and will not be. Rather it is rejected without evidence because it is a bad explanation. Only good explanations are worth testing. (This does not make us miss out on any truths. If it really was true, someone could figure out some explanation of how it works, and then we'd test it.)
In his recent book, _The Beginning of Infinity_, Deutsch further explains that there's no point in testing any theory for which its details can easily be changed around in an ad hoc way, because you can never refute such things with evidence since they will just revise themselves endlessly. The only thing that can refute that sort of approach is philosophical criticism. Only ideas which survive some philosophical criticism and are therefore of higher quality are worthwhile to empirically test.
What makes it hard to change a theory around, ad hoc, to avoid refutation, is if there is some actual connection between the content of the theory and the problem it's trying to solve, so most changes to it would make it no longer address the problem as well.
Deutsch calls that quality "hard to vary" and says it is what makes explanations good. His books expand on this, and offer a version of Popperian epistemology with (relatively small) improvements.
Popperian epistemology, focussed on criticism not justification (which is impossible), and which applies to all types of knowledge without difficulty rather than being narrowly focussed on empiricism, is the solution.
One of the Popperian ideas is that because we are fallible and make lots of mistakes, and need to improve on them (by criticism), irrationality has to do with anything that hampers this process of correcting errors. So that's why I regard difficulty error-correcting priors as irrational. Deutsch even proposes that hampering the correction of mistakes is the most important criterion of immorality.
To contribute something, other than my thanks, here's a link to Deutsch talking about knowledge, evidence, and incredible cosmic relationships:
http://www.ted.com/talks/david_deutsch_on_our_place_in_the_c...
http://groups.google.com/group/beginning-of-infinity/
Thanks for the positive response! Good luck.
Almost nothing in science is justified by direct checks. Think of any clinical trial or say, the LHC experiments. There are so many implicitly assumed background theories about how the instruments respond to their stimuli. Some of these instruments are based on principles which were discovered only in this century. So scientists from previous times wouldn't accept the way the results are derived unless they are shown the results derived from previous experiments justifying the way the instruments work. For example, the atomic hypothesis was controversial till 1900 or so. How would one even interpret LHC electron beams if one one doesn't believe in the atomic hypothesis? A useful analogy here, is minesweeper - a new opened square, whose information, via a very long chain can tell us if there is a mine on a far away square.
I agree with some of what you say, but I'm not sure what you're trying to vindicate.
Bear in mind that one can construct infinitely many theories logically consistent with every piece of evidence or empirical check done in the past, and which predict that the grass cure for the cold will work. These theories will consist of various disjointed assertions in a rather arbitrary and ad hoc manner. The problem with these theories is that they are terrible as explanations -- they are bad philosophically -- but they are not empirically refuted.