Sure. But the man-hours required to build an app without the gems you would get for free on Rails is substantial; you would only want to do this if the app in question really calls for speed.

From time to time I get seduced by the romantic idea of building a simple JSON-spitting Go/C#/Java backend for an app, but then I realize that I would have to hand-code an enormous amount of functionality that I would get for free with Rails.