This is a very interesting issue (to us professional linguists), which (as when a doctor identifies a patient's case as interesting) is probably not good news from your point of view! The truth is that in English, word order with this type of proper name varies and it is not totally clear why or what the regularities are. In England for instance we have the River Severn and River Thames (no-one would think of saying them the other way round). But in N. America it is the St Lawrence River, and where I lived for a while in Connecticut there was the Quinnipiac River; New York City has the so-called East River, and I think I am right in saying that "Mississippi River" is more normal than "River Mississippi" though in practice people usually omit "River" with that one. You might think this is a British/American contrast, but if that is a factor at all it is only one of several I believe. With "Lake" all the N. American Great Lakes are said as "Lake Michigan", etc., though Thoreau's famous Walden Pond would never be called "Pond Walden". (In England we don't have many lakes and the ones we have mostly have one-word names like "Windermere" which inoorporate a root meaning "lake".)
With castles, in England you can find cases in either order. The great majority are X Castle, but for instance there are also Castle Goring and Castle Drogo.
One of the relevant factors is whether the word that isn't "lake", "castle", etc. can be seen as a descriptive modifier of the other word or is just a pure proper name. Most English castles are named after the place where they are located: Windsor Castle is so-called because it is at Windsor. On the other hand, Castle Drogo is not in a place called Drogo (there is no such place); it is a newish building that was arbitrarily named after an antiquated version of a family surname. The more the name-word feels descriptive, the more likely it is to come first; even in England, an equivalent to the East River would definitely not be called "River East", because it is named by reference to the fact that it really does lie to the east, whereas most actual English river names are to us arbitrary noises inherited from before there was an English language. But this rule is only a tendency: Castle Goring is as a matter of fact at the town called Goring (which was there first, of course).
Another relevant factor is usage in local foreign languages. Lake Superior presumably has that name because it is "superior", i.e. higher or further than the other Great Lakes, but I'm sure that one reason why it has that order is that the first white men in that area were French-speaking voyageurs who called it Lac Supérieur in the normal French word order, and we inherited that in English. (Another reason might be analogy with the other Great Lake names; it would seem awkward to have Lake Ont., Lake Mich., Lake Erie, etc., but Superior Lake.)
Thinking of your problem of English names for German castles: it seems to me that if a castle name manifestly referred to the town where it was located, then it would be most natural to put "Castle" second (so, if there is a castle in Hanover -- I don't know whether there is -- we would call it Hanover Castle); but if it seems to be an arbitrary proper name just for the castle, then it would not be odd to put Castle first, particularly when German regularly does that. For instance, in Switzerland the castle of William Tell's enemy Baron Gessler has the name "Küssnacht" but so far as I know it is not at a town called Küssnacht, so I think we would be quite likely to call it "Castle Küssnacht" in English. (Google gives me examples in both orders for this one.)
To sum up: there is not one hard and fast rule about this kind of thing in English, rather a bunch of tendencies which sometimes pull in different directions. And one of the tendencies is to be influenced by the local foreign order; so in your position I would not go to the stake for "X Castle" across the board. However, if X is a well-known town name, like Hanover in my example above (which I chose because it is so significant in English history that it even has a distinct spelling in English), then I do think that "Castle X" would be rather weird.
I hope this helps even if it is not a clearcut solution to your problem.