I wouldn't call Cooperstown,NY, Canton,OH or Springfield, MA major cities, and I think those Hall of Fame museums compare more closely to a specialized museum like the ANA's than a large museum like the Smithsonian. That being said, I don't really care where the museum is located. If I want to see it badly enough, I will make the trip. Other's - especially non-collectors - would obviously feel different.
And, on the subject of the city of Colorado Springs: I have been there on three separate occasions. Aside from one motel that I would not recommend to house a dog, I liked the place, and would not consider it a "prairie dog" or "backwater" town. Just my opinion, of course.