Unless you started collecting prior to a bubble spike or a long up-move in PM's (like pre-1974), coins just don't appreciate for the most part.
I was looking at one of the most beautiful coins I have ever seen, the Duckor-Norweb Saint-Gaudens 1908-S MS67. David Akers bought the coin in 1988 (not an exceptionally elevated time period price-wise) for about $40,000 and then it was sold in 2012 (again, not a bubble spike price or depressed price) for about $160,000. That works out to about 6% a year, probably lower than CD's over that time, and behind bond funds and/or stocks or a balanced portfolio.
I don't think that trophy coins like some of the 1907 High Reliefs or UHRs have made money over the decades, either. You'd have to go back to pre-1974 when the price of gold held all these coins down in price to probably make a great 40 or 50 year return.