We do not currently know of any antonyms for golden years.
The noun golden years is defined as:
See full definition of the word golden years.