We do not currently know of any antonyms for talk of the town.
The noun talk of the town is defined as: