We do not currently know of any antonyms for take one's leisure.
For similar words to take one's leisure, try: