The past tense of go down to is went down to.
The third-person singular simple present indicative form of go down to is goes down to.
The present participle of go down to is going down to.
The past participle of go down to is gone down to.