The past tense of learn by heart is learnt by heart.
The third-person singular simple present indicative form of learn by heart is learns by heart.
The present participle of learn by heart is learning by heart.
The past participle of learn by heart is learnt by heart.