He extracted it from a poem that Solon had addressed to the poet Mimnermus (ed. Diehl, fr. 22). In Solon's time, you tried to write your philosophical reflections, world view, advice to the young, etc. in verse rather than prose, simply to make it more memorable. The actual quotation from Solon is the second, "spondaic" line of an elegiac couplet:
Γηράσκω δ' ἀεὶ πολλὰ διδασκόμενος
— and means literally, "But I grow old always learning many things."