You've done a lot more thinking about this suffix business than I have. Most of the shades of difference with adjectives ending in ‘-ic’ and ‘-ical’ seem to have developed idiomatically and there are no general rules governing them.
Suffixes in general can be quite mysterious. For example, two opposite suffixes (‘-less’ and ‘-ful’) give similar meanings the case of ‘shameless’ and ‘shameful.’ But I may be able to come up with some rough guidelines for certain kinds of suffixes.
In the case of agent nouns formed by adding ‘-er’ and ‘-or,’ there's a generality to be made. Often the ‘-er’ ones come from Old English (like ‘singer,’ with roots in ancient Germanic), while the ‘-or’ ones are derived from Latin (like ‘editor,’ from the Latin edere*, ‘edit’). Even here, though, there are exceptions. When an English word has both endings (like ‘adviser’/‘advisor’), the ‘-er’ ending is often the older one. In the case of some legal terms, it appears that lawyers historically have been fonder of more pompous-looking Latinate endings than of simple Germanic ones. (Historically, English academics, jurists, and churchmen always respected Latin more than Old English, which explains much of the confusion about English grammar.)
Then there's the ‘-ible’-vs.‘-able’ ending. If there's a generality to be made, it's this: Often a word derived from Old English or another Germanic source (like Old Dutch, Old Icelandic, Old Norse and so on) will end in ‘able’ (‘forgivable,’ ‘lovable,’ ‘readable’). But a word derived directly from Latin will end in ‘ible’ (‘terrible,’ ‘audible,’ ‘legible’). Again, this is only a rough generality, since there are exceptions. And oddities too: ‘eatable’ is from the Old English etan (‘eat’) while ‘edible’ is from the Latin edere* (‘eat’).
I can understand how a mathematician might be frustrated by all these generalities. As the grammarian Otto Jespersen once said, this isn't Euclidean geometry.Pat O'Conner
*The Latin edere can mean to eat, to publish, to edit, to give forth.