Dictionary of Ichthyology  

mackerel sky = a type of altocumulus clouds, indicating moisture and instability; hence the saying "Mackerel sky, mackerel sky, Never long wet and never long dry". Presumably named for the wavy pattern as seen on mackerel flanks.

Edmonton, Alberta from Wikimedia Commons: http://commons.wikimedia.org/wiki/File:Mackerelskybig.jpg

Canadian Museum of Nature
www.briancoad.com