The ship image can be a .jpg; you'll save space with the compression.
The mask should be a 1-bit (2 color, black & white) .bmp
The images that fade out, like an exhaust image, have masks that are 8-bit (256 color) greyscale images. White is opaque, black is transparent; the darker the grey, the more transparent that particular pixel will be displayed as.
Look at My Stations and their bitmasks; as examples of the 1-bit bitmasks. My ship downloads have 1-bit bitmasks also.
Then you can look at the recolored exhaust image, and it's bitmask, on This Page for an example of the 8-bit masks.