Currently ImageLoader uses Image I/O for loading images, which means it's using the core ImageReader plugins (reliable but not necessarily fast). We should consider bundling a slimmed down version of the SSE-accelerated JAI IIO Tools JPEG reader for better performance (offers approx. 2x better performance than the core JPEGImageReader). We would need some help from the mediaLib team to provide a minimal native binary, and also some minor tweaks to jai-imageio-core to build a minimal JAR file.
Another option on Mac OS X is to write a little adapter that uses the native ImageIO.framework to load JPEGs (and other images) directly into BufferedImages. The benefit here would be better performance without increasing the download size.