Photo: left image shows night-mode on an iPhone 13 Pro, with mushy details and noticeable star trailing from a 30-second exposure. The right image shows the OPTICA process on an iPhone 13 Pro in a AI-tracked 180 second exposure. OPTICA output is cleaner, sharper, with a lot more detail across the full image.
The Institute of Space Sciences and Astronomy of the University of Malta, in a project funded by the MCST Research Excellence Fund, has worked on the OPTICA project (OPtical Telescope Intelligence for Computational Astrophotography) to build software which allows good astrophotography from small sensors in smartphones.
Computational photography refers to image capture and processing techniques that aim to enhance upon optical processes absent from camera gear, a field that has grown substantially, especially for smartphone engineering.
When clicking the shutter button on your phone, you might think that the image captured was exactly what you saw, but nothing is further from the truth. This data requires heavy interpretation, and with the hardware restrictions on smartphones as well as the limitations dictated by the laws of Physics, the actual raw image received by a sensor can look like a colourful mess. A lot of colour science and image processing sits in between the sensor, and what you see on screen. These problems are compounded for specific uses like astrophotography (astronomical photography) , where light from the heavens is faint and barely detectable by small sensors.
The dynamics of astrophotography have barely changed since their inception. Astrophotography requires long exposures for enough light from distant stars to reach the sensor. Furthermore, many of these frames are needed to enhance the final image, and this requires specialized software that can cancel effects of the earth’s rotation during this time window. Smartphone manufacturers have started automating various ‘n¾±²µ³ó³Ù-³¾´Ç»å±ð’ photo captures, allowing casual photographers to have a go at capturing the night sky.
OPTICA enhanced this process substantially, by making use of tailor-made artificial intelligence algorithms that can track the sky without a sky tracker, stack images without external software and fixing for small sensor noise levels and optical defects.
So far, excellent results, off a casual tripod stand can be obtained, far exceeding night-mode results with sharper, cleaner, and far more detailed images. The OPTICA team is now currently researching ways to remove a tripod from the equation altogether, for anytime, anywhere astrophotography from mobile phones. The key is to leverage the powerful computing power on modern smartphone chips.
Over the next years, it will be increasingly difficult to define what a camera is. The main physical components remain the lens and sensor. But even professional cameras are moving into heavier computational processing. A camera is nowadays a hardware-software package, with intelligent software taking over some of the creative decisions and work for us.
This project was conducted under the investigation of Dr Andrea DeMarco, Dr Ian Fenech Conti, Dr Andrew Finch from the Institute of Space Sciences and Astronomy, UM.
