Sony Industrial Cameras Beyond The Twilight Zone
Case study - Beyond the twilight zone - how imaging systems using Sony FCB Camera module technology are being adopted for scientific research in the darkest depths of our oceans.
By Marco Boldrini, Product Manager, Image Sensing Solutions, Sony Europe.
The deep-sea environment presents several challenges for imaging systems - not just in capturing an image in a low-light environment, but also with silt reflecting light back to cause glare and the reduced field of view that comes with being under the water, and then you get the challenge of transmitting it to the surface from over 2,000m below.
Here we look at how scientists from the Ifremer (French Research Institute for the Exploitation of the Sea) in Brittany have used Sony FCB cameras to monitor for pollution and coral health in the vulnerable marine ecosystems of the Bay of Biscay.
The Bay of Biscay, a body of water in the North Atlantic off the west coast of France and north coast of Spain, is an ideal environment for cold water coral habitats. These have become important biodiversity hotspots that are home to many species, and while the presence of these habitats has been known about since the nineteenth century, little is known about their present-day distribution and diversity.
Recent years have seen the Ifremer research institute adopt imaging systems to investigate the health of these ecosystems, and assess the damage being done by human activity - focusing their efforts on the 135 submarine canyons that lie along the continental margin of the Bay.
Earlier this year, the researchers published their findings in the journal Frontiers in Marine Science. The team surveyed 24 of the Bay’s canyons, as well as three locations between adjacent canyons, with the images that came back enabling them to identify a total of 11 coral habitats, formed of 62 coral species, at depths between 50 and 1,000 metres.
The paper follows a 2016 publication by the same research group using the same vision system to monitor the health of the sea floor, counting and classifying litter in these vital habitats. Surveying 15 canyons and three sites on the edge of the continental shelf, the team analysed 6255 images from the camera and recorded 198 items of unique litter, 42% of which were plastic items, 16% were fishing related objects, 9% were glass bottles and 6% were cables or ropes - 27% being either other or unidentifiable. According to the team, litter “appeared to accumulate at water depths of 801 - 1100 m and 1401 - 1700 m” and was found in all canyons and continental shelfs surveyed. More importantly, the researchers pointed out that “15 to 20% of the marine litter found in corals (including cable and ropes) was related to fishing activities”.
The vision system:
The Ifremer research team used two methods to capture images: from towed stills camera moving at 0.9 m/s, and from a remotely operated vehicle (ROV) running a 2008 Sony FCB-H11 colour block camera; which delivers up to full HD images, and is equipped with a 10X zoom (f1.8-2.1), a 50 degree horizontal viewing angle and is able to capture accurate images down to 1lx, while operating at low temperatures (to 0oC) found in the depths.
Camera systems were placed inside housing capable of surviving over 250 atmospheres of pressure, and a lighting system created to both illuminate the sea floor, but also avoid blinding reflections from silt - see below.
Images coming off the cameras also recorded time stamp, position and depth metadata.
Key challenges of capturing images in underwater environments:
Light deteriorates quickly under water, and while cameras such as the FCB-H11 can capture down to 1 lx (and several cameras, such as the FCB-EV7520A down to 0.35 lx), there is little significant light found below 200 m, and no sunlight beyond the twilight zone (200 m - 1,000 m).
Lighting also needs to be carefully controlled, however, as at depth the presence of particles such as silt cause an effect similar to high-beam headlights in heavy rain. Indeed, the research team highlighted that several images were discarded due to the problem of silt creating “poor quality images due to sediment clouds obscuring the image.”
This makes it very difficult to automate lighting control, and even in a more controlled system such as the ROV an operator would ideally have the ability to calibrate lighting and exposure times to maximise image quality.
Image transmission standards
Corals were found at depths of between 50 m and 1,000 m, with the team searching for marine litter at depths of almost 2,400 m.
While an ROV could be automated to travel at a fixed depth and avoid canyons / objects via sonar, the challenge of how to transmit acquired images back to the surface for analysis in a timely manner and without compromising image quality remains.
Traditional machine vision standards simply aren’t capable of transmitting data over these distances (see graph below), and as such fibre optic cable is therefore needed to control both the lighting and the camera systems, but this adds significantly to the cost of the build.
The corrosive nature of salt water, and the pressures the systems will be exposed to (1 additional atmosphere of pressure for every 10 m in depth) means the housing used needs to be exceptionally well designed. Here systems such as MacCartney Benelux’s Luxus range can be used, these couple Sony cameras with adaptable lighting control and a housing capable of reaching depths below 9,000 m.
Field of view:
Water also has a particularly interesting effect on light. With a refractive index of 1.333, it shrinks the field of view by 25 per cent, as any camera chosen needs to take this into consideration.
The depths of the oceans are some of the most extreme environments that imaging systems are being used in. Not only do they present the complexities of exceptional pressure and very low light, where light is used it is often reflected back by sediment, making images unusable, and the depth makes transmitting data costly.
But, as the research from the Ifremer centre shows, the need to use vision systems - which are evolving quickly to include specifications such as Sony Pregius Global Shutter CMOS technology - in such environments is both vital and urgent if these precious ecosystems are to be preserved