Image Recognition & Object Detection for The Nature Conservancy

SITUATION

The Nature Conservancy is one of the world’s leading conservation organisations, working in more than 70 countries to protect ecologically important lands and waters for nature and people. In the Western and Central Pacific, where 60% of the world’s tuna is caught, illegal, unreported, and unregulated fishing is threatening marine ecosystems, and global seafood supplies. The Nature Conservancy wanted to develop a camera-based solution for automatic detection and classification of fish species caught by the commercial fishing fleet. To find the solution they summoned the global tech community to participate in a Fisheries Monitoring contest at Kaggle.

SOLUTION

Ciklum R&D Engineering team received the training dataset of about 3,700 photos. The team needed to build an algorithm to identify a certain fish type. Traditional algorithms would not be effective enough for the required solution, hence Ciklum engineers used innovative object detection methods based on convolutional neural networks.

RESULT

Ciklum R&D team developed a series of algorithms to automatically detect and classify different species of tuna, sharks and more that fishing boats catch, to accelerate the video review process. Fish detection algorithm – localization precision is 95%, recall – 80% Classification algorithm for different species of fish on a boat with 93% accuracy

Find more information in the case study

Portrait segmentation and background removal to eliminate manual design work

SITUATION

An european recruitment agency receives around 5,000 CVs per month. Each submitted CV needs to be in line with the agency’s corporate standard including the candidates’ photos on a similar corporate background. The client urgently needed an algorithm to separate the head from the background as a professional designer would.

SOLUTION

Around 2,500 CVs/month have to be processed by the agency’s Graphic Designer. They changed the profile picture backgrounds to their corporate background with agency’s logo. This meant over 100 CVs to be processed/day. - Every profile picture required 20-30 minutes to change the background - Routine manual work was demotivating agency’s designer image recognition

RESULT

Ciklum Deep Learning team developed a series of algorithms to automatically detect humans in the image, and replace the background. The agency could maintain trust with employers by providing images, streamline the selection process, and save time and money.

Find more information in the case study

Car Image Masking With Convolutional Neural Networks

SITUATION

Carvana is an online marketplace for buying used vehicles. It provides a fully automated service scheme that resembles a coin-operated vending machine for cars. Carvana has a custom rotating photo studio that automatically captures and processes 16 standard images of each vehicle in their inventory. Carvana takes high-quality photos, but bright reflections and cars with colors similar to the background cause automation errors, which requires a skilled photo editor to change. To manually edit 16 projections of thousands of individual cars, the company would have to hire an army of designers, which would be a time-consuming and expensive process. Customers had to rely on blurry pictures with little information. Carvana urgently needed an algorithm to separate the car from the background as a professional designer would. The company hosted a competition on Kaggle with $25,000 prize, 735 teams, and two months to find a solution.

SOLUTION

Ciklum R&D team had to develop an algorithm to automatically generate masks for cars in images. The algorithm has to predict whether each pixel of the picture belongs to the class of car or the background. For the training set, a GIF file with 2.5K images was provided, containing the manual cutout mask for each image with pixel intensity values of 0 and 1. The testing set was 100,064 images in .jpg format with 1280 x 1918 resolution. The car and the background in the image had to be colored at the pixel level. The input was a JPG file and the output, a binary mask. There are several popular models for semantic segmentation in recent deep learning literature such as SegNet, FCN, E-NEt, U-Net, and this year’s state-of-the-art models, such as PSPNet and LinkNet. After some initial experimentation, SegNet and U-Net were used as a base to build a custom architecture. A model was built from the best results on the basis of weighted values. Depending on internal validation, the values were summed up for the next validation.

RESULT

The evaluation metric was a mean Dice coefficient, which demonstrated how two multitudes are overlapped. The higher the Dice coefficient, the better (max 1). The algorithm demonstrated a mean Dice coefficient of 0.997 for differentiating the cars from the background, which was much better and faster than manual design work. How the algorithm works:

With an efficient algorithm, Carvana could now build trust with consumers by providing better images, streamline the online buying process, and most importantly, save time and money.

Read more details in our DZone blog