Valeria Vishevataya in Artificial Intelligence, IT - Information Technology, Technology Community Manager • Altoros May 29, 2019 · 2 min read · +500

Automotive Insurance with TensorFlow: Estimating Damage / Repair Costs

Originally posted on
Automotive Insurance with TensorFlow: Estimating Damage / Repair Costs
Learn how AI tools, such as TensorFlow and Keras, can help insurers to automate damage assessment and avoid overcharging by a car parts supplier.

Burdens of damage estimation in auto insurance

At TensorFlow meetup in London, Marcel Horstmann and Laurent Decamp of Tractable shared their experience in building a working solution for automotive insurance that will allow for improving the process of damage estimation.

When a vehicle gets damaged in an accident, an insurer has to cover the repair cost. However, the estimation for it is done by a body shop, which may overcharge sometimes. Furthermore, the estimation process is manual and requires human experts and their time to evaluate the damage.

Employing machine learning, it’s possible to train a model that will recognize the damaged car parts, assess damage, make predictions about what kind of repair is needed, and estimate how much it may cost.

To deliver accurate estimation, the model should be capable of predicting what type of maintenance exactly is to follow. As it influences the final cost, it is necessary for the model to distinguish, for instance, whether the damaged car part can be still fixed on has to be replaced.

Visual models, as Marcel calls them, are responsible for image recognition and damage assessment. Repair estimate belongs to the domain of metadata models, while prediction uses both the models.

Enabling multi-instance learning

To build a model capable of estimating repair cost, there is a need for data to train this model on. Partnering with auto insurers, Marcel and Laurent got access to an immerse collection of 130+ million images of damaged cars. Furthermore, these images have annotations—highlighting which vehicle part was particularly damaged.

Within a single insurance claim, Marcel and Laurent usually exploit between 15–50 images to train the model. Some of them depict the overall vehicle condition, some illustrate the damage occurred, and some are just irrelevant for the assessment case.

To make it all work, they rely on the following technology stack:

  • Amazon S3 to store data
  • A custom-built API to download, decompress, and buffer images in parallel
  • Keras and TensorFlow to actually train the model

Marcel referred to multi-instance learning that they were trying to enable. By this, he meant that the model is to check a variety of images—taken from different angles—of the same damaged vehicle before making a final prediction. For the purpose, the team also employed the ResNetconvolutional neural network and tested a bunch of approaches to pooling.

On the way, there were some challenges to address:

  • Providing real-time inference
  • Using a relatively modest volume of data (1.2 million images submitted by an individual insurer daily, which can be treated as 12.5 images/second throughput per model)
  • Ensuring an ability to sustain irregular peaks of traffic

Additional constraints were imposed through training 12 data sets—containing images of 12 car parts—as different models. In addition, the recognition stage and assessment stage required separate treating, so it resulted in 24 models to train.

The solution: automatic car damage recognition model

Car Parts Identification

  • Parts localization and segmentation. The model identifies the main external vehicle parts, such as a hood, bumper, or lamps, and provides a comprehensive auto parts analysis.
  • Photo quality control. The algorithms estimate and check the quality of photos in advanсe ensuring an auto-focus on the main car parts eliminating the risk of blurred, over- and underexposed images.
  • Photostream processing. The model can process numerous vehicle's photos at once with subsequent integration of the results.

Damage Understanding

  • AI-generated car part damage level and cost estimation. As soon as the required part is defined, our algorithms estimate the extent and cost of the damage.
  • Repair/Replace decision support. With the experience gained from service stations that dealt with multiple insurance cases and the ability to evaluate the overall damage level efficiently, the algorithms state whether there is a need for car parts repair.
  • Overall car damage estimation. Estimating the damage level of the auto parts detected, our algorithms form a hypothesis about the general state of a vehicle.