Our transformation to a smart factory
Streamlining manufacturing using “ABEJA Platform Annotation”

DENSO International Asia Co., Ltd, (DIAT)



As one of the world’s largest automotive manufacturers, DENSO drives R&D with cutting-edge technology including artificial intelligence.

DENSO International Asia Co., Ltd. (DIAT), is DENSO's regional headquarters for Asia & Oceania region performing business administration including R&D. DIAT manufactures mainly car components.


Mr Hiroshi Uchino
Regional Production Innovation Division
TIE & SHE PIP Department
DENSO International Asia Co., Ltd. (DIAT)

The background

Transformation to a smart factory by AI-powered solutions

At DENSO, we have employees in charge of “TIE”. TIE stands for Total Industrial Engineering. It means to analyze the entire manufacturing process from an engineering perspective to pursue efficiency and rationalizing the entire manufacturing process.

For example, imagine the work of an assembly line worker in the factory. He picks up “part A” with his right hand, mounts it on another component, “part B”, and then places it back on the line using his left hand. Say, this whole movement takes around 10 seconds.

With the idea of “TIE”, we divide this process into several task elements. Picking up a part, lifting it, and placing it back on the line, etc. We measure the time to perform each task. Then, we compare the time it actually took to the target time, and analyze what is causing the difference.

This analysis helps us to come up with improvement measures that lead to efficiency. E.g. “the movement of the right hand seems inefficient”, “would it avoid irregular manufacturing if they used their left hand instead?”, or “the component shouldn’t be placed there because workers often fail to pick it up”, etc. This is how we eliminate work inefficiency, irregularity and inconsistency.

Amid tough competition in the manufacturing industry in the world, we would not be able to survive if the assembly line workers were to do their jobs in each of their own ways. That is why we place TIE managers to thoroughly manage and analyze the manufacturing process. Their job is to come up with the “standard work procedures” to define the best way, the best speed, and the best timing the assembly line workers should move with. Following these standard procedures, the line workers will be able to produce the products at the best QDC (quality, delivery, cost). We are in charge of the Asia region. We work closely with the local members, analyzing different assembly lines within the region each week to keep improving the process.

“However, there was one issue. The operation still included antiquated manual work. The TIE managers monitor the assembly operations and gather data for their analysis by holding up a stopwatch to measure the time, and manually listing the results on an excel spreadsheet.

The solution

"ABEJA Platform Annotation" evolved as if it was developed by ourselves

We got to know about ABEJA at the end of 2017. At the time, we were still unsure about whether to introduce AI technology into the company or not. Of course we were aware of the potential of AI and IoT. However, we wanted to have a clear purpose beforehand, and we also wanted to see clear need by our workers working in the factories.

To start with, we asked ABEJA to demonstrate AI so we could see how it would work for us. Through the demonstration, we were able to learn about Al while ABEJA learnt about our manufacturing style at the same time. Later on, we discussed and listed the potential areas where AI could be useful in our operations.

Around March to April in 2018, we decided we could possibly introduce AI in the TIE process. TIE is highly specialized and it takes a long period to train future TIE managers. We only had one TIE manager per 100 or more factory workers, and it was not enough considering the scale of assembly lines we had. We were certain that installing cameras and getting AI to analyze the movement of each worker would improve the efficiency dramatically. Also, if we accumulated more and more data, the speed of improvement would accelerate 100 or 1000 times.

We designated one assembly line as a trial line. We installed cameras and started recording the workers movements there. I wouldn’t say everything went easy. When we started, we didn’t know that the best angle for humans and AI to monitor the movements were different. After many experiments, we came to the conclusion that shooting from above was the best angle for AI to observe workers’ movements.

Once we sorted out the filming process, we then came across another big issue, “Annotation”. This process is necessary for Machine Learning and it is about creating teacher data for AI. We had to classify task types carried out by workers in the video clips.

At first, we had to watch the video and categorize scenes into each task type, "picking up the component A" or "mounting the component A onto the part B".

We understood the importance of having as much training data as possible, but we didn’t want to spend a lot of time on annotation, watching the video, breaking it down to element tasks, and timing the task, etc. The ultimate purpose of TIE is to improve assembly work and enhance production efficiency, and not getting swamped by micro tasks.

What helped us there was “ABEJA Platform Annotation”.

Immediately, I felt that this annotation platform was so intuitive. I could easily figure out how to extract objects from the video clip and put labels to indicate tasks. Our local project members were also able to use the tool straight away and intuitively. I thought it would be very easy to introduce it into the factories as well. Furthermore, in the future, if other workers than TIE managers can also start classification tasks in the future, it would reduce our burden.

It was also helpful that ABEJA kept improving and updating the tool for us. During a telephone conference, we once gave them feedback regarding the tool, and in the following week ABEJA came back with the improved version. Since TIE is highly specialized, general-purpose platforms other companies were using weren’t good enough for us. However, “ABEJA Platform Annotation” went through many improvements and updates, and now it has become an extremely convenient tool as if we developed the tool by ourselves.

ABEJA also joined us in discussions on how to improve efficiency, which brought great insights. We realized that monitoring workers and breaking down the work into task elements is just the same as the Annotation process. This helped us reduce the additional work regarding annotation. We started to work more optimally, monitored workers while annotating at the same time. As a result, it also saved our time significantly.

Our initial plan was to select an AI partner with “passion” and “speed” for the project through PoC. After experiencing ABEJA’s dedicated service, we now strongly feel that ABEJA is the right partner that can accurately handle our highly specialized demands.

If human workers measure the time manually using a stopwatch, it would be difficult to always get the exact data. With the help of ABEJA, we now use an AI model to analyze workers’ motion on a test line. As such, AI has improved measurement accuracy significantly. In addition, we are now able to store and accumulate data over time. This will lead to better model development and better analysis accuracy. My belief is in more data, if we have more training data, our AI can be made to perform even advanced analysis like "the following task tends to cause variation in the movement of right/left hand", "placing a component in a certain position would make it difficult for workers to pick it up", etc. We are expecting more insights as we continue.

The future

When introducing AI, it is essential to "understand each other and become one team" with the vendor

We are hoping to expand applications of “ABEJA Platform Annotation” to other production lines. We manufacture a very wide variety of products and have various assembly lines all over the world. Once we make the tool more versatile, and if we are able to analyze all assembly lines by installing cameras, our TIE will develop unparalleled.

In the future, we should be able to obtain and analyze big data. In a factory, unexpected things happen every day, e.g., a worker makes a mistake, equipment suddenly goes out of order, etc. Up to now, we’ve identified the causes and implemented corrective actions based on the experiences or observations of TIE managers, and it could be time-consuming. However, having big data would enable data-driven cause analysis. It could show us new improvement ideas that we never thought of before.

Annotation is an essential process as a preparatory stage to use AI. Originally we thought that it is just a preparatory work for AI, but it has given us many fresh ideas for improvement. We are hoping to continue working with ABEJA to develop AI using “ABEJA Platform Annotation” and to accelerate improvement of our production line performances.

Please feel free to ask us if you have any questions about our products.
We will get back to you promptly.