Investigate approaches to detect and repair data quality errors during the manufacturing process.


November, 2021

Sagar Sen is a Senior Research Scientist at SINTEF Digital within the Secure IoT Systems group. SINTEF is Scandinavia’s largest research institute with over 2000 employees. SINTEF focuses on applied research and has large portfolio of projects spanning domains such as industry, energy, health and wellbeing, oil and gas, materials, biotechnology, ocean space, building and infrastructure, climate and environment to name a few. He investigates methods for verification and validation (V&V) of a wide range of software-intensive and data-intensive systems.  In InterQ, he is investigating the use of data profiling and AI to both detect anomalies and repair erroneous data from sensors in harsh and noisy environments characteristic of the manufacturing industry. His  current work extends his experience as the co-founder of an IoT startup, Sweetzpot, where he developed the Flow sensor for measuring forces in the body such as from breathing and developed virtual sensors (AI models) to detect anomalies such as obstructive sleep apnea and predict minute ventilation.

Could you describe the role of WP4 in InterQ concept?

WP4 InterQ-Data investigates approaches to detect and repair data quality errors during the manufacturing process. Monitoring data quality is a pre-requisite to zero-defect manufacturing as the quality of the data reflects the quality of the process and the product. Data is acquired from the machine’s internal variables, sensors observing the machine tool in operation such as accelerometers, acoustic emission sensors, cameras and from metrology of the product. Given the data available as a 24/7 stream, WP4 entails four main tasks: in-motion data validation, historical data validation, erroneous data repair, and data quality as a service.  In-motion data validation develops approaches to detect data quality errors during real-time acquisition such as the ski-slope problem and electrical noise in high-frequency accelerometers. Historical data validation will investigate unsupervised learning approaches to characterize rare events signifying machine degradation. Erroneous data repair will develop virtual sensors that can repair missing data due to low quality. Finally, data quality as a service aims to bring together activities in all other tasks into a comprehensive architecture that interfaces with the data from the use cases in aerospace, automotive, and energy domains.


Which are the main challenges faced by WP4?

The main challenges faced by WP4 are the following:

  • Eliciting data quality requirements from the various use cases
  • Transforming data quality requirements into modules that can automatically detect data quality issues and anomalies.
  • Unsupervised clustering of machine states from time series data and its application to monitor machine degradation.
  • Understanding the needs and benefits and realizing erroneous data repair in manufacturing data
  • Architectural decisions on how to incorporate data quality checks and erroneous data repair on a stream of data.

Which will be the outcomes of this WP?

WP4 aims to develop several software components to manage data quality in manufacturing process as described below:

  • A system for in-motion data validation that minimizes effects due to electrical noise, temperature, and the ski-slope problem. This system will extract vital information from high-frequency data from accelerometers to a 1Hz signal with important information such as temperature, five most important frequencies and amplitudes.
  • A component to perform comprehensive data profiling and issuing of data quality warnings on streaming manufacturing data.
  • A component to perform temporal clustering of machine data to automatically label degradation patterns in time series data.
  • A component to detect anomalies in machine and sensor data.
  • A component to perform erroneous data repair on machine and sensor data.
  • A software reference architecture to implement the components in the existing cloud infrastructure: KASEM from PREDICT and SAAVY systems used by IDEKO.
  • A component to export data quality hallmarks to a blockchain.

These components will help realize the following key performance indicators on data quality:

  • Ensuring an average data completeness of at least 98%
  • Ensuring an average data consistency of at least 98%

How do you expect that InterQ will change current manufacturing processes?

InterQ will improve manufacturing process in the following ways:

  • It will improve the auditability of manufacturing process through generating hallmarks in data and product quality in the process.
  • It will help reduce defects in products by ensuring high data quality and alerting operators when there is a deviation from normal behaviour or when a degradation in machining is detected.
  • Overall data completeness and consistency will make the data very suitable for machine learning tasks such as tool condition monitoring and predictive maintenance.
  • Promote a data-centric view where quality of data is more important than a model-centric view where an AI model and its accuracy is often the main focus.

INTERQ is supported by the European Union. Have you participated in more projects funded by the EU? How do you evaluate them?

  • I am currently involved in the EU project Dat4.0 in addition to InterQ and previously I have been involved in H2020 project AI4EU and FP7 project S-Cube.
  • I am relatively new to EU projects in the work package leader role.
  • My experience with EU projects has been very good and I evaluate them very highly in terms of multi-disciplinary scientific production and impact on society and industry.
  • EU projects also play a very important role in the financial sustenance of SMEs and at the same time helps rejuvenate larger companies.
  • In general, I think EU projects are a great way to increase the cohesion between enterprises in European countries. The collaboration helps people from diverse backgrounds to interact, understand, develop innovative services and products together. The projects play a silent role in maintaining peace in a very diverse Europe.