SEDIMARK Logo

D3.3 Enabling tools for data interoperability, distributed data storage and training distributed AI models. First version

SEDIMARK · January 18, 2024
D3.3

SEDIMARK D3.3 “Enabling tools for data interoperability, distributed data storage and training distributed AI models” is a report produced by the SEDIMARK project. It aims at providing insights in relation to progress made for:

  • Management of interoperability in data flows by proper definition and usage of metadata and semantic technologies
  • Distributed storage considering interoperability and scalability issues of such storages.
  • Finally, on the use of distributed AI and analytics, considering in particular federated learning.

Data quality is considered to be of the highest importance for companies to improve their decision-making systems and the efficiency of their products. In this current data-driven era, it is important to understand the effect that “dirty” or low-quality data (i.e., data that is inaccurate, incomplete, inconsistent, or contains errors) can have on a business. Manual data cleaning is the common way to process data, accounting for more than 50% of the time of knowledge workers. SEDIMARK acknowledges the importance of data quality for both sharing and using/analysing data to extract knowledge and information for decision-making processes.

Common types of dirty or low-quality data include:

  1. Missing Data: Incomplete datasets where certain values are not recorded.
  2. Inaccurate Data: Data that contains errors, inaccuracies, or typos. This can happen due to manual data entry errors or system malfunctions.
  3. Inconsistent Data: Data that is inconsistent across different sources or within the same dataset. For example, the same entity may be represented in different ways (e.g., "Mr. Smith" vs. "Smith, John").
  4. Duplicate Data: Repetition of the same data in a dataset, which can distort analyses and lead to incorrect results.
  5. Outliers: Data points that deviate significantly from the majority of the dataset, potentially skewing the analysis.
  6. Bias: Data that reflects systematic errors due to a particular group's overrepresentation or underrepresentation in the dataset.
  7. Unstructured Data: Data that lacks a predefined data model or organization, making it difficult to analyse.

Thus, one of the main work items of SEDIMARK is to develop a usable data processing pipeline that assesses and improves the quality of data generated and shared by the SEDIMARK data providers.

This document, along with all the public deliverables and documents produced by SEDIMARK, can be found in the Publications & Resources section.

Subscribe to SEDIMARK!

* required

The SEDIMARK team gathered in Paris for a hands-on technical session to refine our data & ML pipelines – boosting efficiency, interoperability, and security.

Read more 🔗

#SEDIMARK #DataSpaces #AI #Interoperability #DataSharing #MachineLearning

Join our @sedimark Webinar: Intro to the SEDIMARK Toolbox – Building a secure, interoperable Data Marketplace in action!

📅 Learn about:
✅ Tech powering data marketplaces
✅ Real-world use cases
✅ Trust & compliance in data sharing
🔗
#DataSpaces

On June 25–26, the @sedimark consortium met in Dublin for a productive GA hosted by @ucddublin ! Focus: advancing the MVI & MVM components + prep for the upcoming Hackathon in September.
Great energy, clear roadmap ahead! 🚀 #HorizonEU #datasharing #AI

Load More
crossmenu
SEDIMARK
Privacy Overview

This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.