This report presents the design and implementation of the core technical enablers for the SEcure Decentralised Intelligent Data MARKetplace (SEDIMARK) platform. The project aims to address the limitations of centralized data markets by fostering a secure, trusted, and intelligent ecosystem based on Distributed Ledger Technology (DLT) and Artificial Intelligence (AI). The work detailed in this document establishes the foundational components for interoperability, AI-driven services, DLT-based trust, and distributed storage, advancing the platform from a Technology Readiness Level (TRL) of 5 toward demonstration in real-world scenarios.
A key contribution of this work is a comprehensive interoperability framework. At its core, the framework uses the NGSI-LD specification to create a common semantic language for data assets. This is supplemented by a Marketplace Information Model, which defines crucial marketplace concepts such as Self-Description, Offering, and Asset. This model builds upon existing standards like DCAT and ODRL by introducing the "Offering" concept, which allows multiple diverse assets—such as datasets, AI models, and services—to be bundled and transacted together. A suite of software components within the Interoperability Enabler handles data formatting, curation, quality annotation, and validation to ensure data adheres to FAIR principles.
The platform's intelligence is powered by a multifaceted AI Enabler. This component supports advanced local model training with techniques like the transformer-based CrossFormer for multivariate time-series forecasting and model optimization methods like pruning. For collaborative scenarios, the project introduces two frameworks for distributed training: deFLight, a dynamic and fully decentralized framework supporting gossip and federated learning, and Fleviden, a tool for orchestrating complex federated workflows. A significant innovation is the Offering Generator, which uses Large Language Models (LLMs) to automatically create standards-compliant, semantically rich marketplace offerings from unstructured metadata, lowering the barrier to entry for data providers.
Trust and security are ensured by a DLT infrastructure built on a private instance of the IOTA Tangle (Layer 1) and IOTA Smart Contracts (Layer 2). This two-layer architecture provides a non-repudiable ledger for managing participant identities, cataloguing offering metadata, and facilitating secure asset trading. To support the platform's digital assets, a robust Storage Enabler provides a distributed architecture using Minio for AI model storage and NGSI-LD brokers for scalable, interoperable storage of linked data.
D3.4 deliverable can be downloaded from here.
