Lessons Learnt

The ICARUS beta platform release is now featuring a number of new functionalities along the three core workflows provided to data providers and data consumers as described in an earlier announcement of its functionalities (MVP features, beta platform posts).

Data Check-in Workflow

The Data Check-in Workflow allows data providers to efficiently check-in their data to the ICARUS platform by configuring their data check-in jobs and execute it locally in their On-Premise Worker. In more detail, the Data Check-in Workflow involves:

  • Better understanding of the data that you are uploading through semantic enrichment, harmonization and putting to the appropriate context with the help of the ICARUS aviation data model. New features: Enhanced usability in the mapping step, improving the data mapping predictions (with the help of dictionaries) and allowing to copy and paste concepts from the ICARUS data model. If you still cannot find an appropriate concept in the ICARUS data model, you are welcome to propose a new concept that will be considered for inclusion by the ICARUS platform administrator.
  • Support in increasing your data quality through a number of cleaning rules that indicatively allow for removal of null values and detection/replacement of outlier values. New features: Additional cleaning rules have been provided.
  • Application of several anonymization rules depending on the privacy risks that each of your data field represents. New features: Insights on the anonymization step are provided in the On-Premise Worker.
  • Full control over which data columns/fields you must encrypt in an end-to-end manner as they contain confidential/sensitive information for your organization. Improvements on performance for encryption and decryption of the data.
  • Easy definition of the appropriate license for your data, as well as of access policies to regulate who is authorized to access your data if you have decided to make them available in the ICARUS marketplace. New features: Easier license definition considering the intended access level (e.g. for confidential data intended to be used only by the same organization, a “fast-lane” license definition occurs with minimal compulsory metadata). The data provider may also opt for writing the dataset’s license in the ICARUS Blockchain.
  • Progress monitoring of the data upload in the On-Premise Worker. Improvements: An OnPremise Worker’s dashboard in which statistics and information related to this account and its organisation account in the ICARUS platform is provided.
  • New feature: You can now update a dataset you had previously checked in either replacing what you had previously uploaded (with restrictions to avoid any potential misuse) or appending new data to the existing dataset.
  • New feature: With the help of the On-Premise Worker, you can download a dataset you have acquired or a virtual dataset or the results of an analysis you have performed in the ICARUS platform as long as the respective license and contract terms allow it.
  • New feature: As a data provider, you have access to general statistics about the available and accessible platform assets, as well as statistics for your own datasets.

Data Search & Acquisition Workflow

The Data Search & Acquisition Workflow allows data consumers to navigate to the ICARUS marketplace and legitimately acquire the datasets of interest. The Data Search & Acquisition Workflow involves:

  • Searching for the datasets you need, with the help of various filters. New features: additional filters as well as wildcard characters have been added in search to allow data consumers to find more easily the datasets they are interested in. You can also favourite a dataset of interest in order to keep an eye on it for a future potential purchase.
  • Quick request to purchase a dataset for a specific time period, according to your preferences (selecting only the columns you need and applying filters!).
  • Draft contract preparation by the data provider, setting the price and the terms of use you must comply with. Contracts prepared from scratch or based on the data contract templates their organization follows. New features: The terms of the ICARUS platform are also included in all contracts (without allowing any changes to be introduced by the data providers/ consumers).
  • Draft contract online review and offline availability to cross-check with your legal team (by downloading the proposed data contract). Easy action for your organization’s manager to accept/reject it or further negotiate its terms. New features: Negotiation over a data contract is now supported allowing for easily reviewing the changes introduced in the price and the contract’s terms.
  • “Offline” payment remaining entirely on you (as.. aviation data do not come for free!) – confirmation in the ICARUS platform (by the data provider) so as to grant you access to the dataset.
  • Full traceability, immutability and non-repudiation of your contracts with the help of a distributed ledger technology. New feature: The manager of an organization can also import an existing Ethereum wallet (in addition to having the option for creating a new wallet in the ICARUS Platform).

Data Analytics & Visualization Workflow

The Data Analytics & Visualization Workflow allows data providers and consumers to efficiently prepare their own and purchased data to run analytics and visualize the results. The Data Analytics & Visualization Workflow involves:

  • Efficient preparation of your data (in the form of virtual datasets) and application of a number of data manipulation methods that will allow you to create the features you need for an analysis. New features: Additional data manipulation methods as well as a cloning functionality and summary statistics per step are now available. License checks are also performed before allowing a data consumer to use a dataset.
  • Design of an analytics pipeline in an application, containing the algorithms that you want to execute and their parameters along with your organization’s (own and acquired) and virtual datasets. New features: Additional algorithms are now available in addition to various usability improvements (e.g. consistency checks of a workflow, scheduling the execution of an application). License checks for the datasets involved are also performed when the data consumer inserts the application’s metadata.
  • Execution of the data preparation steps and the applications in your Secure Private Space that exclusively belong to your organization for your own analytics use. New features: You can now view the status of the different datasets in your Secure Private Space and decide whether you would like to update them in their latest version.
  • Intuitive visualization of the analytics results, allowing you to customize step-by-step all visualization parameters. New features: Additional visualization charts are now available.

We will soon organize a webinar to demonstrate the functionalities of the ICARUS beta platform so stay tuned for a live demo!

Blog post prepared by Suite5.