After 42 months of intense development and demonstration activities for the ICARUS Platform, we are in a position to identify the quick wins and the painpoints associated with big data platforms for the aviation data value chain as well as to derive significant lessons learnt towards the future exploitation activities of the ICARUS consortium!

The “pentalogue” of an aviation data value chain stakeholder

In order for any aviation data value chain stakeholder to leverage an aviation big data platform and marketplace as ICARUS, the following “pentalogue” needs to be followed:

• Guideline 1. Prepare your data offline. The data need to be extracted from your back-end systems as batch files in csv format or exposed through a well-documented API endpoint. If you opt for the batch file option, you need to pay particular attention to the delimiters, encoding and overall file settings in order to comply with the instructions provided in the ICARUS documentation (available in the Platform’s Help menu).
• Guideline 2. Understand your data. In order to check in your data to the ICARUS Platform, you need to have an inherent understanding of the business meaning hindered behind each field. Knowing the semantics and data types of your data (either directly or through collaboration with IT people or operations experts) is a precondition for properly mapping them to the ICARUS aviation data model and for avoiding errors (that will inevitably follow you afterwards in the ICARUS Platform and will even force you to check in your data again).
• Guideline 3. Invest effort on maintaining your data. Checking in your data once in the ICARUS Platform will undoubtedly provide you with insights in your operations, but you need to keep your data up-to-date in order to really reap the benefits you can gain in a production environment. The effort for updating your data should not be neglected, but appropriately scheduled in appropriate time frames (e.g. when the results of an analysis is also required).
• Guideline 4. Decide internally how you intend to share your data. Data sharing is a mentality that has just started being cultivated across all industries (irrespectively of the fact that there have been aviation data sharing initiatives on a bilateral basis or with aviation authorities for quite some time). The licenses with which you intend to share your data in the ICARUS Platform need to be communicated with your legal team, ensuring that they reflect your organisation’s policies. At the same time, any contracts signed on behalf of your organisation in the ICARUS Platform need to be taken offline and cross-checked with your legal team to include terms that safeguard your organisation’s interests.
• Guideline 5. Adopt a collaboration mentality to really gain insights on your data. Data scientists need to work hand-in-hand with the domain experts in order to prepare your data and run analytics that provide your organisation with new knowledge. The time required to get acquainted with the problem at hand and the business perspective, as well as to iteratively play around with the data, produce new results and explain them to all involved people should not be underestimated.

The anatomy of a data sharing / data science project

The ICARUS Platform is designed to be used by different profiles (i.e. business experts, IT experts and data scientists) that belong to verified organisations in the aviation data value chain and gain organisation-based access (sharing their data check-in jobs, the data assets acquired through a smart contract, the virtual datasets they have created and the analytics applications they have designed). Taking into consideration the ICARUS demonstrators’ experience on the Platform, a typical data sharing and data science project should be designed in 5 phases:

• Phase I: Scoping, aiming at defining and documenting: (a) a concrete business need, (b) the as-is situation versus the desired to-be situation, (c) the challenges to solve and the restrictions to overcome, (d) the data availability status and the profiling of the data needed, (e) the anticipated business impact. Achieving a common understanding of the big picture very early (by aligning the involved experts on objectives and approach), and starting the ICARUS journey on a restricted scope and a more or less complex business scenario (depending on your organisation’s data analytics maturity) while substantiating the value that the ICARUS Platform brings, are key factors to success.
• Phase II: Plan and Implement your Data Check-in Journey in the ICARUS Platform. An organisation’s data need to be properly extracted from their operational systems and prepared in order to be ready to be checked in in the ICARUS Platform: (a) a sample file needs to be created from the full file of the data, (b) different cross-checks need to be performed on the files as instructed in the platform’s documentation, (c) the On-Premise Environment needs to be installed and available in an appropriate infrastructure (server, laptop) with adequate memory and computing resources and high speed and reliable internet connection. Even though the platform provides different data preparation functions for splitting and joining the data for analytics purposes, an organisation should think in advance how their data will be uploaded in light of their future maintenance / update needs, e.g. they can be uploaded per day/month/year or per airport/carrier/location. In this case, a data check-in job is configured in the ICARUS Platform and is executed locally (for batch file upload) or in the cloud (for API data) while for updates (or new airports/carriers/locations, etc), either the same data check-in job is updated or it can be cloned to create a new dataset. In order to collect the right data with the right level of quality, the ICARUS Platform offers a set of mapping and cleaning functionalities for curating the data (i.e. identifying their meaning, handling any missing values, managing outliers, etc) in addition to its anonymisation and end-to-end security features.
• Phase III: Plan and Implement your Data Sharing Strategy in the ICARUS Platform. Deciding how traditionally confidential data can be shared and leveraging aviation data that belong to external stakeholders is the first step towards adopting a data sharing mentality. For many organisations, defining the licenses and determining the terms of a data sharing contract is still not a straightforward path, but requires organisation approval and collaboration with the legal department, both of which require time and effort to ensure a common understanding. Although the ICARUS Platform provides many functionalities to facilitate data sharing (in terms of data search and data acquisition through its query builder and the DLT-based data asset contract mechanism allowing for negotiation), it needs to be acknowledged that it goes far beyond being a mere technical issue.
• Phase IV: Plan and Implement your Analytics Journey in the ICARUS Platform. Although the ICARUS Platform offers an intuitive interface for non-technical users to play around with their organisation’s data, the configuration of an analytics application requires the involvement of a data scientist who knows what to expect from each algorithm and how to utilise it in practice (without having any coding functionalities, though, due to security reasons). In order for an organisation to really gain data-driven insights into its operations, close collaboration between data scientists (that perform data manipulation through virtual datasets, and design an analytics application in the ICARUS Analytics Workspace) and domain experts (who can interpret the results of an analysis) needs to be ensured to capitalise on their experience and bring the best of both worlds. Taking into consideration the acquired feedback and ideas, the data scientist and the business users/domain experts should anticipate many iterations of application training and evaluation to ensure the right hypotheses and interpretations are made until the results and the derived visualisations meet the expectations of the to-be situation defined in Phase I.
• Phase V: Change Management. In the end of the day, any data project is a digital transformation project in terms of requiring a transformation of the ways of working across different operations and departments, which will have to turn to data-driven solutions (and ensure the reliability of the leveraged aviation data) in a context which often relies on experience and more hands-on approaches. To this end, measuring business impact, adapting to the organisation’s culture and including business users in the project design and validation are instrumental to ensure the ICARUS Platform is correctly used for maximum benefits.

When organisations join the ICARUS platform, it is instrumental to organise a training session that will: (a) navigate them to the platform, (b) provide them with the starting material (how to use the available documentation) and make them aware of the guidelines mentioned above, (c) inform them about a live communication channel for getting direct support and the issue tracking mechanism for reporting and monitoring the progress of any issue they may come across.

Since July 1st, 2021, the ICARUS Platform transitions to its Market Push phase when access to external stakeholders is to be provided with priority. If you are interested to join the platform, send us an email at: management@icarus2020.aero

Blog post prepared by Suite5