Data encryption is cryptography process that has been developed and utilised in multiple areas in order to provide enhanced security in the transmission of data between the senders and the legitimate receivers. The scope of data encryption is to address the critical issues of leakage or eavesdropping of sensitive data during the communication of the parties (data in motion) or during the storage of data (data at rest). Hence, data encryption aims to eliminate data disclosure or illegal data interception by malicious users or tools and the preservation of the security, privacy and integrity of data that is stored in any digital form. However, the design and integration of encryption for a big data platform comes with some key challenges.

Symmetric and Asymmetric Data Encryption

As data encryption has attracted remarkable interest over the last year, several cryptographic techniques have been developed by both the research community and the information security industry. These techniques can be classified into two major categories: a) the symmetric encryption, also known as single-key or public-key encryption, which utilizes only one key to encrypt as well as decrypt the underlying sensitive data and b) the asymmetric encryption which utilizes two different keys for the encryption and decryption of the underlying data, namely public key and private key. Depending on the context and the scope of the applied data encryption process, different techniques from both categories are applied.

In the ICARUS perspective, the applied data encryption method employs a dual encryption approach that adheres to: a) the symmetric key encryption exploiting the AES256 symmetric key encryption algorithm in order to encrypt all data assets prior to being uploaded and stored to the platform’s storage and b) the secure SSL handshakes for the secure transmission and sharing of the symmetric between the data providers and the data consumers in order to securely decrypt the data assets before they are utilised in the various platform’s functionalities.

Hence, the data are securely transmitted when: a) they are uploaded from the data provider’s premises to ICARUS platform, b) they are stored in the ICARUS storage or transferred to the ICARUS secure experimentation spaces for the execution of a data analysis and c) they are locally downloaded from a data consumer that has legitimate access (via a smart contract) to the specific data.

Encryption Challenges

Although data encryption provides the most effective solution for the data safeguarding aspects on any ICT platform, nevertheless it raises several challenges and limitations that should be carefully addressed during the design process as well as the implementation process. Especially in the case of big data platform which should effectively handle a large volume of data, data encryption becomes an even more challenging task.

Performance

One of the core challenges is the computation needs and repercussions of data encryption when big data are utilised. Data encryption usually includes computationally intensive and time-consuming tasks when encryption or decryption techniques are applied on the selected data. The performance of these tasks is tightly connected to the size of the data that will be processed, as well as the resources that can be allocated to execute these tasks.

Hence, in the case of big data, this can create a serious bottleneck on the platform’s performance and efficiency. In general, these technologies introduce efficiency problems and delays in data handling in the course of security. However, all big data analytics frameworks are highly dependent in performance and in accessing data for the analysis in order to perform the analysis in a timely and efficient manner. Hence, applying data encryption will certainly introduce an overhead in the data processing and a significant amount of resources will be allocated during its execution. The trade-off for this is the increased level of security that guarantees the security, privacy and integrity of data.

Within the context of ICARUS, the employed dual encryption approach was designed taking into consideration these limitations and ensuring the high performance of the platform without compromising the security level of the data assets.

End-to-end Encryption

Another core challenge is the increase of the trust in uploading data in the platform by the stakeholders of the platform. While data encryption can safeguard the privacy and integrity of data, it should be applied from the beginning of the data upload process. Hence, the option to upload data in an unencrypted format to the data platform in order to be later encrypted and stored in the storage solution is not a viable option for many stakeholders. It is a very common case that stakeholders require that data will never leave their premises unencrypted.

For this reason, in ICARUS, the decision to adopt the end-to-end encryption approach was taken. To implement such as an approach, an investment on building a local client that can be utilised by the stakeholders within their premises environment is required. The specific client should be able to encrypt the data locally and perform the required communication with the platform operating on a cloud infrastructure in order to perform the uploading of the encrypted data. However, the additional challenge of effectively and securely handling the various decryption and key sharing functionalities raises. Within the context of ICARUS, the On-Premise Worker that adheres to the ICARUS dual encryption approach that has been designed and implemented in order to address this challenge.

Key Management

Finally, another challenge of data encryption is the effective and secure key management. The encryption keys are crucial parts of the data encryption process. The disclosure or leakage of the keys will compromise the whole data encryption process. Hence, it is imperative that the encryption keys have constant protection and access is not allowed to them even to the administrators of the platform. The data providers should be given the tool that securely stores and manages the encryption keys within their premises and they maintain full control over their encryption keys.

During decryption, the data providers are the ones that provide the appropriate decryption keys to the data consumers (that have an active data contract) via a mechanism that permits their secure transmission to the data consumers. Additionally, the data providers should be able to revoke these decryption keys for a number of reasons, e.g. if access to these data is revoked or the relevant data contract expired.

Encryption is one the three key mechanisms adopted in ICARUS for the safeguarding of data. If you want to have an overview of the overall ICARUS Data Safeguarding approach, read our relevant blogpost.

Blog post authored by UBITECH.

Featured Image by CoolVid-Shows from Pixabay