[Paper Review] Data collaboration analysis for distributed datasets

Akira Imakura, Tetsuya Sakurai|arXiv (Cornell University)|Feb 20, 2019

Privacy-Preserving Technologies in Data9 references4 citations

TL;DR

This paper proposes a data collaboration analysis method for distributed datasets that centralizes only institution-specific intermediate representations—derived via individual, non-shared mapping functions—while preserving data privacy and avoiding model centralization. By aligning these representations into a common collaboration space using anchor data and SVD-based transformation, the method enables centralized learning without sharing raw data or model parameters, achieving improved prediction performance over individual analyses.

ABSTRACT

In this paper, we propose a data collaboration analysis method for distributed datasets. The proposed method is a centralized machine learning while training datasets and models remain distributed over some institutions. Recently, data became large and distributed with decreasing costs of data collection. If we can centralize these distributed datasets and analyse them as one dataset, we expect to obtain novel insight and achieve a higher prediction performance compared with individual analyses on each distributed dataset. However, it is generally difficult to centralize the original datasets due to their huge data size or regarding a privacy-preserving problem. To avoid these difficulties, we propose a data collaboration analysis method for distributed datasets without sharing the original datasets. The proposed method centralizes only intermediate representation constructed individually instead of the original dataset.

Motivation & Objective

• To enable collaborative machine learning on distributed datasets without centralizing raw data or model parameters.
• To address privacy and scalability challenges in large-scale, decentralized data analysis, especially in sensitive domains like healthcare.
• To improve prediction performance beyond individual institution analyses by fusing insights from multiple distributed datasets.
• To develop a method that avoids reliance on cryptography or differential privacy, reducing computational overhead.
• To ensure that original data cannot be reconstructed from intermediate representations by not sharing map functions.

Proposed method

• Each institution computes a unique, institution-specific intermediate representation $\widetilde{X}_i = f_i(X_i)$ using a private, non-invertible map function $f_i$.
• Anchor data (public or dummy) is used to create a common reference space: $\widetilde{X}_i^{\text{anc}} = f_i(X^{\text{anc}})$.
• The intermediate representations from all institutions are centralized and combined into a matrix $[\widetilde{X}_1^{\text{anc}}, \dots, \widetilde{X}_d^{\text{anc}}]$.
• Singular Value Decomposition (SVD) is applied to this combined matrix to extract left singular vectors $U_1$, forming a shared basis for alignment.
• A transformation matrix $G_i = U_1^T (\widetilde{X}_i^{\text{anc}})^\dagger$ is computed per institution to map its own intermediate representation $\widetilde{X}_i$ into the common collaboration space.
• The aligned representations $\widehat{X}_i = G_i \widetilde{X}_i$ are merged into a global training set $\widehat{X}$, which is used to train a model $h$ with ground truth $L$.

Experimental results

Research questions

• RQ1 Can we achieve higher prediction performance in distributed datasets by enabling collaboration without sharing raw data?
• RQ2 Is it possible to align intermediate representations from different institutions with institution-specific maps into a common, usable space?
• RQ3 Can we avoid privacy risks associated with model centralization or cryptographic techniques in federated learning?
• RQ4 How effective is the proposed method compared to individual analysis or fully centralized learning?
• RQ5 Does the use of anchor data and SVD-based transformation preserve utility while ensuring data privacy?

Key findings

• The proposed method enables centralized learning on distributed datasets by aligning institution-specific intermediate representations into a common collaboration space without sharing raw data or model parameters.
• The method avoids privacy risks by not sharing map functions $f_i$, making reconstruction of original data from intermediate representations infeasible.
• By using anchor data and SVD-based transformation, the method achieves alignment of heterogeneous representations across institutions with varying dimensions and mapping functions.
• The approach is computationally efficient compared to privacy-preserving cryptography and differential privacy, as it avoids noise injection or complex encryption.
• The method enables training a single model on the aligned representations, achieving better performance than individual institution analyses.
• The framework is general and compatible with various dimensionality reduction and feature extraction techniques, including PCA, FDA, and deep neural network components.