[Paper Review] Dimension Reduction for Origin-Destination Flow Estimation: Blind Estimation Made Possible

Jingyuan Xia, Wei Dai+2 more|arXiv (Cornell University)|Oct 14, 2018

Economic and Environmental ValuationEconomics, Econometrics and Finance17 references3 citations

TL;DR

This paper proposes a novel dimension reduction approach for origin-destination (OD) flow estimation by modeling flows based on origins (O-flows), reducing the inverse problem's complexity from O(n²) to O(n). Using a Gauss-Seidel method and a necessary uniqueness condition, the method enables blind estimation—accurately recovering true OD flows without prior information in bidirectional networks, with average relative errors below 0.1% in simulations.

ABSTRACT

This paper studies the problem of estimating origin-destination (OD) flows from link flows. As the number of link flows is typically much less than that of OD flows, the inverse problem is severely ill-posed and hence prior information is required to recover the ground truth. The basic approach in the literature relies on a forward model where the so called traffic assignment matrix maps OD flows to link flows. Due to the ill-posedness of the problem, prior information on the assignment matrix and OD flows are typically needed. The main contributions of this paper include a dimension reduction of the inquired flows from $O(n^2)$ to $O(n)$, and a demonstration that for the first time the ground truth OD flows can be uniquely identified with no or little prior information. To cope with the ill-posedness due to the large number of unknowns, a new forward model is developed which does not involve OD flows directly but is built upon the flows characterized only by their origins, henceforth referred as O-flows. The new model preserves all the OD information and more importantly reduces the dimension of the inverse problem substantially. A Gauss-Seidel method is deployed to solve the inverse problem, and a necessary condition for the uniqueness of the solution is proved. Simulations demonstrate that blind estimation where no prior information is available is possible for some network settings. Some challenging network settings are identified and discussed, where a remedy based on temporal patterns of the O-flows is developed and numerically shown effective.

Motivation & Objective

• Address the ill-posed nature of OD flow estimation, where link flows are fewer than OD pairs, leading to non-unique solutions.
• Overcome reliance on prior information (e.g., historical data, assignment matrices) in traditional OD estimation methods.
• Enable blind estimation—recovering true OD flows with no or minimal prior knowledge—by reformulating the inverse problem.
• Demonstrate that unique recovery of OD flows is possible under specific network topologies, particularly bidirectional networks.
• Develop a scalable framework that preserves full OD information while drastically reducing the number of unknowns.

Proposed method

• Introduce a new forward model based on O-flows (flows originating from each node), replacing direct modeling of OD flows.
• Construct a linear mapping from O-flows to link flows, preserving all OD flow information while reducing the number of unknowns from O(n²) to O(n).
• Formulate the inverse problem using a Gauss-Seidel iterative method to solve for O-flows from observed link flows.
• Derive a necessary condition for the uniqueness of the solution, showing that bidirectional networks allow unique recovery under this model.
• Incorporate transform domain sparsity (e.g., wavelet or Fourier) as a regularization strategy for unidirectional networks where uniqueness does not hold.
• Use an optimization framework (Equation 4.11) to minimize error between estimated and observed link flows, with convergence monitored via objective function improvement.

Experimental results

Research questions

• RQ1 Can OD flow estimation be achieved without any prior information on assignment matrices or OD flow patterns?
• RQ2 Is it possible to reduce the dimensionality of the OD flow estimation problem from O(n²) to O(n) while preserving full information?
• RQ3 Under what network structures can the true OD flows be uniquely identified in the absence of prior knowledge?
• RQ4 How effective is the proposed O-flow model in handling real-world network topologies, especially unidirectional ones?
• RQ5 Can transform domain sparsity improve estimation accuracy in networks where uniqueness of solution is not guaranteed?

Key findings

• For bidirectional networks (3×3, 8×8, and GÉANT), the proposed method achieves average relative estimation errors below 0.1%, with over 95% of OD flows having errors within ±0.72%, ±1.14%, and ±0.48% respectively.
• In the 3×3 bidirectional network, more than 95% of estimated OD flows have relative errors between -0.66% and 0.72%, demonstrating high accuracy without prior information.
• The method enables blind estimation—unique recovery of true OD flows—under the proposed O-flow model, particularly in bidirectional networks where the uniqueness condition is satisfied.
• For unidirectional networks, where uniqueness does not hold, applying transform domain sparsity reduces the average relative error from 53% to 3%, with 95% of estimates within [-13%, 7%].
• The Gauss-Seidel algorithm shows slow convergence in unidirectional networks, as the objective function continues to decrease beyond termination, suggesting the solution is not a local minimum.
• The framework is validated on synthetic networks including bidirectional and unidirectional grids and the GÉANT backbone network, showing robustness across diverse topologies.