[Paper Review] A data Grid testbed environment in Gigabit WAN with HPSS

A. Manabe, Kohki Ishikawa+12 more|ArXiv.org|Jun 12, 2003

Distributed and Parallel Computing Systems1 references3 citations

TL;DR

This paper presents a high-performance data Grid testbed connecting ICEPP and KEK via SuperSINET's 1-Gbps network, integrating HPSS as a scalable storage solution for LHC-tier data. Using NorduGrid middleware and GSI-enabled pftp, the study evaluates HPSS performance over high-speed WAN, finding that WAN access speed is approximately half of LAN speed due to latency, and identifies architectural bottlenecks when pftpd and disk movers are separated on different hosts.

ABSTRACT

For data analysis of large-scale experiments such as LHC Atlas and other Japanese high energy and nuclear physics projects, we have constructed a Grid test bed at ICEPP and KEK. These institutes are connected to national scientific gigabit network backbone called SuperSINET. In our test bed, we have installed NorduGrid middleware based on Globus, and connected 120TB HPSS at KEK as a large scale data store. Atlas simulation data at ICEPP has been transferred and accessed using SuperSINET. We have tested various performances and characteristics of HPSS through this high speed WAN. The measurement includes comparison between computing and storage resources are tightly coupled with low latency LAN and long distant WAN.

Motivation & Objective

• To establish a high-speed Grid testbed for large-scale HEP data processing using HPSS as a scalable storage solution.
• To evaluate the performance of HPSS over a high-speed WAN (SuperSINET) in comparison to local LAN conditions.
• To integrate Grid Security Infrastructure (GSI) into HPSS via GSI-enabled pftp for secure, scalable data access in a Grid environment.
• To identify performance bottlenecks in HPSS data transfer, particularly when pftpd and disk movers are on separate hosts.
• To assess the feasibility and performance of using GSI-enabled pftp as an interim solution for HPSS-Grid integration.

Proposed method

• Deployed a Grid testbed with NorduGrid middleware on 1-Gbps Ethernet across ICEPP and KEK, connected via SuperSINET.
• Integrated 120TB HPSS at KEK as a centralized storage system with disk and tape movers, using HPSS 4.3 and pftp for data access.
• Conducted end-to-end performance measurements using netperf for TCP bandwidth and latency, and HPSS client API for file transfer speeds.
• Evaluated GSI-enabled pftp (from LBL) as a Grid-compliant interface to HPSS, comparing it with standard pftp and GridFTP.
• Measured transfer performance across LAN and WAN using 2GB files, varying buffer size, number of parallel sessions, and data path configurations.
• Analyzed communication patterns between clients, disk movers, and pftpd servers to identify performance bottlenecks in data flow.

Experimental results

Research questions

• RQ1 How does WAN latency affect HPSS data transfer performance compared to LAN in a high-speed Grid environment?
• RQ2 What is the performance impact of using GSI-enabled pftp versus standard pftp for HPSS access in a Grid setting?
• RQ3 How does separating the pftpd server from the disk mover affect end-to-end data transfer throughput?
• RQ4 What are the scalability limits of HPSS under concurrent multi-stream transfers over a 1-Gbps WAN?
• RQ5 Can GSI-enabled pftp serve as a viable interim solution for integrating HPSS with Grid infrastructure?

Key findings

• WAN access speed using HPSS pftp and client API was approximately half of LAN speed due to round-trip time (RTT) of 3–4 ms, despite minimal packet loss (<0.1%).
• Network throughput saturated beyond 0.5MB TCP buffer size, with no further improvement from increasing buffer size beyond 1MB.
• Using up to four parallel TCP streams improved throughput up to 100KB buffer size, but multiple streams did not improve aggregate speed beyond 1MB buffer size.
• GSI-enabled pftp achieved equivalent transfer times to standard pftp in single-stream scenarios, but transfer speed was halved when pftpd and disk movers were on separate hosts due to double network hops.
• The data path in GSI-enabled pftp required two network hops (disk mover → pftpd → client), unlike standard pftp where the data path was direct (client → disk mover), causing performance degradation.
• The pdata-push protocol in HPSS 5.1 could improve performance by reducing protocol handshake latency, but was not yet available in the test environment.