나형호 교수

This paper proposes an optimal arrangement framework of ballistic missile defense systems, such as a launcher and radar to maximize the survivability of protected assets. In order to solve the optimization problem through a probabilistic approach, we derive a single shot kill probability (SSKP) model based on engagement geometries, trajectory characteristic of an interceptor and a ballistic missile, and error models embedded in both an interceptor and radar. This SSKP model is useful to predict

This paper proposes a new approach on the impact angle control guidance problem for the multistage interceptor with nonmaneuvering phases, called coasting phase, and high acceleration buildup during the boosting phase. First, the required coasting time is determined to reach the target within the given time span considering operation time of the kill vehicle. Then, the required flight path angle profiles and corresponding guidance commands during the boosting phase are derived, in order to achie

This paper proposes a two-phase framework to operate multiple ship defense missiles against the threat of an anti-ship missile. During the preparation phase, single-shot kill probability models associated with various evasive maneuvers of the anti-ship missile are developed and used to assess the overall kill probability obtained by combined operation of multiple defense missiles. An optimization problem to maximize the total kill probability of an engagement by determining the firing times of d

This paper proposes a time-efficient model for solving the Weapon-target assignment (WTA) problem with actor-critic reinforcement learning. While typical heuristic algorithms and recently studied artificial neural network methodologies have shown good performance results, the previous approach has not been time-efficient in large-scale WTA problems. This paper utilizes the actor-critic framework to resolve the WTA problem, and this framework enables retrieving solutions 23 times faster than the

In cooperative multi-agent reinforcement learning (MARL), agents aim to achieve a common goal, such as defeating enemies or scoring a goal. Existing MARL algorithms are effective but still require significant learning time and often get trapped in local optima by complex tasks, subsequently failing to discover a goal-reaching policy. To address this, we introduce Efficient episodic Memory Utilization (EMU) for MARL, with two primary objectives: (a) accelerating reinforcement learning by leveragi

This paper introduces a concept, a baseline design, and a trade study for a new space-based global continuous disaster monitoring system composed of a dual-mode satellite constellation and on-orbit propellant depots. The proposed constellation operates in two different modes: a normal mode and a disaster mode, which are responsible for atmospheric/oceanic imaging and disaster monitoring, respectively. The dual-mode concept enables the system to manage the uncertainties associated with the unknow

View Video Presentation: https://doi.org/10.2514/6.2023-2637.vid This paper proposes a methodology to optimally determine the coasting time of a multi-stage interceptor, considering the engagement zone. Proper coasting time determination is critical for a multi-stage interceptor to extend its engagement boundaries and to engage with a target at a specified engagement point at the estimated impact time. Hence, we first define the optimization problem to determine multiple coasting times for a mul

This paper proposes the initiative to design the continuous and global disaster monitoring satellite constellation supported by a flexibility element referred to as the “onorbit supply depot.” The goal statement, concept of operations, high-level requirements, and key design variables of the proposed monitoring system are presented. The proposed system is operated in two different modes – the nominal mode and the disaster mode. In the nominal mode, the system performs the weather forecasting mis

This paper proposes a framework for the cost/benefit analysis of global disaster monitoring constellation architectures that is operated in two modes: normal weather monitoring mode and disaster monitoring mode. Three architectures are proposed for quantitative cost/benefit analysis: 1) an independent weather monitoring satellite constellation and disposable disaster monitoring satellites, 2) dual-mode satellite constellations supported by an on-orbit fuel depot, and 3) dual-mode satellite const

In cooperative multi-agent reinforcement learning (MARL), agents collaborate to achieve common goals, such as defeating enemies and scoring a goal. However, learning goal-reaching paths toward such a semantic goal takes a considerable amount of time in complex tasks and the trained model often fails to find such paths. To address this, we present LAtent Goal-guided Multi-Agent reinforcement learning (LAGMA), which generates a goal-reaching trajectory in latent space and provides a latent goal-gu

In the context of multi-agent reinforcement learning, generalization is a challenge to solve various tasks that may require different joint policies or coordination without relying on policies specialized for each task. We refer to this type of problem as a multi-task, and we train agents to be versatile in this multi-task setting through a single training process. To address this challenge, we introduce TRajectory-class-Aware Multi-Agent reinforcement learning (TRAMA). In TRAMA, agents recogniz

This paper presents a novel multi-agent reinforcement learning (MARL) approach that incorporates agent priorities to address weapon–target assignment (WTA) with constraints, such as heterogeneous engagement time windows. The proposed approach begins by defining the decentralized Markov decision process (Dec-MDP) formulation for WTA involving heterogeneous, multiple agents. Our approach employs a hierarchical structure for MARL training, comprising an agent selector and a target selector, which s

이 논문은 교전영역을 고려한 다단 요격탄의 최적 설계 프레임워크를 제시한다. 이 프레임워크는 시스템 및 서브시스템 요구 사항을 고려한 복잡한 설계 절차를 처리하기 위해 교차 엔트로피 방법(CEM)을 활용한다. 각 단의 효과적인 질량 분배를 달성하기 위해 최적 단설계는 적용된다. 주목할만한 점은, 제안된 인구 기반 설계 방법은 공력 산출을 위한 Missile DATCOM과 교전 평가를 위한 비선형 시뮬레이션 등의 복잡한 설계 구성 요소에도 훌륭한 수렴 성능을 나타낸다. 비선형 시뮬레이션에서의 요격탄 모델은 무유도 단계 및 관련 유도법칙을 적용하였으며, 따라서 제안된 프레임워크는 최적 설계 단계에서 다단 요격탄의 운용 특성을 고려할 수 있다. 사례 연구를 통해 제안된 프레임워크가 시스템 요구 사항과 서브시스템 제약 사항 만족하는 설계 결과를 효과적으로 도출할 수 있음을 보인다.

Large language models (LLMs) are commonly aligned with human preferences using reinforcement learning from human feedback (RLHF). In this method, LLM policies are generally optimized through reward maximization with Kullback-Leibler (KL) divergence regularization of the reference policy. However, KL and its $f$-divergence variants only compare token probabilities at identical indices, failing to capture semantic similarity. We propose Wasserstein Policy Regularization (WPR), a semantic-aware reg