[Paper Review] Maximally Localized Wannier Orbitals, Interaction Models and Fractional Quantum Anomalous Hall Effect in Twisted Bilayer MoTe2

Cheng Xu, Jiangxu Li+3 more|arXiv (Cornell University)|Aug 18, 2023

Quantum and electron transport phenomena27 references4 citations

TL;DR

This study constructs maximally localized Wannier functions and minimal two-band Hubbard-like models for twisted bilayer MoTe2, revealing that multi-band effects—particularly band mixing—shift the optimal twist angle for fractional quantum anomalous Hall (FQAH) states away from the magic angle. Exact diagonalization shows that strong inter-band coupling suppresses FQAH stability in the single-band approximation, necessitating multi-band treatments for accurate predictions of topological phases in this system.

ABSTRACT

We investigate the moiré band structures and the strong correlation effects in twisted bilayer MoTe$_2$ for a wide range of twist angles, employing a combination of various techniques. Using large-scale first principles calculations, we pinpoint realistic continuum modeling parameters, subsequently deriving the maximally localized Wannier functions for the top three moiré bands. Simplifying our model with reasonable assumptions, we obtain a minimal two-band model, encompassing Coulomb repulsion, correlated hopping, and spin exchange. Our minimal interaction models pave the way for further exploration of the rich many-body physics in twisted MoTe$_2$. Furthermore, we explore the phase diagrams of the system through Hartree-Fock approximation and exact diagonalization. Our two-band exact diagonalization analysis underscores significant band-mixing effects in this system, which enlarge the optimal twist angle for fractional quantum anomalous Hall states.

Motivation & Objective

• To understand the emergence of fractional quantum anomalous Hall (FQAH) states in twisted bilayer MoTe2 at experimentally relevant twist angles.
• To construct accurate minimal interaction models from first-principles calculations for studying many-body physics in MoTe2 moiré systems.
• To investigate the role of band mixing and multi-band effects in stabilizing or suppressing FQAH states beyond single-band approximations.
• To provide a minimal, physically motivated model for future numerical and theoretical studies of topological and correlated phases in TMD moiré materials.

Proposed method

• Large-scale density functional theory (DFT) calculations are used to fit continuum model parameters for twisted MoTe2 across a range of twist angles.
• Maximally localized Wannier functions (MLWFs) are constructed for the top three moiré bands, revealing strong layer polarization and sensitivity to gating.
• A band disentanglement procedure reduces the three-band model to a minimal two-band effective model including Coulomb repulsion, correlated hopping, and spin exchange.
• Hartree-Fock and exact diagonalization (ED) are applied to the two-band model to map phase diagrams and probe FQAH states at fillings ν = -1 and ν = -2/3.
• Multi-band projected ED is used to compare with single-band ED, revealing significant band-mixing effects that alter the optimal twist angle for FQAH states.
• The analysis incorporates the trace condition and Berry curvature, but emphasizes its insufficiency in multi-band regimes due to strong inter-band coupling.

Experimental results

Research questions

• RQ1 How do multi-band effects influence the stability and optimal twist angle for fractional quantum anomalous Hall states in twisted MoTe2?
• RQ2 What is the role of band mixing and inter-band coupling in modifying the phase diagram of the two-band model at fractional fillings?
• RQ3 Why do experimental observations of FQAH states occur at larger twist angles (θ ≈ 3.3°–3.7°) rather than at the magic angle with minimal bandwidth?
• RQ4 How does the inclusion of a displacement field affect the topological phase transition from quantum anomalous Hall to trivial charge-transfer insulator?
• RQ5 What is the nature of the gate-induced phase transition observed at ν = -2/3, and how does it challenge conventional Landau-Ginzburg paradigms?

Key findings

• The top three moiré bands in twisted MoTe2 are topologically trivial as a whole, despite each individual band having a nontrivial Chern number, indicating complex many-body topological behavior.
• Maximally localized Wannier functions for the top three bands exhibit strong layer polarization, indicating high sensitivity to external gating fields.
• The optimal twist angle for FQAH states shifts away from the magic angle in the two-band exact diagonalization due to significant band-mixing effects, contradicting single-band predictions.
• A field-driven continuous topological transition from quantum anomalous Hall to trivial charge-transfer insulator is observed at ν = -1, consistent with experimental transport data.
• Exact diagonalization reveals a gap closure in the FQAH state under gating at ν = -2/3, suggesting the presence of an exotic quantum critical point beyond the Landau-Ginzburg framework.
• The compressible state at ν = -1/2 remains unexplained, and the study positions the derived minimal model as a foundation for future investigation of such exotic states.