---
title: "Cutoff-inclusive generalized effective index method for waveguide design"
authors: ["Valery N Konopsky,1"]
affiliation: "1 Institute of Spectroscopy, Fizicheskaya, 5, Troitsk, Moscow, 108840, Russia"
journal: "Applied Optics"
year: 2025
volume: ""
issue: ""
article_number: ""
pages: ""
doi: "10.1364/AO.567847"
type: journal-article
site_group: ""
url_abstract: ""
url_pdf: "https://valery.konopsky.com/kvnlocal/Cutoff-inclusive%20generalized%20effective%20index%20method%20for%20waveguide%20design.pdf"
language: en
source_tex: "Z:\\ValeryData\\Valery_New\\my_articles\\ApplOpt2025\\ApplOpt\\Manuscript\\Konopsky2AO2025.tex"
source_pdf: "Z:\\ValeryData\\Valery_New\\my_articles\\ApplOpt2025\\ApplOpt\\published\\Cutoff-inclusive generalized effective index method for waveguide design.pdf"
---
Preprint submitted to and published in [ “Applied Optics”, v. 64, No 24 p. 7230-7234](https://doi.org/10.1364/AO.567847) 20 August 2025
The effective index method (EIM) is a simplified semi-analytical technique for analyzing 2D optical waveguides by decomposing them into two sequential 1D calculations. A main limitation of conventional EIM is its inability to account for regions with thickness below the cutoff, where guided modes cannot exist. Furthermore, the accuracy of the method degrades noticeably for modes near the cutoff. To address these shortcomings, we propose a generalized effective index method (gEIM) that handles cutoff problems. This cutoff-inclusive gEIM is particularly advantageous for designing 2D waveguides fabricated on 1D photonic crystals, where modes frequently operate near cutoff and exhibit significant penetration into external media. The proposed method is implemented as a free Windows program, which computes both the effective refractive indices and the spatial profiles of 2D waveguide modes.
# Introduction
Optical waveguide design and analysis are critical for integrated photonics, enabling diverse applications from telecommunications to sensing. Fast and accurate methods to analyze waveguide modes and their spatial profiles are essential for optimizing the design of integrated devices such as lasers, modulators, and sensors.
Although direct numerical methods such as the finite-difference time-domain (FDTD) [1], finite-element [2] and finite-difference eigenmode (FDE) [3] have gained computational feasibility with advancing hardware, the semi-analytical effective index method (EIM) retains prominence in preliminary design phases due to its conceptual simplicity and low computational cost.
The EIM was originally introduced by Knox and Toulios [4] for millimeter-wave structures; however, its most widespread applications emerged in integrated optics [5], where it has been utilized to model various types of 2D optical waveguides. The primary strength of the EIM lies in its intuitive framework, which avoids computationally intensive numerical calculations while providing reasonable estimates for propagation constants and mode profiles. Its efficiency makes it particularly valuable for rapid early-stage prototyping and educational purposes.
However, the conventional EIM has two major limitations. First, it fails in regions where the waveguide thickness is below the cutoff, as the absence of guided modes precludes defining effective indices in such regions. Even attempts to approximate these indices using substrate or cladding refractive indices yield unsatisfactory results and serve only as a palliative. Second, the method’s accuracy deteriorates significantly already for modes near cutoff, where evanescent fields extend deeply into the cladding or substrate [6].
Significant efforts have been devoted to address EIM’s shortcomings and enhancing its accuracy, as evidenced by numerous studies [7, 8, 9, 10, 11, 12, 13]. A comprehensive analysis and critical comparison of numerical and approximate approaches, including these advancements, are provided in [14].
These limitations become particularly critical in applications where a 2D waveguide designed on top of an 1D multilayer structure, which operates near cutoff to allow deep penetration of optical mode intensity into external media [15].
In this work, we present the cutoff-inclusive generalized effective index method (gEIM), which overcomes the traditional EIM’s limitations without compromising its computational efficiency. By redefining the effective index for sub-cutoff regions, the gEIM enables accurate modeling of waveguides on 1D photonic crystals or on other substrates. The method is implemented in a user-friendly Windows application [16], providing both effective indices and field distributions, thereby bridging the gap between analytical simplicity and rigorous numerical accuracy.
# Methods
## Effective Index Method
The traditional EIM concept is illustrated in Fig. 1. The method determines the effective refractive index $N_\mathrm{eff}$ of a 2D waveguide by solving two sequential 1D problems: vertical confinement and lateral guidance. At the first stage, the vertical confinement is resolved, after which the lateral guidance is modeled using the effective indices obtained in the first stage.
*Schematic of the Effective Index Method. The first step: for each lateral slice, the effective refractive index neff.i is computed by solving a 1D mode equation (along x-axis) under the assumption that wi = ∞. The corresponding mode wavevectors are ki = neff.i ω/c. The second step: the 2D waveguide mode is obtained by solving the second 1D problem (along y-axis, top view) with the precomputed indices neff.i, yielding the global effective index of 2D waveguide mode Neff.*
As shown in Fig. 1, during the first stage, the structure is divided into lateral slices of width $\mathrm{w_i}$ along the y-direction. For each slice, the effective index $n_\mathrm{eff.i}$ of a slab waveguide mode in the i-th slice is calculated under the assumption that the slice extends infinitely in the lateral direction (i.e., $\mathrm{w_i} =\infty$).
Then, at the second stage, the structure is analyzed from a top-down perspective (see the top plane in Fig. 1), transforming it into the second 1D problem where each lateral slice is assigned the effective refractive index $n_\mathrm{eff.i}$. Solving this 1D problem yields the 2D waveguide modes of the original structure, propagated in z-direction and characterized by the effective refractive indices $N_\mathrm{eff}$.
The polarization dependence in the EIM is accounted for by inverting the polarization treatment across its two stages. For s-polarized modes (electric field parallel to the layer interfaces), solutions for s-polarization are applied in the first stage, while solutions for p-polarization are applied in the second stage. Conversely, for p-polarized modes, the sequence is reversed: solutions for p-polarization are applied in the first stage, followed by s-polarization solutions in the second stage.
## EIM Near Cutoff: Limitations and Case Study
To demonstrate the limitations of the standard EIM and elucidate the physical causes behind these limitations, we apply the standard EIM to analyze the rib waveguide structure proposed in [15]. The structure is represented as:
``` math
\mathrm{substrate / (LH)^5L''HL_i' / vacuum} \,,
```
where $L$ is the $SiO_2$ layer (thickness $d_L=170. 73$ nm), $H$ is the $TiO_2$ layer (thickness $d_H=86. 46$ nm), $L''$ is the extended $SiO_2$ layer (thickness $d_{L''}=304$ nm), and $L_i'$ is the final $SiO_2$ layer (thickness $d_1=207$ nm for the central slice and $d_{0,2}=182$ nm for the lateral slices). The refractive indices (RIs) of $SiO_2$ and $TiO_2$ at $\lambda=850$ nm are 1.4666 and 2.3137, respectively. The substrate is glass (with a refractive index $\sim 1.5$), but the photonic bandgap of the $(LH)^5$ stack isolates the surface mode from substrate leakage, making the substrate’s index irrelevant. The ability to excite surface modes with effective indices $n_\mathrm{eff}$ close to 1 is the primary motivation for using a 1D photonic crystal here.
This structure sustains a mode near cutoff in the central slice ($n_\mathrm{eff.1}=1.0054$, calculated for $\mathrm{w_1} =\infty$), while no guided modes exist in the lateral slices. The effective index in the central slice can be calculated using the impedance approach [17, 18, 16] or by other methods used for finding modes in multilayer waveguides (see [19] and references therein). The cutoff thickness for the final layer is $d_\mathrm{cutoff}\simeq 193$ nm, below which guided modes cannot exist. Thus, the central slice ($d_1=207$ nm) operates above cutoff, supporting guided modes, whereas the lateral slices ($d_{0;2}=182$ nm) are sub-cutoff,where no guided modes are present. The design objective is to confine a waveguide mode within the rib ($\mathrm{w_1}=3\,\mu$m) that exhibits strong evanescent field penetration into the vacuum cladding, enabling interaction with cold rubidium atoms above the waveguide.
*Effective refractive index Neff of the 2D waveguide as a function of the final layer thickness d0; 2 in the lateral slices, calculated using the standard EIM, FDE and generalized EIM. The dashed line marks the cutoff thickness.*
The results of EIM calculations for the effective refractive indices $N_\mathrm{eff}$ of the analyzed structure are depicted as blue squares in Fig. 2. The figure illustrates $N_\mathrm{eff}$ as a function of the final layer thicknesses in the lateral slices ($d_{0;2}$), while the final layer thickness of the central slice remains fixed at $d_1=207$ nm. For sub-cutoff thicknesses, where guided modes are absent in the lateral slices, the cladding refractive index (RI of vacuum in this case) is employed as the effective indices in these regions (i.e., $n_\mathrm{eff.0;2}=1.0$ for $d_{0;2}<193$ nm — this is a typical palliative in EIM).
Reference data, serving as a standard, were obtained using the FDE solver from Lumerical Inc.’s MODE software (Version 7.21.3262, 2023 R1). These reference points are depicted as black circles in Fig. 2. A significant deviation between standard EIM results and reference data is observed in the sub-cutoff regime. Furthermore, discrepancies persist even near the cutoff region, specifically for $d_{0;2}\sim 199 \dots 194$ nm.
## Generalized Effective Index Method
### Physical causes of EIM failures
To elucidate the physical origin of the discrepancies observed in Fig. 2, we analyze the field intensity profiles in a lateral slice operating near cutoff (for $d_0 =195$ nm, $n_\mathrm{eff.0}=1.00014$) and compare them with the primary mode profile in the central slice (for $d_1= 207$ nm, $n_\mathrm{eff.1}=1.0054$). As revealed in Fig. 3, even small variations in the thickness of the final layer near cutoff induce significant alterations in the field profiles. This behavior arises from a sharp increase of the evanescent field penetration depth into the external medium, while the lateral waveguides approach cutoff conditions ($n_\mathrm{eff.0} \rightarrow 1.0$).
Consequently, the primary field from the central slice will perceive the dielectric permittivity distribution within the lateral multilayer structure ($\varepsilon_\mathrm{0}(x)$) differently than the local field within the lateral region itself. It should be noted that this behavior never occurs at conventional three-dimensional interfaces, where identical intensity profiles on both sides of the boundary are assumed (e.g., plane waves in the Fresnel framework).
Thus, in the studied system, the effective refractive indices of i-th (non-primary) slices, $n_\mathrm{eff.i}$, must be corrected (renormalized) to $n_\mathrm{corr.eff.i}$ prior to the second stage of the EIM to account for the interplay between the central mode $I_1(x)$ and the dielectric permittivity distribution in the lateral regions $\varepsilon_\mathrm{i}(x)$.
*Vertical 1D intensity profiles I(x) obtained during the first stage of the EIM for two final layer thicknesses: d1 = 207 nm (central slice, solid red line) and d0 = 195 nm (lateral slice, dotted blue line). Corresponding dielectric permittivity distributions ε1(x) and ε0(x) are shown as a solid magenta line and a dashed cyan line, respectively.*
### Modification of effective refractive indices in lateral slices
The equations of the weighted index method [20] will be implemented to calculate the changes in the lateral effective refractive indices from the “point of view" of the mode intensity distribution in the central slice.
The weighted index method provides an approximation for modal refractive index $n_\mathrm{eff}$ by relating it to the dielectric permittivity distribution $\varepsilon(x)$ of individual layers via a weighted spatial average, where the weighting factor corresponds to the optical mode intensity profile across the layers:
``` math
\begin{equation}
n_\mathrm{eff}^2 = f(\varepsilon, I) \simeq \frac{ \int \varepsilon(x) I(x)\,\mathrm{d}x }{\int I(x) \,\mathrm{d}x}\, .
\end{equation}
```
In fact, as noted in [20], an additional term — dependent on the mode’s electric field and its second derivative with respect to the spatial coordinate — has to be added to the right-hand side of (1) to ensure mathematical rigor. In our further considerations, however, this term will be embedded within the functional $f(\varepsilon, I)$, manifesting itself as an implicit dependence on $I(x)$, since we focus exclusively on the variation of $n_\mathrm{eff}$ with respect to $\varepsilon(x)$ and $I(x)$.
The variation in the effective index is expressed as:
``` math
\begin{equation}
\Delta n_\mathrm{eff}^2 \simeq \frac{\partial f(\varepsilon, I)}{\partial \varepsilon} \Delta \varepsilon + \frac{\partial f(\varepsilon, I)}{\partial I} \Delta I\, ,
\end{equation}
```
where the partial derivatives reflect the sensitivity of $n_\mathrm{eff}$ to perturbations in spatial distribution of dielectric permittivity $\varepsilon(x)$ and mode intensity $I(x)$.
The corrected (renormalized) effective index $n_\mathrm{corr.eff.i}$ for the i-th lateral slice is derived from Eq. (2) by substituting the intrinsic mode intensity $I_\mathrm{i}(x)$ with the primary mode intensity $I_1(x)$ from the central slice in the weighting:
``` math
\begin{equation}
n_\mathrm{corr.eff.i}^2 - n_\mathrm{eff.i}^2 \simeq \frac{\partial f(\varepsilon_\mathrm{i}, I_\mathrm{i})}{\partial I_\mathrm{i}} (I_1-I_\mathrm{i})\, ,
\end{equation}
```
where the first term from Eq. (2) vanishes, as the dielectric permittivity distribution remains unchanged, and $\Delta \varepsilon =0$. Physically, this correction accounts for how the central mode’s field ‘sees’ the dielectric environment in the lateral slices, which differs from the local mode’s perspective in those slices.
To estimate the partial derivative in Eq. (3) and eliminate the dependence on the intensity distribution in lateral slices, $I_\mathrm{i}(x)$ — which may be undefined for sub-cutoff slices where guided modes are absent — we introduce the following auxiliary expressions:
``` math
\begin{equation}
n_\mathrm{eff.i}^2 - n_\mathrm{eff.1}^2 \simeq \frac{\partial f(\varepsilon_1, I_1)}{\partial \varepsilon_1} \Delta \varepsilon + \frac{\partial f(\varepsilon_1, I_1)}{\partial I_1} (I_\mathrm{i} - I_1) \, ,
\end{equation}
```
where $\Delta \varepsilon = \varepsilon_\mathrm{i}(x) - \varepsilon_1 (x)$. Consequently,
``` math
\begin{equation}
n_\mathrm{eff.i}^2 \simeq n_\mathrm{eff.1}^2 + \frac{ \int \varepsilon_\mathrm{i}(x) I_1(x) \,\mathrm{d}x}{\int I_1(x) \,\mathrm{d}x} - \frac{ \int \varepsilon_1 (x) I_1(x) \,\mathrm{d}x}{ \int I_1(x) \,\mathrm{d}x} + \frac{\partial f(\varepsilon_1, I_1)}{\partial I_1} (I_\mathrm{i} - I_1)\, .
\end{equation}
```
Here, in Eqs. (4) and (5), we utilize expression (2) to relate the modal effective index of a lateral slice, $n_\mathrm{eff.i}$, to that of the primary slice, $n_\mathrm{eff.1}$. Substituting $n_\mathrm{eff.i}$ from (5) into (3) and assuming the approximation
``` math
\begin{equation}
\frac{\partial f(\varepsilon_1, I_1)}{\partial I_1}
\approx \frac{\partial f(\varepsilon_\mathrm{i}, I_\mathrm{i})}{\partial I_\mathrm{i}} \, ,
\end{equation}
```
we obtain:
``` math
\begin{equation}
n_\mathrm{corr.eff.i}^2 \approx n_\mathrm{eff.1}^2 +\frac{ \int \varepsilon_\mathrm{i}(x) I_1(x) \,\mathrm{d}x}{\int I_1(x) \,\mathrm{d}x} - \frac{ \int \varepsilon_1 (x) I_1(x) \,\mathrm{d}x}{ \int I_1(x) \,\mathrm{d}x}\, ,
\end{equation}
```
which does not depend on the intensity distribution in the lateral slices, $I_\mathrm{i}(x)$, and can be applied for all non-primary slices.
### Implementation of gEIM
The equation (7) of the corrected modal effective index for non-primary slices is the main improvement of gEIM that addresses both limitations of the standard EIM. It depends only on the intensity distribution $I_1(x)$ in the primary slice and allows to compute modal effective indices for all lateral slices — including sub-cutoff regions — without requiring guided mode solutions in these regions (using only distributions of their dielectric permittivities).
For 2D structures with more than three slices, the primary slice is chosen by maximizing the product of the modal field intensity within its highest intensity layer and the slice width:
``` math
\begin{equation}
\max_\mathrm{i} \left[ \max_{x} \left( I_\mathrm{i}(x) \right) \cdot \mathrm{w_i} \right]
\end{equation}
```
This criterion ensures the slice dominating the optical mode’s energy distribution is prioritized, enhancing fidelity of gEIM calculations for multi-slice geometries.
# Results
## Practical Implementation in the Program
An implementation of the gEIM approach described in this paper is available as a free Windows program at www.pcbiosensors.com/1DPC4all.htm [16, 18]. Both the program interface and numerical calculations were implemented using the C# programming language within the .NET 8 framework. The program is distributed as a self-contained single file, which includes all necessary components, such as the .NET libraries and target runtime libraries. It is isolated from other .NET applications and does not rely on a locally installed shared runtime. The executable file, `1DPC4all.exe`, runs on any 64-bit Windows system with Windows 7 or later. Users are not required to download or install any versions of .NET.
## Computations of $N_\mathrm{eff}$ for 2D Waveguids
The effective refractive indices $N_\mathrm{eff}$ of the analyzed waveguide structure, computed via the generalized Effective Index Method in the program [16], are plotted as red diamonds in Fig. 2, demonstrating improved agreement with reference numerical data compared to conventional EIM results (blue squares). Thus, replacing all lateral effective indices $n_\mathrm{eff.i}$ with their renormalized counterparts $n_\mathrm{corr.eff.i}$ from Eq. (7), before solving the lateral guidance problem (second stage of EIM), resolves both fundamental limitations of conventional EIM: inaccuracies near the cutoff and failure in sub-cutoff regions.
## Optical Field Intensity Visualization in 2D Waveguides
The two-dimensional optical field intensity distribution is reconstructed by multiplying the 1D intensity profiles along the vertical ($x$) and lateral ($y$) directions, obtained during the first and second stages of the conventional EIM, respectively. This approach preserves the mathematical simplicity of EIM while enabling visualization of the full 2D mode profile. As illustrated in Fig. 4, the reconstructed 2D intensity distribution for the studied waveguide structure exhibits confinement in the central rib region ($\mathrm{w_1}=3\,\mu$m) and significant evanescent tail penetration into the vacuum cladding — a critical feature for applications involving interactions of optical fields with cold atoms.
*Full 2D intensity profile I(x, y) of the fundamental mode in the analyzed waveguide structure, reconstructed via the gEIM by multiplying the orthogonal 1D solutions.*
# Discussion
The development of the Effective Index Method has a long history and likely traces its origins to Marcatili’s method, which was introduced for approximating modes in rectangular waveguides [21]. In subsequent refinements of these approaches, the effective indices in the lateral regions are often approximated using the expression (in the present notation): $n_\mathrm{eff.0;2}^{*2}= \varepsilon_{0;2} - \gamma (\varepsilon_{1} - n_\mathrm{eff.1}^2),$ where the parameter $\gamma$ is empirically fitted to specific perturbation correction models [6, 9, 12, 13]. These studies highlight that setting $\gamma = 0$ recovers the conventional EIM, whereas $\gamma = 1$ corresponds to Marcatili’s method:
``` math
\begin{equation}
n_\mathrm{eff.0;2}^{*2}= n_\mathrm{eff.1}^2 + \varepsilon_{0;2} - \varepsilon_{1} \, .
\end{equation}
```
By comparing Eq. (7) with Eq. (9), we observe that gEIM represents the Marcatili’s model as a special case when the dielectric permittivity contrast spatially correlates with the intensity profile of the central mode. While Marcatili’s approximation may suffice for simple rectangular rod waveguides, it becomes inadequate in more challenging scenarios, such as near-cutoff regimes or complex geometries like the multilayer structure examined in this study.
In contrast, the gEIM framework overcomes these shortcomings by explicitly accounting for the interplay between the central mode’s intensity profile and the lateral dielectric profiles, as captured in Eq. (7). This enhancement enables gEIM to provide a more accurate and versatile approach for modeling a wide range of waveguide configurations. The advancements introduced by gEIM not only improve the precision of mode calculations but also extend the applicability of the effective index approach to sophisticated photonic structures, facilitating the design and analysis of advanced integrated optical devices.
# Conclusion
The generalized Effective Index Method (gEIM), while inherently approximate, provides a rapid and intuitive framework for designing integrated photonic devices, particularly during the initial stages of engineering. By resolving key limitations of conventional EIM — such as inaccuracies near cutoff and failures in sub-cutoff regions — gEIM significantly expands the range of waveguide geometries that can be effectively modeled, including multilayer and near-cutoff structures, bridging the accuracy gap between semi-analytical methods and full numerical simulations. We anticipate that gEIM will serve as a versatile tool for analyzing complex photonic systems where traditional EIM proves inadequate, such as photonic crystal waveguides and sensing platforms requiring strong evanescent field interactions.
# Funding
This work was supported by the Institute of Spectroscopy RAS under state task No. FFUU-2025-0004
10
K. Yee, “Numerical solution of initial boundary value problems involving Maxwell’s equations in isotropic media,” **14**, 302–307 (1966).
B. Rahman and J. Davies, “Finite-element solution of integrated optical waveguides,” **2**, 682–688 (1984).
Z. Zhu and T. G. Brown, “Full-vectorial finite-difference analysis of microstructured optical fibers,” **10**, 853–864 (2002).
R. Knox and P. Toulios, “Integrated circuits for the millimeter through optical frequency range,” in *Proc. of MRI Symp On Submillimeter Waves,* vol. 8 J. Fox, ed. (Brooklyn, NY, 1970), pp. 497–516.
D. Marcuse, “Chapter 8 - Approximate and numerical methods,” in *Theory of Dielectric Optical Waveguides (Second Edition),* D. Marcuse, ed. (Academic Press, 1991), pp. 305–334.
K. Chiang, “Analysis of rectangular dielectric waveguides: effective-index method with built-in perturbation correction,” **28**, 388–390 (1992).
K. Van De Velde, H. Thienpont, and R. Van Geen, “Extending the effective index method for arbitrarily shaped inhomogeneous optical waveguides,” **6**, 1153–1159 (1988).
K. S. Chiang, “Effective-index method for the analysis of optical waveguide couplers and arrays: an asymptotic theory,” **9**, 62–72 (1991).
A. Sharma, “Analysis of integrated optical waveguides: variational method and effective-index method with built-in perturbation correction,” **18**, 1383–1387 (2001).
K. S. Chiang, “Performance of the effective-index method for the analysis of dielectric waveguides,” **16**, 714–716 (1991).
A. Kumar, D. Clark, and B. Culshaw, “Explanation of errors inherent in the effective-index method for analyzing rectangular-core waveguides,” **13**, 1129–1131 (1988).
K. S. Chiang, K. M. Lo, and K. S. Kwok, “Effective-index method with built-in perturbation correction for integrated optical waveguides,” **14**, 223–228 (1996).
Q. Wang, G. Farrell, and T. Freir, “Effective index method for planar lightwave circuits containing directional couplers,” **259**, 133–136 (2006).
K. Chiang, “Review of numerical and approximate methods for the modal analysis of general optical dielectric waveguides,” **26**, S113–S134 (1994).
V. Konopsky, “Photonic crystal surface modes for trapping and waveguiding of ultracold atoms,” **23**, 8812 (2023).
V. N. Konopsky (2025). , Version 2.0.9258.40874.
V. N. Konopsky, “Plasmon-polariton waves in nanofilms on one-dimensional photonic crystal surfaces,” **12**, 093 006 (2010).
V. Konopsky, “Design of 1D photonic crystals sustaining optical surface modes,” **12**, 1489 (2022).
E. Anemogiannis and E. N. Glytsis, “Multilayer waveguides: efficient numerical analysis of general structures,” **10**, 1344–1351 (1992).
M. J. Robertson, P. C. Kendall, S. Ritchie, *et al.*, “The weighted index method: A new technique for analyzing planar optical waveguides,” **7**, 2105–2111 (1989).
E. A. Marcatili, “Dielectric rectangular waveguide and directional coupler for integrated optics,” **48**, 2071–2102 (1969).
---
Generated: 2026-08-30 from Z:\ValeryData\Valery_New\my_articles\ApplOpt2025\ApplOpt\Manuscript\Konopsky2AO2025.tex, validated against Z:\ValeryData\Valery_New\my_articles\ApplOpt2025\ApplOpt\published\Cutoff-inclusive generalized effective index method for waveguide design.pdf.