Geoelectric fields at the Earth's surface caused by magnetic storms constitute a hazard to the operation of electric power grids and related infrastructure. The ability to estimate these geoelectric fields in close to real time and provide local predictions would better equip the industry to mitigate negative impacts on their operations. Here we report progress toward this goal: development of robust algorithms that convolve a magnetic storm time series with a frequency domain impedance for a realistic three‐dimensional (3‐D) Earth, to estimate the local, storm time geoelectric field. Both frequency domain and time domain approaches are presented and validated against storm time geoelectric field data measured in Japan. The methods are then compared in the context of a real‐time application.

Magnetic storms can pose significant risks to critical technology infrastructure such as the electric power grids. Real‐time estimation of geoelectric fields can give the grid operators an opportunity for prompt intervention that would mitigate the long‐term consequences. We develop and validate a method to efficiently and accurately (to within 85–90%) “nowcast” storm time geoelectric fields in close to real time in places where both geomagnetic field data are collected or estimated, and a magnetotelluric impedance has been previously measured. This method can be applied in all regions where magnetotelluric data have been collected, including in the United States, where USArray magnetotelluric data survey is ongoing and by 2018 will cover about two thirds of the country. In this work, we focus on the algorithm development only.

Magnetic storms can induce geoelectric fields in the Earth's electrically conducting interior, interfering with the operation of electric power grids and pipelines [e.g., *Albertson et al.*, ; *Boteler*, ; *Samuelsson*, ; *Piccinelli and Krausmann*, ]. The potential risks were dramatically demonstrated during the magnetic storm of March 1989 [e.g., *Allen et al.*, ], which caused strong geomagnetically induced currents (GICs) to flow along the power lines, collapsing the entire Hydro‐Québec electric power grid in Canada [*Bolduc*, ; *Béland and Small*, ], as well as causing severe damage to step‐up transformers in Delaware Bay (see *North American Electric Reliability Corporation (NERC)* [], for a compilation of effects). However, in all likelihood, humanity's technological age has not yet lasted long enough to experience the solar activity to its full potential. Under some scenarios, the future occurrence of a rare but extremely intense magnetic superstorm could well cause a similar or more severe cascading failure in power transmission, resulting in a widespread blackout that would cause significant harm to our society's economy and well‐being [*Baker et al.*, ; *NERC*, ]. Significant hazards that solar activity poses to our modern day infrastructure have motivated regulatory agencies to require utility companies to take measures to mitigate the potential impact of geomagnetic disturbances on the operation of bulk‐electric power systems [*Federal Energy Regulatory Commission (FERC)*, , Order No. 779; *FERC*, , Order No. 830], and the significance of the hazard is also recognized at the U.S. National Strategic level [e.g., *National Science and Technology Council*, , Goals 1.1 and 5.5].

Accurate calculation of geoelectric fields, ideally in close to real time, is an important goal for studies concerned with induction hazard monitoring and assessment [e.g., *Thomson et al.*, ; *Love et al.*, ; *Pulkkinen et al.*, ]. A focused effort by the U.S. government agencies to provide an electric field nowcasting capability to the public was originally motivated by the industry consensus at the NOAA Space Weather Workshop 2011. Indeed, a high‐resolution regional nowcast is a more precise indicator of induction than *K**p* index (a mean standardized 3 h index measure from 13 geomagnetic observatories between 44° and 60° northern or southern geomagnetic latitude). Incorporated into daily industrial workflow, nowcasting could improve situational awareness and allow for more targeted and therefore less economically damaging action. It would also provide a key stepping stone toward a prediction and mitigation workflow.

Such calculations convolve a time series of geomagnetic field data, streaming in real time, with a surface impedance tensor that is, itself, a function of the subsurface electrical conductivity structure. Calculation of synthetic storm time geoelectric fields have been performed for many geographic regions [e.g., *Boteler*, ; *Gannon et al.*, , ; *Viljanen et al.*, ; *Wei et al.*, ; *Marti et al.*, ] and used in scenario simulations of geomagnetically induced currents in power grids [e.g., *Pulkkinen et al.*, ; *Viljanen et al.*, ; *Torta et al.*, ]. These calculations have used Earth impedances derived under the assumption that local stratigraphy, tectonic structure, and rock properties can be reasonably modeled as a one‐dimensional (1‐D) function of depth [e.g., *Ferguson and Odwar*, ; *Fernberg*, ]. Other analyses have demonstrated that 3‐D Earth structure can have a significant effect on the local amplitude, polarization, and phase of geoelectric fields [e.g., *McKay et al.*, ; *Bedrosian and Love*, ]. For example, *Bedrosian and Love*[] showed that the amplitude of geoelectric fields can differ by an order of magnitude between two locations 70 km apart due to a 2–3 orders of magnitude variation in Earth's conductivity often observed for North American lithosphere. A recent publication by *Weigel* [] performed a comparative analysis of estimated quiet time geoelectric fields that further illustrated the limitations of using the 1‐D approximation, compared to realistic 3‐D Earth impedances, for geoelectric field estimation. Another recent work by *Bonner and Schultz* [] has put forward a procedure to interpolate sparse observatory geomagnetic fields to an arbitrary site location and use the 3‐D Earth impedances to estimate surface geoelectric fields using a frequency domain algorithm. *Bonner and Schultz* [] validated their method using a quiet time USArray time series [*Schultz et al.*, ] and compared the results with those using a 1‐D approximation of the Earth. They confirmed that 3‐D Earth impedances result in significantly better estimates of geoelectric fields compared to those obtained from 1‐D electrical conductivity profiles. We revisit this issue in the appendices of this manuscript, where we consider the applicability of a 1‐D method to geoelectric field estimation in Japan and justify our choice to work with a 3‐D Earth impedance. We note, however, that empirical or modeled 3‐D Earth impedances are accurate at the precise location where they are obtained. In this sense, 3‐D Earth impedances and the estimated geoelectric fields are local and cannot be used in a regional sense. For GIC computations, local geoelectric field estimates would need to be integrated along a power line, which amounts to an averaging procedure that may, in some places, be comparable to using a regional 1‐D impedance. Such a comparison is beyond the scope of this manuscript.

As we further discuss in section 2, the Earth impedance is best represented in the frequency domain. Converting geomagnetic field time series to the frequency domain is also reasonably straightforward, so prior electric field computations have mostly adopted that approach. While a time domain approach, in which the Earth impedance is converted to an impulse response, might have some advantages over frequency domain for real‐time natural hazard applications, no comprehensive comparisons of these two formulations have ever been performed. Variants of time‐domain approaches have been used in the past for 1‐D conductivity models [e.g., *Marti et al.*, and *Kappenman et al.*, ]. In contrast, the current work considers the application of time‐domain methods for real‐time electric field “nowcasting” using more realistic 3‐D Earth impedances. As compared to, e.g., *Marti et al.* [], our method is uniquely capable of not only using the empirical magnetotelluric (MT) impedances for electric field estimation but also of accounting for their uncertainties: a necessary step to progress to using empirical MT impedances such as those provided by USArray [*Schultz et al.*, ] for practical geoelectric field estimation. This has allowed us to validate our technique against measured storm time geoelectric field data and to quantify storm time electric field estimation errors with unprecedented accuracy. We focus on the development and validation of a novel time domain geoelectric field computation algorithm based on *Egbert* [] discrete time impulse response (DTIR) technique developed initially for the 1‐D impedance. We also provide a side‐by‐side comparison of our method with the more traditional frequency domain approach to the problem.

We start with an introductory section that provides a high‐level overview of the method of magnetotellurics (MT) and defines the magnetotelluric response function (section 2). We then describe the standard frequency domain geoelectric field estimation method (section 3) and introduce the discrete time impulse response technique (section 4). For convenience of the reader, technical details of the time domain algorithm implementation are provided in the appendices. We then compare the time domain method with the frequency domain discrete Fourier transform method and discuss the relative benefits of both approaches while also validating these methods with respect to measured storm time geoelectric field data (section 5). We discuss the goodness of fit and the limitations of using a local response function for geoelectric field estimation in section 6. In section 7, we apply both methods in real time and discuss the additional errors that are involved due to a real‐time application of the time domain algorithm, as compared to the errors resulting from a similar application of the frequency domain method. Due to the additional complexity related to a causal algorithm implementation and its validation, a general real‐time implementation of this method is outside of the scope of this work and will be explored in detail in a subsequent publication. We conclude our analysis with a summary and suggestions for future work in section 8.

The methods of magnetotellurics (MT) [e.g., *Cagniard*, and *Chave and Jones*, ] are generally concerned with using ground level geomagnetic and geoelectric field measurements to estimate the electrical conductivity of the solid Earth. In magnetotellurics, conductivity models are obtained from a derived data product known as the *magnetotelluric (MT) response tensor* **Z**(*f*,*x*,*y*), which is also sometimes referred to as an *MT transfer function*. The MT response tensor is a frequency‐dependent relationship between the horizontal components of the geomagnetic field at the Earth's surface and the horizontal components of the ground level geoelectric field:*Weidelt and Chave*, , Chapter 4.1.2], where the horizontal geomagnetic **B**_{h}(*t*) and geoelectric **E**_{h}(*t*) time series have been Fourier transformed into a frequency domain representation at a discrete set of frequencies, (*x*,*y*) is Earth's surface location and *f* is frequency. At each discrete frequency *f* and location (*x*,*y*), **Z** is a 2 × 2 complex‐valued matrix and is related to the *MT impedance* *μ***Z** (measured in ohms), where *μ* is the magnetic permeability via a simple rescaling of units. To avoid potential confusion caused by the multiple unit conventions used to describe the MT impedance, throughout this manuscript we employ the term “MT response” for **Z** and use the following practical units: **B** is measured in nT, **E** in mV/km, and the MT response tensor has the units of (mV/km)/nT, a convention that is commonly accepted within academia and industry.

Equation can be simplified for certain idealized Earth conductivity structures. For a half‐space of uniform conductivity, the MT response is proportional to the square root of the inductional frequency *f*, and the time‐varying sinusoidal amplitude of the geoelectric field leads that of the geomagnetic field by a phase of *π*/4 (or 45°). For an idealized Earth conductivity that varies only with depth (1‐D) [e.g., *Simpson and Bahr*, , Chapter 2.5], the response tensor is antisymmetric,*Z* is complex‐valued and frequency‐dependent. For an idealized 1‐D response, geomagnetic activity **B**_{h}(*t*_{i}) aligned in a given horizontal direction will induce an orthogonal geoelectric field **E**_{h}(*t*_{i}), so that **E**_{h}·**B**_{h}=0. In the presence of a clear geoelectric strike such that the Earth conductivity varies with depth and in one lateral direction, a two‐dimensional (2‐D) approximation may be appropriate,*E*_{x}=*Z*_{xy}*B*_{y} and *E*_{y}=*Z*_{yx}*B*_{x}, with the fields rotated to point along and perpendicular to geoelectric strike.

In general, none of these simple functional relationships are observed in empirical response tensors [e.g., *Schultz et al.*, ] which are generally fully populated and asymmetric [e.g., *Chave*, ],*Z*_{xx}, has no simple mathematical relationship to the other response tensor elements, *Z*_{xy}, *Z*_{yx}, and *Z*_{yy}.

Traditionally, magnetotelluric methods have allowed for 1‐D or 2‐D interpretations in terms of the Earth conductivity, using the simplified versions of the MT responses appropriate for these special cases (equations and , respectively). The past decade, however, has seen the advent of 3‐D magnetotellurics, which allows estimation of electrical conductivity structure within a 3‐D Earth volume with reasonably high resolution that is commensurate with station spacing, and ranges from hundreds of meters for local studies to tens of kilometers on continental scales.

Magnetotelluric inversion is a computationally intensive, nonlinear and nonunique inverse problem [e.g., *Siripunvaraporn et al.*, ; *Egbert and Kelbert*, ]. Thus, there could be a variety of 3‐D models that fit the measured MT responses to within their estimated precision. A phenomenon known as *galvanic distortion*, which refers to small‐scale, local structures obscuring the regional signal in the data, is also a problem [e.g., *Groom and Bailey*, ]. However, large‐scale 3‐D MT inversions of recent years [e.g., *Meqbel et al.*, ; *Bedrosian and Feucht*, ; *Yang et al.*, ] have demonstrated significant correlations with independently observed geological structures. Three‐dimensional MT responses can be either empirical, i.e., estimated from measured data, or computed from a 3‐D electrical conductivity model obtained by inverting the empirical responses. Here we explore storm time prediction efficiency of empirical 3‐D MT responses obtained from measured geomagnetic and geoelectric field data (section 5).

A historical approach to estimation of the MT response [e.g., *Sims et al.*, ] is to Fourier transform the magnetic and electric field time series directly and to solve equation for the MT response via a least squares procedure. The simple direct (unweighted) least squares approach relies on the assumptions that the error variances are independent of signal power and that the errors are Gaussian and uncorrelated. In MT applications, these assumptions have been repeatedly shown to fail in the presence of outliers or significant source effects, such as during a magnetic storm (see, e.g., *Egbert and Booker* [] for a thorough analysis of this issue). Instead, a variety of data adaptive robust estimation schemes for MT responses have been developed over the years (most notably, *Egbert and Booker* [], *Chave et al.* [], *Smirnov and Egbert* [], and *Chave* []. These techniques employ variants of least squares method that are resistant to violations of distributional assumptions, leverage points, and outliers. Robust estimates produce an MT response tensor that does not vary depending on the segment of the time series that is used for the estimate, except within the error bars, thus reducing the dependence of the MT response on the ionospheric and other nonperiodic (e.g., cultural) source effects.

By design, the robust MT response estimates are affected less by inclusion of storm time data than the simple direct least squares approach. Except in the presence of periodic cultural, tidal, or temperature effects, or at high latitudes where a non‐plane wave signal could dominate during a magnetic storm, a robust MT response is a function of the Earth conductivity and will not vary whether obtained using storm time data or during a quiet time. The enhanced signal‐to‐noise ratio observed during a magnetic storm does allow better constraint of the error bars, but this signal is otherwise downweighted in the robust estimation procedure. Indeed, the basic assumption of the MT method is a plane wave structure of the ionospheric source: an approximation that may well fail during a severe storm, especially at higher geomagnetic latitudes. This has led some to reconsider the need for robust estimation in application to the GIC prediction problem. For instance, *Weigel* [] demonstrates that under certain circumstances (such as an unusually clean time series segment), the simple direct least squares technique allows for a slightly better estimate of the geoelectric field. However, it should be noted that in the examples that were considered by *Weigel* [], the adjacent in‐sample and out‐of‐sample time series segments were likely affected by the same or very similar ionospheric source configurations. As all storms are distinct from one another, it is an open question whether these estimates would still do a reasonable job at estimating the geoelectric fields at a different time. Another word of warning is that clean and unbiased electric time series are an extremely rare occurrence and it often takes an expert to identify noise and bias. Careful preprocessing of the time series is needed to minimize the effects of noise and bias during robust MT response estimation and especially so for simple direct least squares.

Finally, the components of the MT response tensor at frequency *f* are often represented as so‐called apparent resistivity *ρ*_{ij} and phase *ϕ*_{ij} [*Vozoff*, ]:*i**j* stands for the response tensor components *x**x*, *x**y*, *y**x*, and *y**y*. Typically, only the two off‐diagonal components are plotted for interpretation on apparent resistivity and phase plots of the MT response. This is because the phases are not well‐defined as the diagonal impedance components approach zero, as is observed for near 1‐D conductivities. Apparent resistivity has the units of Ohm m and has direct physical interpretation for simple Earth models such as a uniform half‐space. In section 5, we plot apparent resistivities and phases along with the full MT response, as a function of period.

As is evident from section 2, estimation of geoelectric fields at the Earth's surface is not a traditional goal for the science of magnetotellurics. It does however come naturally as a byproduct of the well‐developed MT tools. In the United States, for example, many robust MT response tensors are now available at the database of *Kelbert et al.* [], with more MT data being collected and processed daily by the NSF's USArray MT program [*Schultz et al.*, ] and many other initiatives. These data can be reused, in conjunction with continuous geomagnetic field measurements [e.g., *Love and Finn*, ; *Lam*, ], to estimate surface geoelectric fields in close to real time.

The frequency domain method for estimating the electric field time series begins by carrying out a discrete Fourier transform (DFT) of the horizontal components of geomagnetic field time series **B**_{h}(**t**) to derive the frequency domain components *N* discrete samples collected over a finite interval of *N*∗*δ**t* seconds, at times *t*_{k}=*k*∗*δ**t*, *k* = 0,1,2,…,*N* − 1, the N‐point DFT is obtained by*f*_{m}=*m*/(*N**δ**t*), which defines *δ**f* = 1/(*N**δ**t*) as the frequency resolution of *Boteler* []).

We note that the exponent 2*π**f*_{m}*t*_{k}=2*π**m**δ**f**k**δ**t* = 2*π**m**k*/*N*, so the functions *T*) and are also periodic in frequency on the interval [0,*N**d**f*), for *m* = 0, 1, 2, …, *N* − 1 and for *k* = 0, 1, 2, …, *N* − 1, i.e.,

The next step is to infer the MT response tensor at the discrete frequencies *f*_{m}=*m**δ**f*, which typically involves interpolation and resampling. We do the resampling for all the frequencies from 0 up to the Nyquist frequency (*N*/2) × *δ**f*. For real‐valued time series such as the **B**_{h}(*t*_{k}), the frequencies with *m* > *N*/2 are conjugates of frequencies at negative indices (*m* − *N*). For each such conjugate frequency, there is a corresponding positive frequency index (*N* − *m*), and it can be shown that since both **B**_{h}(*t*_{k}) and **E**_{h}(*t*_{k}) are real‐valued functions, the MT response at a negative frequency is the complex conjugate of the MT response at the corresponding positive frequency.

Having obtained **f**, we can apply equation to obtain the geoelectric field in the frequency domain,

We note that the calculated geoelectric field **E**_{h}(**t**) is a linear superposition of periodic functions on the interval [0,*T*), which necessarily means that **E**_{h}(**t**) is also periodic on [0,*T*). In practice, for a given sample of observations **B**_{h}(**t**) there often will be a significant difference between **B**(0) and **B**(*t*_{N − 1}). The DFT analysis presumes that **B**_{h}(**t**) is periodic, implying that the point **B**(*t*_{N}) = **B**(0). Typically, then, the DFT sees a step function in the time series from *t*_{N − 1} to *t*_{N}, which will add unphysical spectral power at the higher frequencies in *Viljanen et al.*, ], or by “padding” the start and end of the time series by a certain number of points. For the demonstration presented in this paper, we have chosen to use 83 min of padding or 5000 points for 1 s data, which we found to be optimal for our purposes. Rather than padding with zero values as would be more standard but introduces a step function just before *t*_{0} and just after *t*_{N − 1}, we pad by repeating the first and the last values in the time series for the chosen number of minutes—a strategy that we observed to work well, validated by practical tests with multiple MT responses.

The geoelectric fields at the Earth's surface exhibit nontrivial variability in both space and time. The process of electromagnetic induction in a (essentially time stationary) 3‐D Earth causes the geoelectric fields to vary from one location to another; however, the time variations in the geoelectric fields are caused by transient and nonstationary magnetic disturbances. These field characteristics are reflected in the geomagnetic field measurements that come as a discrete time series at a certain sampling rate (these days, usually at 1 s intervals) [e.g., *Love and Chulliat*, ]. While the frequency domain approach of section 3 allows one to obtain reasonably accurate geoelectric field estimates using empirical MT response tensors [e.g., *Weigel*, ; *Bonner and Schultz*, ], it is of interest to also explore time domain approaches that may be better suited for treating transient signals. We therefore consider an alternative approach that presents geoelectric field prediction as a discrete time domain problem. We proceed to describe the methodology here and will revisit its real‐time application in section 7.

By the convolution theorem [e.g., *Bracewell*, , Chapter 3], the continuous time‐domain expression of is given by the integral*Weidelt*, , p. 264], so that *t* < 0. But equation is an idealization; most importantly, it assumes **B**_{h}(*t*) to have infinitesimal time resolution, which is equivalent to no frequency band limit. In practice, **B** data are band limited, since they are collected as discrete samples after analog filtering through an acquisition system, as well as over a finite duration of time. In this case, the continuous equation is replaced by the discrete equation approximation*N*_{t−},*N*_{t+}] is implied, *N*_{t−}≤0 and *N*_{t+}>0 being integer lengths of time measured in the number of samples in the negative and positive time directions, respectively.

Here *Egbert* []. In contrast with its continuous form in equation , equation is acausal [*Egbert*, ] even for the simplest 1‐D problem that is strictly causal in continuous time. In other words, *t*_{k}<0. This can be understood if we notice that the discrete sampling in the frequency domain can be represented by a set of “boxcar” functions in the continuous frequency domain, where each impedance value is constant over a frequency band, then jumps discontinuously to the following frequency band. The Fourier transform of a boxcar is proportional to a sinc function,

In practice, we obtain an estimate of the DTIR by solving equation 11 of *Egbert* [] using a regularized least squares approach:*N*_{f} is the number of frequencies (equivalently, periods) in the discretely sampled frequency domain MT response estimate, *f*_{m} is the *n*th sampling frequency, and *t*_{n}=*n**δ**t*, where *δ**t* is the sampling rate of the time series, and *ε*_{m} is the corresponding complex estimation error for the *m*th response tensor, with variance

Consider a real‐valued vector of MT response tensor components at a single frequency *f*_{m}:**z**_{f} the discrete vector of frequency‐domain MT responses **Z**(*f*_{m}) for all frequencies *m* = 1,…,*N*_{f}; **z**_{f}, then, is of length 8 × *N*_{f}. The truncated DTIR representation of *n* = *N*_{t−},…,0,…,*N*_{t+}, where *t*_{n}=*n*∗*δ**t*, is denoted by the vector **z**_{t}. The length of the impulse response vector is thus *N*_{t}=*N*_{t+}−*N*_{t−}+1. Note that since the DTIR is real and the frequency domain MT response is complex, we solve the system in real space by considering the real and imaginary components of the MT response tensor elements as separate data points.

Then, equation can be expressed as a linear system*A* is a full real‐valued matrix of size 8*N*_{f}×*N*_{t} that relates the real‐valued impulse response to the real‐valued MT response components as defined by equation .

In this paradigm, it may be appropriate to talk about the *model* being the vector unknown **z**_{t} (DTIR), and of the known **z**_{f}, the frequency domain MT response, as the *data*. Then, regularized least squares solution minimizes**z**_{f}, the MT response, while the *Q* matrix and the scalar damping parameter *λ* jointly determine the regularization term. The regularized least squares solution is then found by solving a discrete linear system,*Q* matrix defined in *Egbert* [] and found that the variant which imposes smoothing in the log‐frequency space is the regularization of choice that both improves the long‐period fits and minimizes artifacts in the DTIR (see Appendix B for details).

High‐quality, long‐term geoelectric field measurements are notoriously difficult to obtain. The three geomagnetic observatories in Japan, Kakioka (KAK), Kanoya (KNY), and Memambetsu (MMB) have pioneered long‐term high‐quality geoelectric field data collection: geoelectric field data recordings at these three observatories commenced, respectively, in the years 1932, 1948, and 1949 and are still ongoing. Long‐period MT responses have also been estimated for these three locations by *Fujii et al.* [] using the robust method of *Chave et al.* []. Because of the long time series that covers several major storms, the quality of the data, and the three‐dimensional (3‐D) complexity of Japan (in fact, all three sites are in complex coastal areas), this unique data set provides an appealing opportunity for validation of our method. For clarity of presentation, from now on we focus solely on the Memambetsu (MMB) observatory data. Similar results have been obtained at the other two locations.

To apply the DTIR technique to the time series, we first of all invert the MT responses using the linear least squares method of section 4, to obtain a well‐fitting DTIR. A truncated snapshot of DTIR is shown in Figure . We note that in this rare example, the length and quality of the time series has allowed for estimation of the MT response at the unprecedented long periods of up to 200,000 s. This gives us a better chance of reproducing the long‐period structure of the geoelectric field in our prediction. On the other hand, it comes with its own challenges: an impulse response with a much longer time window is necessary to fit this MT response, than would be typical for, e.g., EarthScope MT responses in the USA which are normally estimated for the periods from 10 to 20,000 s. In fact, the linear smoothing procedure, which is usually sufficient for EarthScope MT data, no longer produces adequate fits for these long‐period responses (see the Appendix B for details, and Appendix C for efficiency considerations). We empirically found that a value of the damping parameter *λ*= 5e–12 using the log linear smoothing and a time window that spans from negative *N*_{t−}=−600 to positive *N*_{t+}=21,600 samples of 1 s time series (i.e., from −10 min to 6 h) allow us to fit the MT response adequately. We used a diagonal data covariance matrix to account for the estimated errors in the MT response. For validation of DTIR, we recompute the MT response **z**_{f} at 300 logarithmically spaced frequencies using equation and compare the results to the measured MT response. The root‐mean‐square misfit of our DTIR fit to the MT response tensor is 0.53, 0.64, 0.77, and 0.48 for XX, XY, YX, and YY components, respectively. We plot the apparent resistivity and phase at MMB, and their DTIR fits (continuous lines), as well as the full MT response with DTIR fits, in Figures and , respectively.

For this analysis, we have chosen the series of “Halloween storms of 2003,” at the peak of solar activity of 29–31 October 2003, hereafter referred to as the Halloween storm. This storm is nicely illustrated with a clean geomagnetic and geoelectric field series recording at MMB observatory. At its peak, the Halloween storm of 29–31 October exhibited a series of two intense, *K*_{p}=9 durations of disturbance, one closely following the other. In Figure , we plot the two components of measured and detrended geomagnetic field **B** in nT for this time period.

We proceed to convolve our DTIR with the measured and detrended geomagnetic field time series, to obtain an estimated electric field time series that could be compared with the measured data. Similarly, we also apply the FFT technique of section 3 to get a second estimated electric field time series. In Figure , we plot the two components of **E** in mV/km for a period of 1 day at the onset of the second storm. The time series plotted in blue represents the observed MMB geoelectric field, except that the linear trend is once again taken out before plotting. The estimated geoelectric fields are also plotted in red (FFT) and green (DTIR). It is clear that the waveform of the measured geoelectric field is followed very closely by both estimates.

We now proceed to analyze and quantify the errors in our geoelectric field estimates. For this, we refer the reader to Figure . Figure (top left) and (top right) show the three geoelectric field time series plotted on top of one another using the same conventions as in Figure for both geoelectric field components *E*_{x} and *E*_{y}, except that now the full 3 days of the data are plotted. The DTIR estimated geoelectric field (shown in green) does not start until after the first 6 h of the time series, reflecting the length of the impulse response filter and the convolution process. In Figure (bottom left) and (bottom right), we plot the deviations in the estimate relative to the observations, on a different scale that would best allow a visual analysis. We can immediately see that the estimation errors never exceed 15% of the signal, and the error magnitudes are similar between the two methods. We further refer the reader to Figure to note that by band‐pass filtering all geoelectric field time series to the period range from 10 s to 12 h (twice the length of DTIR window, and likely the extent of validity of MT impedance), we further reduce the magnitudes of our misfits to generally no more than 5% of the (filtered) signal.

To better quantify the errors, let us define the correlation coefficient between two discrete time series *p* = (*p*_{1},*p*_{2},…,*p*_{N}) and *t* = (*t*_{1},*t*_{2},…,*t*_{N}), with *p* denotes the prediction and *t* denotes the target time series, we follow *Weigel* [] in defining the prediction efficiency (PE) as PE = 1 − ARV, where the average relative variance*t*. Perfect correlation and perfect fit would correspond to the values of 1 for both. The goodness of correlation and fit is quantified in Table .

Multiple factors contribute to discrepancies that might occur between estimated and measured geoelectric field data. Naturally occurring processes in the ground include electrochemical interactions between minerals and subsurface fluids, as well as the flow of ionic fluids, and thermoelectric mechanisms caused by temperature gradients in the subsurface. Such transient signals in measured geoelectric data can last minutes to hours, and due to their noninductive nature we cannot expect to reproduce them with a geoelectric field estimation algorithm. Other effects of non‐plane wave inductive nature are not typically reflected in the estimated MT response. Variations of loads in power lines and electric trains, for example, are sources of cultural noise in measured geoelectric field data. Diurnal signals in geoelectric field measurements are further caused by local temperature variations, while other longer‐term effects are caused by factors such as charge buildup in the electrodes and the thermal electric potential. Coastal geoelectric field measurements are also affected by motional induction due to ocean tides. Most of these effects would not be observed in the measured geomagnetic field time series that we use to obtain our estimates, nor do they matter from a GIC perspective, but they do contribute to the misfits observed during a validation exercise that involves real geoelectric field data.

There are additional constraints that are associated with using an MT response tensor as the “filter” to obtain the geoelectric field. Both geomagnetic and geoelectric time series are discrete and finite and therefore require filtering before the MT response is constructed. The MT response is therefore band limited, and accurate estimation of geoelectric fields outside of the constraints of the empirical MT response is not, in general, feasible. Moreover, at the lower and upper stretches of the MT response period range, the measured geoelectric time series are affected by the measurement system filtering configuration. At short periods, the discrete sampling of geoelectric field measurements requires the use of filtering. This is typically dealt with in MT instruments by employing built‐in low‐pass filters. For example, the 1 s time series of the geoelectric and geomagnetic fields at Memambetsu are influenced by measurement systems at periods shorter than several tens of seconds. On the other hand, the MT responses at Memambetsu at periods shorter than 30 s have already been corrected for system filter configuration. So this could result in very high frequency discrepancies between the observed and predicted 1 s geoelectric data.

By contrast, the longest periods, which are transmitted farthest, are also most likely to be affected by various large‐scale processes such as a buildup of charge at interfaces (e.g., coastlines) with large conductivity contrasts, long‐term temperature variations and water movement in the ground, just as the geoelectric field signal is getting weaker. A high‐pass filter is often used to filter out diurnal signals that are primarily thermal. Longer‐period geoelectric fields are also most affected by non‐plane wave ionospheric sources such as Sq variations. The MT response uses a uniform source approximation and should not in general be able to account for Sq variations.

In spite of an overall excellent fit and correlation shown in section 5, we do however notice coherent structure in the misfits in Figure that we are not able to reproduce with either of the two independently developed methods. This additional signal constitutes as much as 10–15% of the magnetic storm signal at its peak. We believe that such artifacts are an inherent limitation of our approach: specifically, certain physical processes contributing to measured, surface geoelectric fields cannot be recovered from the local geomagnetic field measurement obtained at a single point, in conjunction with an MT response. Alternative approaches might be developed that would account for the regional inhomogeneity of storm time ionospheric sources, and their inductive effects. Such effects would cause very large scale currents in the Earth's lithosphere, especially in the presence of conductivity contrasts. It is therefore not surprising that our misfits are dominated by longer periods. In fact, if we band‐pass filter both the measured and estimated geoelectric time series to the period range of 10 s to 12 h, which most certainly limits the validity of the MT impedance under the usual plane wave assumptions, the misfits are reduced to no more than 5% of the signal (Figure ). The coherence and nonstationarity of our misfits, as well as an overall excellent data fit, can also be vividly observed in the scatter plots of Figure .

In applying this technique to more traditional MT responses such as the EarthScope MT data set [*Schultz et al.*, ], one needs to be even more careful in comparison with measured data, as additional complexities come into play in trying to measure (and reproduce) ground geoelectric field data in an array‐style temporary installation. Collection of noise‐free geoelectric field data becomes very difficult in proximity to most human activity or a power line. In such areas, data spikes and cultural noise sometimes overwhelm geoelectric field measurements to an extent that the estimation of the geoelectric field using a real‐time geomagnetic field measurement and a clean MT response may give more accurate results than direct geoelectric field as measured. Electric fields in USArray data set are also highly susceptible to temperature‐related variations, which further contaminates (and eventually, dominates) the long‐period structure (at a permanent observatory installation such as Memambetsu, the electrodes are buried deeper and are not affected by temperature as much as the electrodes at the MT sites are). With certain specific site exceptions, reproduction of the long‐period signal in the measured geoelectric field is ill‐advised because it is predominantly a temperature‐driven effect and would not (in reality) affect GICs, which typically have the most power in the period range of 1 to 1000 s.

As is shown by *Egbert* [], even in a simple 1‐D situation for which its continuous counterpart has been shown to be causal, DTIR exhibits numerical noncausality. In practice this means that the modeled geoelectric field **E**_{t} at time *t*_{0} depends not only on the present and past states of the geomagnetic field **B**_{t}, but also on a few of its future states. In a general 3‐D situation, both the geomagnetic and geoelectric field are causally affected by the ionospheric source current, however strict causality of the relationship between the total geomagnetic and total geoelectric fields has been debated in the literature [e.g., *Egbert*, ; *Berdichevsky*, ; *Alekseev et al.*, ] and has never been shown to hold in general. The analysis in section 5 has been performed with what would equate to a 10 min delay in a real‐time geoelectric field estimation service, to take this effect into account. This can potentially create a problem for a data service that aims to provide the geoelectric field estimates as soon as the measured geomagnetic field is available. Noncausality of the algorithm will by necessity introduce an additional prediction delay into such an algorithm. From this point of view, it is useful to briefly explore the extent of noncausality that is involved. The results of this analysis are presented in this section. We first of all convert the algorithms of sections 3 and 4 to real time.

The FFT algorithm (section 3) is typically applied to a known segment of the time series. This is equivalent to constructing a periodic time series where the given segment is infinitely repeated. In section 3, we dealt with the edge effects at the ends of the segment by “padding” both ends with their end value for a certain length of time. This is, then, equivalent to assuming that the time series stays constant from the given time, “now,” for a stretch of time. In a real‐time application of this method to 1 s data, we take 10 point steps and use a sliding time window of 20,480 s. At each step, we assume that the last value that is known repeats itself into the future. We then perform the FFT geoelectric field estimation on this time segment and compare the last 10 points only with those that we would have obtained, had we known the future at the time of computation, as we assumed in section 5. Figure illustrates the estimation errors of this real‐time algorithm, as compared to the errors discussed in section 5. For a 1 h delay variant of this FFT real‐time estimation that is also shown in Figure , we assume that we know the future 1 hour of data, and only then do we pad. This procedure is illustrated in Figure . In this case, application of FFT in real time does not affect the accuracy of *E*_{x} estimates, but multiplies the errors in *E*_{y} by a factor of 2, resulting in a 25% maximum error during the more intense phases of the storm.

The DTIR approach (section 4) is natively suited for real‐time application. For this analysis, we merely constrain the DTIR to have a shorter negative lag than in section 4 (which was 10 mins or 600 samples), and proceed to solve the linear system under these constraints. Here we consider two variants of DTIR, computed with no negative lag (i.e., real time), and a 10 s negative lag (resulting in a corresponding prediction delay). In Figure , we compare these two variants of DTIR geoelectric field estimation with that employed in section 5. Estimation errors are not noticeably affected by a real‐time application of the method. Nevertheless, low‐amplitude, high‐frequency noise is introduced in the estimates when we apply DTIR algorithm in real time. We zoom in on this effect in Figure .

As is evident from Figure , forcing strict causality upon a DTIR is essentially equivalent to introducing high‐frequency noise in the geoelectric field estimate. The amplitude of the noise is of the order of a few percent of the total E field amplitude but will vary depending on the structure of the MT response, the sampling of the time series, and the type of regularization used to obtain DTIR. Our analysis shows that, in general, the use of the log linear smoothing in the DTIR estimation allows us to mitigate much of the unfortunate effects of the causal DTIR truncation on the shorter periods in the E field time series prediction (see Appendix B). We can also eliminate the bulk of the noise by allowing for a modest 10 s time lag in geoelectric field prediction. A 1 minute time lag provides no more than a little additional benefit in terms of accuracy.

Extreme magnetic storms can be hazardous for ground‐based infrastructure, potentially resulting in a significant economic disruption in the United States, as well as globally. The hazards associated with space weather call for a national service that would provide real‐time estimates as well as predictions of the relevant ground level geoelectric fields. Ground‐level geoelectric fields, if estimated with sufficient accuracy and in a timely manner, would help the power grid industry and other affected entities to take prompt action that would mitigate infrastructure damage.

This manuscript focuses on the development of a discrete time domain impulse response (DTIR) methodology for geoelectric field estimation using a geomagnetic field data stream, and a fully 3‐D MT response of the Earth. We apply this methodology to the Halloween storm measurements in Japan and validate our geoelectric field estimates using both the frequency domain approach and the measured data. We demonstrate in section 5 that both the newly developed time domain and the frequency domain methods are equally accurate for storm time geoelectric field estimation, with errors to within 10–15% of peak geoelectric field amplitude for our sample storm and geographic location.

An important word of warning is of note. We have seen that while we are able to recover as much as 85–90% of the geoelectric field signal throughout our case study, additional long‐period structure is present in the geoelectric fields during storm times that cannot be recovered (sections 5 and 6). We suggest that there can be inherent limitations to using traditional MT responses to compute storm time geoelectric fields but that these are not prohibitive for operational geoelectric field estimation, as long as the full 3‐D MT response is employed.

We have also briefly considered the issue of causality in a time domain geoelectric field prediction algorithm. We show that for the MT response we consider in this paper, a delay of as little as 10 s might be sufficient with very little loss in prediction accuracy. Enforcing strict causality, however, introduces a certain amount of high‐frequency noise in the predicted geoelectric field (a few percent of the signal, section 7). By contrast, applying the frequency domain algorithm in close to real time leads to additional estimation errors of up to 10% of the signal (Figure ). We conclude that while both approaches can be successfully applied in real time, a time domain approach such as the one developed in this manuscript, applied with a modest estimation delay results in the smallest errors in real‐time geoelectric fields.

In addition to using MT response functions estimated from empirical data, they can also be obtained from a model of regional electrical conductivity structure. Employing a regional 1‐D conductivity profile has not been shown to produce an accurate estimate of the local geoelectric field [e.g., *Weigel*, ] due to the highly variable Earth conductivity within regions [e.g., *Bedrosian and Love*, ]. It is nevertheless conceivable that 1‐D inversion of GIC itself [see, e.g., *Pulkkinen et al.*, ] and the predicted 1‐D response obtained from it at certain locations might provide a better regional representation of geoelectric field, more directly relevant to observed GICs, than any single local empirical MT response. This possibility arises from the fact that empirical MT responses are often significantly affected by very small scale, anomalously high or low conductivity features in the immediate vicinity of the site, a phenomenon known as distortion. Because GICs flow along grounded power lines over a significantly larger area, the quantity of interest is the integrated electric field along that path. Therefore, certain averaging is involved in going from local electric field estimates to GIC computations. We should therefore caution against using a local empirical MT response to estimate a regional geoelectric field. Instead, local geoelectric field values should be obtained at a number of points along the path of the grounded power line, ideally using a high‐resolution 3‐D electrical conductivity model and averaged to obtain the GIC correlate.

This analysis can be readily applied in the United States. While using the USArray MT responses [*Schultz et al.*, ] for geoelectric field computations would constitute a great improvement over the current practice [see, e.g., *Bedrosian and Love*, ; *Weigel*, ; *Bonner and Schultz*, ], a much better solution is to obtain the synthetic, modeled responses on a fine regular grid and use these for real‐time geoelectric field computations. This strategy potentially provides a means to eliminate the distortion from the MT response estimates, allowing for better predictive power. We outline this approach as a useful future direction for research. In fact, it is the intention of the USGS Geomagnetism Program to obtain such synthetic MT responses and to keep releasing annual updates to the community, as better electrical conductivity models become available. Having said that, any model of ground conductivity is only as good as the data that come into it. There is no substitute for completing the USArray MT survey.

Measured magnetic and electric time series are freely available from the International Real‐Time Magnetic Observatory Network (INTERMAGNET; *Fujii et al.*, ] will be made available at SPUD EMTF (

The location of Memanbetsu geomagnetic observatory makes it an exemplary site to illustrate the effects of three‐dimensionality of the Earth conductivity on the local geoelectric fields. An earlier study by *Pulkkinen et al.* [] obtained the best regional 1‐D model of Earth's electrical conductivity at the site. This model, in conjunction with certain power grid system‐specific parameters, does a good job of reconstructing the measured GIC in the system. However, as we can see in Figure , the best fitting regional 1‐D model provides us with an MT response that does not reconstruct the measured local electric fields at the site with precision comparable to what the 3‐D MT response provides (Figure ). While still correlated with the signal (Table ), the amplitudes of the misfits are of about the same magnitude as those of the measured signal. This suggests to us that while a regional 1‐D model may still be useful for regional GIC modeling, it is of limited utility for local geoelectric field estimation. Conversely, a single‐point geoelectric field estimate or measurement, in general, contains insufficient information for regional GIC computations. Instead, geoelectric field estimates at multiple locations are required. The 3‐D effects observed in these estimates may be smoothed out to some extent when the fields are integrated along a power line for GIC computations, but that would depend on the location and orientation of the power line relative to the geometry of Earth structures. Earlier 1‐D‐based GIC studies could be revisited with the updated information to explore this issue.

As we discussed in section 4, discrete time impulse response **z**_{t} can be obtained by solving the linear system . This is a system of *N*_{t} equations, with *N*_{t} unknowns (the real‐valued entries of vector **z**_{t}). The matrix *A* is 8*N*_{f}×*N*_{t} and defines, essentially, discrete Fourier transform as in equation (with entries *m*th frequency and the *k*th impulse response sample). The matrix Σ is 8*N*_{f}×8*N*_{f} and defines the covariance of the MT response components. In our implementation, Σ is diagonal and populated with variances of the four MT response components, real and imaginary parts, for each frequency. The full MT response covariance could also be used. The matrix *Q*, which determines the regularization, is *N*_{t}×*N*_{t}. We tested two variants proposed by *Egbert* []:

*α* = *k**l*/((*k* − *l*)^{2}), which achieves a DTIR that corresponds to a spline fit in log frequency (i.e., log linear smoothing). We further multiply this variant of *Q* by the factor 1e+9 to give the damping parameter *λ* similar meanings for both variants of *Q*.

We found the choice of *Q* to be key in the accurate computation of DTIR, especially as it pertains to the fit at extremely long periods that greatly exceed 10,000 s. At such long periods, linear smoothing is no longer an appropriate regularization for the DTIR least squares fit. A relaxed linear smoothing allows us to fit the long periods, but introduces artificial signal in the midperiod band. On the other hand, a stronger linear smoothing first decreases the fit at the longest periods and only then addresses the midperiod artifacts. Using the log linear regularization gives significantly better results, fitting the longest periods adequately with a smooth frequency domain curve as in Figures and , provided that the length of the impulse response window [*N*_{t−},*N*_{t+}] is sufficient to capture the bulk of the signal. As discussed in section 5, for a 1 s time series sampling, a 6 h window (*N*_{t+}=21,600) was empirically found to be sufficient to produce a good fit for periods up to 200,000 s.

As is discussed in Appendix B, fitting the MT response at longer periods (beyond 10,000 s) requires a rather long time series (*N*_{t}>20,000) to represent the DTIR adequately for 1 s data sampling. Numerical solution of the linear system , also known as the *model space formulation*, which has *N*_{t} unknowns, becomes computationally expensive, as well as ill‐conditioned. For example, a linear inversion for a DTIR of length ≈10,000 s requires approximately 5 min run time in Matlab on a modern laptop or desktop; anything much bigger than that challenges the computer memory limits. (By contrast, we find that DTIR time window from negative 10 min to positive 2 h (or [−600 7200]) is sufficient to fit all USArray MT responses, which span the period range from 7 s to 20,000 s.)

The alternative *data space formulation*,*N*_{f} unknowns (typically ≫ *N*_{t})and requires 3 s run time in Matlab for an equivalent problem. In fact, this alternative approach is so efficient that optimality can easily be achieved by solving the linear system repeatedly for a range of values of the damping parameter *λ* and then choosing a solution based on an *L* curve or a discrepancy principle (i.e., opting for the smoothest solution that still fits the data to within the chosen misfit threshold).

We note, however, that this alternative formulation requires an inverse of the *Q* matrix discussed in Appendix B. This inverse is trivial for the diagonal *Q* of equation but is less than trivial for the full matrix of equation , which we have chosen to use for this manuscript. This precludes us from using the more efficient data space formulation with an *L* curve for the Japanese observatory DTIR obtained in section 5. Instead, we empirically found a value of damping parameter *λ* that allows us to fit the MT response well and not introduce any midperiod artifacts, using the log linear smoothing in model space as described by equations and . A relatively straightforward extension of the technique to compute *Q*^{−1} for would allow for a data space formulation with a log linear smoothing that would increase computational efficiency. We note that *Q*^{−1} needs only to be computed once and may then be reused.

We thank C.A. Finn, P. Bedrosian, J. McCarthy, and J.L. Slate for reviewing a draft manuscript. This work was supported by the USGS Geomagnetism Program. The results presented in this paper rely on the data collected at Memambetsu Geomagnetic Observatory. We thank the Kakioka Magnetic Observatory for supporting its operation and INTERMAGNET for promoting high standards of magnetic observatory practice. Any use of trade, firm, or product names is for descriptive purposes only and does not imply endorsement by the U.S. Government.