Judith Gu on diagnosing model failure in high-frequency trading

You’ve cleaned the data, built the model, tuned the parameters. The signal is still flat. What next? In this episode of Data in the AI Era , Judith Gu, Head of Equities and FX Quant Trading Strategies at Scotiabank, joins Peter Finter to unpack a real-world modeling breakdown in high-frequency trading and explain what really matters when your models don’t work.

Episode overview

If you’ve ever built a pipeline that looked solid on paper but failed in production, this episode will hit home. Judith walks through her experience building predictive models for high-frequency FX returns. She used a rigorous modeling progression, including linear models, machine learning, and deep learning, supported by full tuning and feature analysis. None of the models performed better than the naïve benchmark.

The root problem wasn’t in model selection or parameter settings. The data itself had no predictive structure. Judith tested for autocorrelation and found the returns behaved like white noise. That finding reframed her entire approach. Instead of searching for the right model, she treated the data as a noisy signal and applied wavelet denoising to extract structure.

Volatility, in contrast, showed more promise. It held structure and responded well to simpler models. Judith also highlights issues with time-based features in 24/5 markets. Encoding hour-of-day as a raw number can mislead models that don’t understand market context. Her solution involved circular encoding and session-aware features.

This is a case study in data diagnostics, signal testing, and model restraint. Judith’s experience shows what to do when complexity isn’t the answer and how to identify when the data itself needs closer inspection.

Key takeaways

1. Clean data doesn’t guarantee useful results

Judith discovered that her high-frequency return data had no autocorrelation and no learnable structure. It passed all conventional checks for cleanliness but offered no predictive value. This exposed a common trap in quant modeling. Just because data is well-formatted and complete doesn’t mean it contains signal. The only way to be sure is to test the statistical properties directly. If the data behaves like white noise, no model will make it useful.

2. When models stall, check your data, not your algorithm

Judith applied multiple model types, from simple regressions to advanced neural networks. None of them outperformed the baseline. The issue wasn’t the model, it was the input. All signs pointed to a lack of structure in the return series. This is a clear signal to stop tuning and start diagnosing. If every approach gives you the same weak output, you’re likely working with unlearnable data.

3. Signal processing can help when standard tools fail

To address the lack of structure, Judith turned to wavelet denoising. This method separates a signal into different frequency components, helping to filter out noise and surface patterns. By applying this approach, she was able to transform the return series into something the model could learn from. In high-frequency environments where noise overwhelms signal, tools like wavelets can reveal structure that time-domain models miss.

4. Volatility holds more structure than returns

Judith found that volatility data was far more predictable than returns. Even simple models produced usable outputs. This difference reflects the nature of fast markets, where price direction is often reactive and noisy, but volatility clusters in observable patterns. If directional modeling fails, shifting focus to volatility can still yield valuable insights for execution, risk, or strategy.

5. Time features need better treatment in continuous markets

Time-based variables like hour-of-day can mislead models if encoded as plain numbers. In global markets with continuous trading, 23 and 0 might be adjacent in real time, but they are distant on a numerical scale. Judith addressed this by using circular encoding and defining trading sessions explicitly. If you’re modeling on streaming data, how you represent time can shape the model’s entire view of market behavior.

Technical highlights

Knowing when to stop modeling and recheck your assumptions

You’ve tried multiple models. The results don’t improve. Now what? Judith’s experience shows the importance of recognizing when model tuning is a dead end. If your models plateau early and consistently, it’s often a sign to pause and reassess your inputs. Knowing when to stop is as important as knowing where to start.

Benchmarking against a true naïve model in HFT

Judith used a zero-drift, last-price benchmark to measure prediction accuracy. In high-frequency environments, this is a meaningful test. If your model can’t outperform this baseline, the added complexity likely isn’t justified. A rigorous benchmark prevents false confidence and provides a reality check before going live.

Reframing the problem with signal processing tools

Judith didn’t just change techniques, she changed her lens. Instead of thinking about price prediction as a standard time-series task, she approached it as a signal processing problem. That shift unlocked access to tools like wavelet decomposition, which helped reveal structure that statistical models alone could not find.

Why models that work in equities often fail in FX

Judith transitioned from equities to FX and quickly found that techniques she once relied on didn’t apply. FX trades 24 hours a day, responds to macroeconomic events, and behaves differently across time zones. Features that worked in fragmented equity markets broke down in FX. Every asset class has its own modeling logic. What works in one won’t automatically work in another.

Keep an eye out for an upcoming blog where we will explore Judith’s discovery process with more depth.