Chirath Soithong GitHub Résumé PDF

Research reportRutgers Quant Finance ClubIndependent, AI-assistedFeb–May 2026

Does a bigger model predict prices better? I tested seven to find out.

BeforeThe brief: rebuild one published model and its results.AfterSeven models on the same data, and every simulated trade charged its cost.

ResultSubmitted to the Rutgers Quant Finance ClubIn my tests on one public dataset, a model 3.8 times smaller scored about the same.

Jump to the resultSee the benchTechnical deep dive

Old Queens, the brownstone main building of Rutgers University with its white cupola, seen across a lawn
Old Queens, Rutgers University, New Brunswick.

The 60-second version

What I did
Rebuilt a published model that predicts short-term price moves from the order book, then tested it against six others, from simple statistics to variations on its design: 95 experiments, written up in a research report.
  1. Same public data
  2. Seven models, 95 runs
  3. Weigh accuracy, size and cost
  4. Write the report
The runs are automatic; weighing the results was my part.
Who did what
The club set the brief. I took it further on my own, with AI assistants helping me explore the research, write the code and draft the report.
Tools
Python and PyTorch. I asked for access to the Rutgers Amarel GPU cluster; each run was a settings file, queued there and saved with its results.

Part 1 of 4: The question

The brief was to rebuild one published model. I asked whether all of it was needed.

The model, DeepLOB, reads the order book, the list of open offers to buy and sell a stock, and predicts whether the price goes up, down or stays flat.

What I didKept the published model as the yardstick and tested six others beside it.

Exhibit 1The published model, part by part

Point at a number

Illustrative picture of the published model. On the left, the order book goes in. Three layers work on it in turn: one spots patterns, one looks at several time scales at once, one remembers what came before. The last two carry the question: needed? On the right comes the prediction: up, flat or down.

Sell Buy Order booklast 100 updates

The published model

Spots patternsCNN
Several time scalesInceptionNeeded?
MemoryLSTMNeeded?
Up Flat Down Prediction
  1. The order book. Offers to buy and sell, ten price levels on each side, over the last 100 updates.
  2. Spots patterns. Finds shapes in the book.
  3. Several time scales. Looks at short and longer stretches at once.
  4. Memory. Carries what came before into the next step.
  5. The prediction. Up, down or flat, 10 to 100 updates ahead.
Illustrative The published design, drawn simply; the technical names are in small type.
The data, in plain wordsKey · under 1 min
The stocks
Five Finnish stocks over ten trading days, from a public benchmark (FI-2010).
One example
The last 100 order-book updates, ten price levels on each side.
The question
Up, down or flat, 10, 20, 30, 50 or 100 updates ahead.
Learn, then test
Models learn on the first seven days and are scored on the last three.

Part 2 of 4: What I built

I built a test bench: every model gets the same data and the same question.

95 experimentsSeven models, five horizons, four ways to train for trading.The published model, rebuilt, is one row. The other six range from simple statistics to variations on its design.

Exhibit 2Every experiment on the bench

Scroll to run all 95

Each square is one experiment. Left: 35 comparisons, seven models times five horizons (10, 20, 30, 50 and 100 updates ahead): three classical baselines, the published model rebuilt, and three variants of it. Right: 60 runs trained for trading, four training goals, three settings each, five horizons. 95 experiments in all.

35 comparisons7 models × 5 horizons

60 trained for trading4 goals × 3 settings × 5 horizons

Classical baselinesThe published model, rebuiltDeep modelsTrained for tradingEach square is one experiment.
How one experiment runs, step by step1 exhibit · under 1 min

Exhibit 2.1One experiment, from settings file to saved result

  1. A settings fileWhich model, which horizon, how to train.
  2. Queued on the clusterA job on the Rutgers GPU cluster.
  3. Trained and checkedThe best version is kept, judged on held-back data.
  4. Scored and savedScored on the test days, saved with its history.
The same four steps for all 95 experiments.

Part 3 of 4: Trading costs

Then I charged every simulated trade its cost.

A second scorecard turns each model’s predictions into simple trades, and each change of position pays half the bid–ask spread. On that scorecard, a higher accuracy score did not reliably mean better simulated trading.

What I didTested ways to trade less: holding longer, acting only when several horizons agree, sitting out volatile moments.

Exhibit 3Every change of position pays a cost

Pick a way to trade

Illustrative example of sixteen signals from a model. Acting on every signal changes position 13 times, so it pays 13 charges of half the spread. Holding each position longer changes position 3 times: 3 charges.

  1. Act on every signal

    13 changes of position, so 13 charges

  2. Hold each position longer

    3 changes of position, so 3 charges

Illustrative Sixteen example signals; the counts follow from them.
What the bid–ask spread is, in one picture1 exhibit · under 1 min

Why each change costs

The price is tracked in the middle, between the best offer to buy (the bid) and the best offer to sell (the ask). A real trade happens at the bid or the ask, half a spread away from the middle.

Exhibit 3.1Bid, ask and the middle

Illustrative picture of a price line. The bid, the best offer to buy, sits on the left; the ask, the best offer to sell, on the right; the middle price halfway between. The gap from bid to ask is the spread. Half of it, from the middle to either side, is the charge for each change of position.

Illustrative Not to scale.

Part 4 of 4: What I found

Submitted to the Rutgers Quant Finance Club

A model 3.8 times smaller scored about the same.

On one public dataset, averaged over the five horizons. My takeaway: a more complicated model is not necessarily a better one.

SkillsResearch design · Model comparison · Backtesting · Report writing

Exhibit 4Size and accuracy, side by side

Scroll to compare

Size, in learned parameters: the published model, rebuilt, has 143,907; the patterns-only model has 37,891, about 3.8 times smaller. Average accuracy over the five horizons: 77.5% for the published model, rebuilt, and 78.1% for the patterns-only model.

Size counted from the code. Accuracy on the three test days; bars run from 0 to 100%.
All seven models’ accuracy, side by sideChart · under 1 min

Exhibit 4.1Average accuracy, all seven models

  1. Patterns + attentionCNN + Transformer79.0%
  2. Patterns onlyCNN78.1%
  3. Published, with attention for memoryCNN + Inception + Transformer77.8%
  4. The published model, rebuiltDeepLOB77.5%
  5. Decision treesXGBoost72.7%
  6. Small neural netMLP58.7%
  7. Straight-line statisticsLogistic regression58.3%
Average test accuracy over the five horizons, one public dataset; bars run from 0 to 100%.

Built with

  • Python, research code
  • PyTorch, the deep models
  • Jupyter, analysis and backtests
  • Rutgers Amarel, GPU cluster