Volver
Multimodal AIMixture of ExpertsMoEQwen3.5Qwen3Financial RAG

Multimodal AI vs Mixture of Experts: What Building a Financial RAG Pipeline Taught Me About Cost, Latency, and Accuracy

Harsh Sinha⭐⭐⭐Sitio del autorOct 30, 2026Visitar Qwen
93% Útil7% No útil
0 Comentarios

Mixture-of-Experts (MoE) is particularly interesting because sparse expert activation can reduce inference compute. But multimodal and MoE are not alternatives. One describes the types of data a model understands; the other describes how computation is organized.

In this review, I separate my firsthand production experience from a current, externally sourced price and latency comparison of three Qwen architectures.

1. My Real-World Use Case: Financial Documents to Structured Data

I previously worked on a financial-research assistant backed by more than 100,000 documents and approximately 20 million structured records. The wider product served over 500,000 user queries.

One part of my work involved transforming unstructured financial documents into machine-readable information.

The inputs included:

  • IPO Red Herring Prospectuses (RHPs)
  • ETF factsheets containing portfolio holdings and valuation ratios
  • Corporate earnings documents and investor presentations
  • Tables, charts, and scanned PDF content

For document extraction, I worked with GPT-4o, Gemini, document-understanding tools, OCR, and schema-based validation.

The objective was not to generate a convincing summary. It was to recover specific financial data accurately enough for downstream systems to consume.

For example, an ETF factsheet might contain its P/E ratio, P/B ratio, expense ratio, holdings and reporting date across multiple visual sections.

A model that identifies the right numbers but associates them with the wrong reporting period is not accurate enough.

What I actually tested

My practical workflow involved extracting document content, mapping it into structured fields, validating the result against a defined schema, and making it available to retrieval and application layers.

A representative, simplified version of the extraction instruction looks like this:

Analyze the supplied ETF factsheet.

Extract:
- Fund name
- Reporting date
- P/E ratio
- P/B ratio
- Expense ratio
- Top holdings

Return structured JSON.

Use null for missing values.
Do not infer numbers absent from the source.
Preserve the reporting date and distinguish
reported values from derived values.

This is a reconstructed example of the task, not a verbatim archived production prompt.

The actual document pipelines used structured extraction and validation rather than trusting unconstrained model responses.

What was the outcome?

The financial-document extraction workflow reduced manual analysis from hours to minutes.

I also worked on prompt design, batching, caching, and architectural optimizations that reduced overall LLM token costs by approximately 90% in the broader system.

That 90% reduction is a system-level engineering result. It was not caused solely by switching to an MoE model, and I would not attribute it to MoE without an isolated experiment.

My main lesson was that model selection is only one component of AI cost engineering.

2. The Architectural Difference Most Comparisons Miss

A multimodal model processes more than one input modality, such as text, images, audio, or video, depending on its capabilities.

A sparse Mixture-of-Experts model uses a learned router to activate selected expert subnetworks for a token rather than evaluating every expert.

Consider three models:

ModelArchitectureWhat it can process
Qwen3-VL-8B-InstructDense multimodalText, images and supported visual inputs
Qwen3-30B-A3B-Instruct-2507Text-focused MoEText
Qwen3.5-35B-A3BMultimodal MoEText, images, video

Qwen3-30B-A3B has approximately 30.5 billion total parameters, but only around 3.3 billion are active per token.

Qwen3.5-35B-A3B combines a native vision-language architecture with sparse expert routing. Its language model has approximately 35 billion total parameters and 3 billion active parameters.

This is the important distinction: multimodal processing and sparse expert routing can coexist in the same model.

MoE also does not mean that the model automatically knows which expert is best for finance, coding, or biology. Routing specialization is learned during training, and its behavior cannot be inferred from parameter counts alone.

3. Cost and Latency: What the Public Provider Data Shows

I checked OpenRouter's listed prices and provider performance on October 28, 2026.

These are third-party provider statistics, not latency measurements from my own API requests.

Model / providerInput / 1MOutput / 1MListed p50 latencyThroughput
Qwen3-VL-8B / Parasail$0.25$0.750.71s45 tok/s
Qwen3-30B-A3B / DekaLLM$0.09$0.300.41s76 tok/s
Qwen3.5-35B-A3B / DeepInfra$0.14$1.000.52s63 tok/s
Qwen3.5-35B-A3B / Darkbloom$0.065$0.751.90s62 tok/s

The listed latency values come from OpenRouter's provider telemetry. They should not be treated as a controlled time-to-first-token (TTFT) benchmark using identical prompts, networking conditions and output lengths.

I did not preserve client-side TTFT traces for my earlier financial-document extraction work.

Two observations stand out.

First, the text-only MoE model has attractive token economics, but it cannot directly replace an image-reading model. It needs text extracted beforehand.

Second, even two providers serving the same multimodal MoE checkpoint can have meaningfully different listed latency and pricing.

This makes provider selection a deployment decision, not merely an infrastructure detail.

4. What Would 10,000 Financial Extraction Requests Cost?

Consider an illustrative workload of 2,000 billable input tokens and 300 output tokens per request.

Using the selected provider rates above:

Processing routeEstimated cost for 10,000 requests
Dense multimodal model$7.25
Text-only MoE$2.70 + OCR
Multimodal MoE, DeepInfra$5.80
Multimodal MoE, Darkbloom$3.55

These are token-only calculations, not real invoices from my experiments.

The example assumes equivalent billed token counts across models. Actual image tokenization varies by model and provider, and a text-only route also requires document preprocessing or OCR.

This is why the text-only MoE's $2.70 figure is not a complete financial-document processing cost.

A useful production calculation is:

Total cost per successful extraction = model inference + OCR/preprocessing + retries + validation + infrastructure.

For a pipeline processing thousands of documents, even a cheap model becomes expensive if poor extraction quality creates additional retries or manual corrections.

5. Answer Quality: The Metric I Care About Most

Generic knowledge benchmarks are useful references, but they do not directly measure whether a model extracts financial data correctly.

I would evaluate these architectures on five dimensions:

  1. Field accuracy: Are financial values copied correctly?
  2. Source fidelity: Does every extracted value correspond to the supplied document?
  3. Period accuracy: Does the model distinguish standalone from consolidated values, or quarterly from annual figures?
  4. Schema validity: Does the response follow the required JSON structure?
  5. Hallucination rate: Does the model invent missing values instead of returning null?

In my earlier extraction work, structured output and downstream validation were essential parts of the workflow. Simply receiving plausible-looking text was insufficient.

For a controlled comparison, I would give the two vision-capable Qwen models the same document image and give the text-only MoE model OCR output from that exact image.

I would then compare extracted fields against manually verified ground truth.

That comparison has not yet been run by me, so I cannot honestly report a Qwen-versus-Qwen field-accuracy winner.

There is also an important fairness issue: OCR errors become part of the input to the text-only model. A bad extraction could make a capable MoE model appear inaccurate.

For that reason, I would report both model-only latency and end-to-end pipeline latency.

6. The Hidden Cost of MoE

The main advantage of sparse MoE is that only a fraction of the available experts participate in computation for each token.

However, fewer active parameters does not mean proportionally lower total memory requirements.

The inactive expert weights still need to be stored or made accessible. Expert routing also introduces communication and serving considerations, especially when experts are distributed across accelerators.

Therefore, hosted API pricing is not determined by active parameter count alone.

For self-hosting, I would examine GPU memory requirements, batching efficiency, expert placement, utilization and tail latency before assuming an MoE checkpoint is cheaper.

7. Where RaDeR Fits Into the Picture

I also looked at RaDeR, the reasoning-aware dense retrieval research published at EMNLP 2025.

RaDeR is not a competing multimodal or MoE generation model. It addresses retrieval for reasoning-intensive queries and reports improvements on benchmarks such as BRIGHT.

That makes it potentially relevant to the retrieval layer of a financial-research pipeline.

A stronger retriever can improve which evidence reaches the answering model, but it does not replace visual extraction or generation.

In a complete RAG system, retrieval quality, document understanding and answer generation are separate optimization opportunities.

8. My Verdict: Which Is Worth Using?

For raw PDFs, screenshots and visually complex financial documents: I favor a multimodal extraction route. It can inspect the original visual source rather than depending entirely on OCR.

For clean, normalized text and structured context: A competitively priced text-only MoE deserves evaluation. Paying for visual capabilities on every request may be unnecessary.

For a unified model that handles both modalities and sparse computation: Qwen3.5-35B-A3B is a particularly interesting candidate. Its architecture is promising, but I would require a controlled accuracy and end-to-end latency test before deploying it in place of a proven extraction pipeline.

My real production takeaway is that separating extraction, validation, retrieval and reasoning can be more valuable than forcing everything through one large model.

The strongest evidence from my own work is that document understanding and architectural optimization substantially reduced manual processing effort, and that prompt, batching and caching improvements reduced token costs across the system.

Is MoE worth it? Potentially yes, especially for well-structured, high-volume text workloads.

Is multimodal worth it? Yes, when visual information is necessary to answer correctly.

Is multimodal MoE the best of both worlds? Architecturally, it is a compelling direction. Operationally, the answer depends on measured accuracy, actual billing, TTFT, retry rates and end-to-end latency.

I would rather deploy the model that correctly extracts a financial value and cites its source than save a fraction of a cent on a plausible but incorrect answer.

Comentarios0

Inicia sesión para comentar
  • Aún no hay comentarios.

Artículos relacionados