Use temporal redundancy
Adjacent frames contain repeated observations of edges, textures, and sub-pixel motion. Mini-DLSS aggregates that context in both temporal directions and predicts the high-resolution center frame at a fixed 2x scale.
Temporal video super-resolution
Mini-DLSS
A lightweight BasicVSR-style system that reconstructs a high-resolution center frame from five low-resolution frames, then exports the trained model for reproducible CPU deployment.
Mini-DLSS is inspired by temporal reconstruction systems. It is not NVIDIA DLSS and does not use renderer motion vectors, depth buffers, or proprietary reconstruction logic.
- PSNR-Y
- 38.33 dB
- over bicubic
- +1.52 dB
- tPSNR
- 36.60 dB
- ONNX CPU / frame
- 21.59 ms
- parameters
- 0.716M
Measured on a local Vimeo-derived REDS-style validation and demo set, not the official REDS benchmark.
01 / Evidence
The result
The trained temporal model improves PSNR-Y over bicubic while preserving stable frame-to-frame reconstruction on the fixed validation clip.
02 / Objective
The reconstruction problem
Geometric resizing can enlarge a frame, but it cannot recover missing detail. Framewise neural super-resolution can sharpen individual images while producing flicker because each frame is reconstructed without neighboring evidence.
Build the complete system
The work covers dataset manifests, aligned temporal sampling, model training, image and temporal metrics, comparison media, ONNX export, and measured MP4 inference rather than treating the neural network as an isolated artifact.
03 / Model
BasicVSRMini
A deliberately compact architecture uses shared convolutional features, bidirectional ConvGRU propagation, center-frame fusion, residual reconstruction, and PixelShuffle upsampling.
01
Five-frame LR window
An odd temporal window keeps one center frame as the supervised reconstruction target.
02
Shared feature extraction
The same convolutional encoder transforms each low-resolution frame into spatial features.
03
Forward ConvGRU
A recurrent state propagates evidence from earlier frames toward the center.
04
Backward ConvGRU
A second recurrent state moves backward so the center also receives future context.
05
Center feature fusion
The center feature and both recurrent states are concatenated and refined by residual blocks.
06
PixelShuffle output
A sub-pixel reconstruction head emits one RGB center frame at a fixed 2x scale.
Unlike full BasicVSR and EDVR-style systems, this model does not estimate optical flow or use deformable alignment. Motion handling is learned implicitly through convolutional recurrent state.
04 / Pipeline
From dataset to deployment
Config-driven entry points keep training, evaluation, artifacts, and deployment reproducible across local and Colab workflows. The best validation checkpoint occurred at 80,000 steps during the 150,000-step training run.
01
Configs and manifests
TOML run definitions and sequence manifests lock scale, temporal length, data roots, and training settings.
02
Temporal dataset windows
Vimeo-90K sequences become aligned five-frame inputs with a high-resolution center target.
03
PyTorch training
Config-driven runs use resumable checkpoints, fixed validation intervals, and logged experiments.
04
Quality evaluation
PSNR and SSIM are measured on cropped Y-channel outputs alongside temporal stability diagnostics.
05
Evidence artifacts
The evaluation CLI writes tables, JSON, four-panel stills, and labeled comparison video.
06
ONNX and MP4 inference
The best checkpoint exports to ONNX and runs on sliding video windows for reproducible deployment tests.
Best checkpoint
80,000 of 150,000 stepsEvaluation domain
Y channel, two-pixel border cropExport contract
Fixed five-frame input, dynamic spatial axes05 / Evaluation
Measured results
Image fidelity and temporal behavior are reported separately. The temporal model improves tPSNR by +0.2578 dB over bicubic. The single-frame comparison is a 300-step pipeline baseline, not a fair or budget-matched temporal ablation.
| Method | PSNR-Y | SSIM-Y | tPSNR | Diff. energy |
|---|---|---|---|---|
| Bicubic | 36.8033 | 0.9601 | 36.3441 | 0.0217 |
| Single-frame SR fast cyclePipeline sanity baseline; not budget-matched. | 32.3755 | 0.8807 | 33.8817 | 0.0183 |
| Temporal SR 5f small | 38.3265 | 0.9604 | 36.6018 | 0.0201 |
Target-relative temporal error energy is 0.0105 for the temporal model, 0.0108 for bicubic, and 0.0145 for the single-frame baseline. This diagnostic compares predicted motion with target motion rather than measuring raw output smoothness alone.
PyTorch CPU demo
34.941 ms/frameONNX Runtime CPU
21.589 ms/frameONNX Runtime CPU latency is 38.2% lower than the PyTorch CPU demo on the same evaluation clip: 240 frames at 64x64 LR. These local-machine deployment measurements are device-specific, not device-independent real-time claims.
06 / Frames
Qualitative evidence
Four-panel stills make local edge and texture differences inspectable without relying on a single aggregate metric.
07 / Judgment
Limits and next steps
The current system is a reproducible research scaffold, not a state-of-the-art benchmark submission or production renderer integration.
Current limitations
- No explicit optical-flow or deformable alignment.
- Overlapping inference windows repeat feature extraction and recurrent work.
- Local validation rather than official REDS evaluation.
- The single-frame result is a fast-cycle baseline, not a budget-matched ablation.
- Global frame-difference energy can penalize legitimate scene motion.
- Each window predicts one high-resolution center frame.
Next experiments
- Evaluate the checkpoint on the official REDS validation set.
- Train a budget-matched single-frame baseline.
- Compare three-, five-, and seven-frame temporal windows.
- Test flow-assisted or deformable alignment.
- Cache recurrent state across sequential inference windows.
- Measure quantized and mixed-precision deployment paths.
08 / Full breakdown
Read the technical paper
The concise Technical Summary and Evaluation Report covers scope and claim boundaries, model design, the data and evaluation protocol, final evidence, deployment, limitations, and next steps.