Hi everyone,
I’m looking for some advice on expanding my local AI video-generation capabilities without replacing my RTX 3090 or paying inflated prices for an RTX 5090.
I’m developing a model-agnostic AI video application using ComfyUI and PyTorch. My current focus is MiniMax H3, but I want the architecture to support different AI video models and workflows as they evolve.
The HP ZGX Nano with 128GB of unified memory caught my attention. I’m wondering whether it could complement my existing workstation by taking over memory-intensive AI operations, allowing my RTX 3090 to concentrate on video generation.
However, my workstation already has 128GB of DDR4 system RAM, which raises another interesting question: could I accomplish much of what I’m considering through intelligent CPU/RAM offloading without purchasing a separate machine?
My current hardware and software
-
GPU: NVIDIA RTX 3090, 24GB VRAM
-
System RAM: 128GB DDR4
-
Operating system: Windows
-
PyTorch: 2.10.0 with CUDA 13.0
-
Backend: ComfyUI running headlessly
-
Primary model: MiniMax H3, currently using its pruned INT8 Ref2VA diffusion checkpoint
-
Application: Custom model-agnostic AI video application under development
I’m interested in supporting a wide range of workflows, including text-to-video, image-to-video, video-to-video, character replacement, reference-conditioned generation, motion and pose control, video inpainting, and video enhancement/upscaling.
My objective is higher-quality generation, larger models, longer sequences, higher resolutions, and better conditioning—not simply faster rendering.
What I’m trying to accomplish
Ideally, I want to dedicate as much of my RTX 3090’s 24GB VRAM as possible to the primary generative model and its sampling operations.
Many AI video pipelines include substantial supporting components:
-
Text and vision encoders
-
Reference-image processing and conditioning
-
Video/audio VAEs
-
Segmentation and masking models
-
Pose, depth, and motion estimation
-
Optical flow and tracking
-
Video decoding, enhancement, and upscaling
Rather than running everything on the RTX 3090, I’m exploring whether these components could execute elsewhere, freeing GPU resources for the most demanding generation tasks.
I understand that ComfyUI already unloads and offloads models as needed, and that moving a stage to another device won’t automatically reduce peak VRAM during sampling if that stage was already unloaded.
I’m trying to understand where a more deliberate architecture could provide meaningful benefits.
Option 1: Make better use of my existing 128GB DDR4 RAM
Before buying additional hardware, could I use my existing system memory more effectively?
For example:
-
Keep larger model weights in system RAM and transfer required portions to GPU memory.
-
Use CPU offloading or block swapping for transformer layers.
-
Keep supporting models on the CPU when practical.
-
Optimize memory allocation, model loading, and unloading.
-
Potentially use pinned memory or asynchronous transfers to reduce offloading overhead.
I’m aware that DDR4 bandwidth is much lower than dedicated GPU VRAM bandwidth, and that PCIe transfers introduce overhead.
But if I’m willing to trade some speed for the ability to generate higher-quality video or use larger models, could this be a reasonable approach?
Are there particular PyTorch, ComfyUI, or CUDA memory-management techniques that are especially useful here?
Option 2: Add an HP ZGX Nano as a secondary AI processor
The HP ZGX Nano offers 128GB of unified memory through NVIDIA’s GB10 platform.
Instead of using it as a replacement renderer, I would use it as a complementary AI computing system.
Conceptually:
HP ZGX Nano:
-
Text/vision encoding
-
Reference conditioning
-
VAE encoding and decoding
-
Segmentation, pose estimation, depth processing
-
Video preprocessing and enhancement
-
Potentially portions of larger generative models
RTX 3090 workstation:
-
Main diffusion transformer or other generative backbone
-
Video sampling and generation
-
GPU-intensive operations that benefit from dedicated VRAM and high memory bandwidth
My application would coordinate the two machines over a network.
For example, one machine could prepare reference conditioning and video latents, transfer them to the 3090 for generation, and then receive the output latents for decoding and postprocessing.
I understand this isn’t equivalent to combining 128GB and 24GB into one shared VRAM pool.
What I want to know is whether the separation could meaningfully reduce the workload and memory demands on the 3090.
Option 3: Distribute portions of the generative model itself
This is perhaps the most interesting, but also the most experimental, possibility.
What if part of a large diffusion transformer or other generative backbone could run on a secondary machine?
For example, could some transformer layers or model components execute on the GB10 while the RTX 3090 handles others?
I’m aware that inter-machine communication, incompatible architectures, tensor transfers, and the GB10’s GPUDirect RDMA limitations could make this difficult.
But I’m interested in whether heterogeneous distributed inference has advanced enough to make something like this practical for AI video models.
Could this let me use higher-precision models or larger generative architectures than my 24GB GPU can otherwise support?
MiniMax H3 as a practical example
MiniMax H3 is my current testing ground.
It contains substantial components beyond its main video diffusion transformer, including a large Qwen3-VL text/vision encoder and video/audio VAEs.
I’d like to explore whether those components can be separated from the diffusion process and run on different hardware.
This could potentially make it possible to use higher-precision conditioning or reduce model-swapping overhead.
However, I’m not looking for an H3-specific solution. My application is model-agnostic, and I want the same underlying architecture to support other video-generation models and future developments.
Questions for the community
1. With 128GB of DDR4 RAM already available, how far can I push CPU/RAM offloading before investing in another machine?
Could optimized model offloading, memory pinning, or transformer block swapping provide most of the benefit I’m seeking?
2. Would a GB10-based machine provide a meaningful advantage over my existing system RAM?
Would the Nano’s GPU-accessible unified memory make it substantially more useful for auxiliary AI processing than ordinary DDR4-based CPU offloading?
3. Is separating encoders, VAEs, and diffusion sampling across two machines practical?
Are there existing frameworks for distributing these pipeline stages across heterogeneous Windows/x86 and Linux/ARM NVIDIA systems?
4. Is splitting the actual generative model across machines realistic?
Could I offload some diffusion transformer blocks or weights to a second machine without network latency making the approach impractical?
5. How much does network speed matter?
Would 10GbE be sufficient for stage-to-stage tensor and latent transfers? Would 200GbE provide a worthwhile advantage?
6. Would another hardware approach make more sense?
Perhaps a second conventional NVIDIA GPU, a used 48GB workstation GPU, a dedicated Linux GPU server, or a different combination of hardware and memory management would be more effective.
7. What should I benchmark first?
I’d like to understand how to measure actual peak VRAM use, GPU memory residency, CPU offloading overhead, PCIe transfers, and performance at different stages of video generation.
I don’t want to buy hardware based on theoretical memory capacity alone.
My overall objective
I’m trying to design an intelligent, model-agnostic AI video-processing architecture that uses available compute and memory resources as efficiently as possible.
I’d like to push beyond the limitations of a single 24GB GPU, particularly for higher-quality video generation and more demanding AI workflows.
I’m willing to accept longer rendering times if that allows significantly better visual results or the use of models that otherwise wouldn’t fit.
If you already had an RTX 3090 and 128GB of DDR4 RAM, would you invest in something like the HP ZGX Nano, pursue more sophisticated CPU/RAM offloading, or take a completely different approach?
I’d really appreciate thoughts from anyone experienced with CUDA, PyTorch, distributed inference, ComfyUI memory management, NVIDIA GB10 systems, or AI video-generation architectures.
Thanks in advance!