MaverIQ Shows Fragmented GPU Memory Is Capacity You Already Paid For
The standard rule for serving a big model across GPUs is to split its layers evenly, so no stage becomes the bottleneck. UT Austin's MaverIQ measured what happens when you ignore that rule. In the project's own data, Llama-2-70B spread across eight GPUs as 2, 4, 6, 8, 12, 14, 16 and 18 layers ran 0.11% slower than a perfectly balanced 10 layers each. That one result turns stranded slivers of GPU memory into usable capacity.
