Graphics cards have a small amount of very high bandwidth dedicated RAM close to the GPU. PC ram is further from the CPU which increases latency due to the speed of light lag. CPU's get around this by using a lot of L2/3 cache. Which means PC's don't really care as much about RAM bandwidth. Instead they want lots of RAM.

Thus main memory is much slower than GPU bandwidth. So, now your on chip GPU is memory starved and reducing the bandwidth avalabel to the CPU. There are some advantages if you want to pass stuff back and forth, but the standard rendering pipeline is designed for GPU's.