I suspect that they would still share the same COG. Presumably a rocket could be treated in the same way by considering the unchanged initial COG of rocket plus fuel. To extend this idea - a rocket which reached a nearby star would still have its COG at its starting point on the launch pad. The rocket has gone to a star but its COG hasn't gone anywhere. This seems to be an odd concept but I suspect it is true in theory.
I think this model only works if it starts in space with no other friction, gravity, rotations, etc. I also don't know if it accounts for parts the spaceship might shed. Gases thrown at the launch pad don't go past it, but end up swirling around next to it in the same atmosphere the spaceship is being pushed through. Consider Apollo 10 100,000 miles out on its way to the moon. It's "coasting," burning no fuel, most of the Saturn V has been dropped into the ocean, and its COG is allegedly rotating around on the surface of the earth in Florida. A few days later, the command module is back to the same distance from the earth, falling to it, but a huge amount of fuel has been thrown out around the moon, the descent stage dropped onto the moon, and the ascent stage fired off into the solar system, and the COG is supposed to still be in Florida?