Deinstall Debian Desktop, X, etc.

List all the gorp that is running

tasksel --list-tasks

Now remove the gorp

apt-get purge $(tasksel --task-packages desktop)
apt-get purge $(tasksel --task-packages gnome-desktop)

And remove the gorp's dependencies

apt-get autoremove