The simple way is to find and delete that world's folder -- in Windows 7 that's C:\Users\O-Bakasama\Saved Games\Noxico. For any other OS it's the same place screenshots end up.
But I suppose if I replace the MessageBox.List() call with a bit of custom UI, I can add Delete and Rename buttons...
I think there should be a way to delete worlds, if so desired. Currently it seems that every time I want a new world it will just keep adding to the list of worlds in the start screen.