We had a group of people coming from the Boston area, a few who were in Philadelphia, and a scattering of people from all over the rest of the world. The far-flung folks made their way to cities on the drive, and the rest of us picked them up on our way through town.
On the way home, we had a few people who couldn't stay the final night, and that all worked itself out as well - a person who needed to catch a bus back to NYC was dropped off at the station by someone else who was leaving early.
Overall there was really no drama around travel, despite representing 7 states and 2 countries. If you have clusters of folks coming from a single city or metro area, assign a captain who will coordinate getting those people together and heading to the retreat. Check to see who needs to leave early, and hook those folks up with each other to make sure everyone has a ride home.