Not at all, the Oxford Tube is super flexible
For example, If you get off the coach at Notting Hill Gate, you can make your return journey from the stop at Victoria
You cannot however, reboard to continue the journey in the same direction