He may leave anytime if he wishes to leave. Likewise, I may leave anytime if I wish to leave. That is our privilege as unmarried people; we don’t have to ask permission from the government on whether we may terminate our relationship or not. It is, after all, a private matter, and should stay that way.
There is nothing less romantic in a relationship than involving the government and the church in what should be a private matter between (or among) consenting adults.