Our party has a Lawful Neutral Oathbreaker Paladin that has been very much a murder hobo from the beginning. We have been able to keep him in check in most cases but in our recent session he went off on his own and went full murder hobo.
We encountered a hag in the woods but she never attacked us and let us all go. When we got to the next town he went off on his own to ask about it. They said that she protected the town and was allowed to stay in the woods. He decided this made them all evil by association and stabbed a clerk and threatened all of their lives. Due to the number of guards called against him he backed off but had fully intended on killing everyone in that building.
He managed to escape on horseback and rejoin the party without any of our characters knowing. We all expressed our displeasure with him about his actions out of character but he argued that he was playing in character and refused to see our side.
What should we do about this? Our DM is debating changing his alignment and possible consequences later but is unsure what to do.