Concerned mum, child custody

I have 3 children (12, 7 and 5) the 7 and 5 year old share the same father.
The father and I were together for 6 years, during this time, he worked away and was rarely around or present in their lives, I have been the primary parent since the day they were born.
Fast forward to now, we are now divorced, he has a new relationship and I am re married.
So far co-parenting was going fine, he would come and see them, stay in our home (as he was in a truck and didn't have anywhere to go to spend time witj them) for 24 hours once every few months, longest stint was 18 months without seeing them.
He now as of recently lives in Victoria with his girlfriend (they have only been together for 5 months and she hasn't met the children) I live in Queensland, where he and I originally lived, was his choice to move interstate.
He was in Western Australia, then moved here to Queensland so he could build a relationship up with the children, but within a few weeks of being here, he met his girlfriend and moved to her home in Victoria. He was here and could have had a relationship with the children but then just left. And now wants to take them, makes no sense.
He has never wanted full custody or even part custody, has never said a bad word about my parenting or to whom I'm now married to, in fact I have messages from him saying that we're doing an amazing job at raising them.
The only bad thing was he refuses to pay the correct child support, I brushed it off, cause frankly i didn't and still dont care about the money
Anyway suddenly out of nowhere, he has threatened to take full custody, all the way to Victoria, away from their home, family (the childrens whole extended family are here, grandparents, aunts, uncles) their school and friends, he just simply wants to take them and uproot them from their whole lives
No idea why. I'm terrified for my children. They don't do well with strangers or unfamiliar places. Their father is their father, but they hardly know him, he's hardly seen them since they were born. All the children know him as 'dad' who brings them presents, but they don't actual have any father/children type relationship. And being an interstate truck driver, he wouldn't even be around, it would be his girlfriend looking after them and she's a stranger to them. She wouldn't know the first thing about them, what they like or dont like, how to answer any medical questions
My 7 year old has medical needs on top, he requires medications, it's just so very concerning.
I have sought legal advice, and am waiting for the mediation process to start to get court orders in place, but I'm terrified he will just simply rock up and forcefully take them