Recently, Blake Shelton announced he was stepping away from his job as a coach on NBC’s hit reality competition show, “The Voice,” after 23 seasons. Now he’s opening up about his reason why: his family.

In a new interview with PEOPLE, Shelton acknowledges how much he loves and will miss the show, but he’s ready to reprioritize his life to focus on his wife, Gwen Stefani and being a stepdad to her three sons.

“Gwen and I have done so much traveling and touring and work the first half of our lives that now we’re like, ‘Hey, I’m good to put my sweatpants on at 6 p.m. and watch ‘Ozark’ eight times,'” he tells the magazine. “That’s our life now, and we love it.”

Related: Gwen Stefani shares rare family photo of all 3 sons at her wedding

After spending the last several years focusing on his career, he says it’s time to settle down and be more present with his family. And being a stepdad is not something he takes lightly. Stefani shares her sons Kingston, 16, Zuma, 14, and Apollo, 8, with her ex-husband, Gavin Rossdale.

“If I walked away from my career at this time, the only thing that I run the risk of is having regrets that I’m missing out on some more important things in life. For now, that’s our kids. This isn’t about me anymore and never will be again,” Shelton acknowledges. And it’s true—it doesn’t matter if those children are biologically his or not, they’re still his kids and he’s responsible for raising them along with their other parents.

“Even though I’m a stepparent, I take that job very seriously. The kids see me as a very important person in their life,” he continues. “[When they ask], ‘Why isn’t Blake here?’ I take that stuff to heart. I’ve made plenty of money, but you can’t buy time back. I don’t want any regrets.”

Of Stefani, whom he married in July 2021, Shelton says he’s more than happy to feel “settled” with her at this point in their lives.

Related: These viral photos of a dad and stepdad are co-parenting #goals

“The relationship that we have is so natural and so normal. This feels like the easiest thing I’ve ever been a part of,” he says of marriage. “To me, she’s my best friend and everything that I need and lean on.”

He also says that he doesn’t feel guilty about “quitting” his job for the time being to enjoy life in their shared family home in Oklahoma, where they can get away from “everything” in L.A.

“I’ve had people say, ‘Man, come on. You’re not quitting ‘The Voice’ [to do] nothing!’ But I really am. Please, I’m accepting ideas,” he jokes. “It’s time to push some of this [work] stuff out and let more family and personal life in.”