Inside James Caan and Son Scott Caan's Bond: 'He's Always Been There for Me No Matter What'

James Caan and son Scott Caan had a bond that remained strong throughout life's ups and downs.

On Thursday, a statement shared on James' official Twitter page announced that the actor died on Wednesday at 82 years old: "The family appreciates the outpouring of love and heartfelt condolences and asks that you continue to respect their privacy during this difficult time."

James, who was previously married four times, is survived by his five children: Scott, James, Jacob, Alexander and Tara.

Actor Scott, 45, of Hawaii Five-0 fame, has previously been open about making peace with his famous dad after tumultuous periods in his childhood. He told PEOPLE back in 2010, "To say our relationship wasn't dysfunctional would be a lie." James, who had been candid about combatting a past drug addiction, added at the time, "He saw me when I was a little rough around the edges."

"Now," James, who shared Scott with ex-wife Sheila, said at the time, "we're really best buddies. ... I'm proud that he's grown up so well."

Scott added to PEOPLE that he had "no resentment" toward his dad. "He made mistakes that affected him — and me — but he's always been there for me no matter what."

For more on James Caan, listen below to our daily podcast PEOPLE Every Day.

James told Esquire back in 2003 about how his parenting differed from his own father's. "I never saw my dad cry. My son saw me cry. My dad never told me he loved me, and consequently I told Scott I loved him every other minute. The point is, I'll make less mistakes than my dad, my sons hopefully will make less mistakes than me, and their sons will make less mistakes than their dads."

"And one of these days, maybe we'll raise a perfect Caan," he joked.

In 2010, Scott and James spoke to the Los Angeles Times about costarring together in the 2009 movie Mercy, which Scott wrote. James said at the time, "I have a very unique relationship with Scott. He grew up with me," alluding to his years-long hiatus from Hollywood, when he paused his career to coach Little League and eventually sought help for his drug addiction.

"I went through some bad times. I lost my sister and I got a little goofy and I got on drugs — coke — and I went through all of that. He was attached to my hip," James said of son Scott.

"My dad is a good example of someone who digs into life and goes, 'Yeah, that tore my heart out, but I'm going to try again,' " Scott said at the time, adding that "his family was more important" than his work.

Added James, "The best advice I give to young actors … was what comes first is my family and my friends. No matter what heights you achieve in this business, what is inevitable in every single case is there is a slide backwards. Those people who've put all their eggs in the basket where it's everything, those are the people who hurt themselves. They get drugged out, they get really destructive. I'm not saying you shouldn't be the best. But when things go bad, he's got to have his dad, his girlfriend. He's got to have his brothers."