Tom Felton Recalls Lessons from 'Kind and Caring' Robbie Coltrane in New Memoir: 'He Was a Joker'

Tom Felton shares fond memories of working with Harry Potter costar Robbie Coltrane in his new memoir, Beyond the Wand.

In the book, released Tuesday, Felton describes Coltrane — who died Friday at age 72 — as "one of the few actors I recognized when we started making the Potter films, thanks to his roles in GoldenEye and Cracker," as well as someone who "had a knack of lightening the atmosphere" on set.

"He was a joker, but he was also the person that the jokes happened to. Or rather, he was the person who allowed the jokes to happen to him, and his reactions were priceless," writes Felton, 35, of Coltrane, who played Rubeus Hagrid — the kindhearted, half-giant Hogwarts keeper of keys and grounds who is often taunted by Felton's Draco Malfoy in the films.

Felton recalls a certain "phase" he and franchise star Daniel Radcliffe went through in which they'd "go around changing the language on people's phones so that it was difficult to find your way back to the English setting."

"Robbie was definitely the butt of that joke several times, because his reaction to it was so enjoyable," he writes. "He'd narrow his eyes, peer around and mutter: 'What f---er did that?' He acted as if he was ready to kill the culprit, but really he was just entering into the spirit of things."

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human-interest stories.

For Felton, the "kind," "caring" Coltrane passed on a bevy of crucial lessons to the young cast, which also included Emma Watson and Rupert Grint.

"Robbie was always keen to remind us that we weren't there to cure cancer," he says. "We weren't saving the world. We were simply making a film. We should remember that, not get too big for our boots and try to have a laugh along the way."

"He had a good dose of Hagrid in him: the big friendly giant who never lost sight of what was important in life," Felton adds.

He says that his "main contact" with Coltrane after the films "would be on press junkets and publicity tours," but they always "shared [a] passion" for automotive technology and Felton "always looked forward to publicity appearances with him because you were guaranteed a lark and a laugh."

RELATED VIDEO: Celebrating Robbie Coltrane's Beloved Character Hagrid: "I'll Not Be Here Sadly, but Hagrid Will"

Coltrane's agent and friend Belinda Wright confirmed the news of his death to PEOPLE on Friday, sharing that he died that day and calling the actor "a unique talent."

"He will probably be best remembered for decades to come as Hagrid in the Harry Potter films, a role which brought joy to children and adults alike all over the world, prompting a stream of fan letters every week for over 20 years," Belinda continued.

Felton joined much of the Harry Potter cast in sharing individual tributes to Coltrane, including Radcliffe, 33, Watson, 32, and Grint, 34, plus James and Oliver Phelps, Matthew Lewis, Bonnie Wright and David Thewlis.

"One of my fondest memories of filming Harry Potter was a night shoot on the first film in the forbidden forest," Felton wrote in his tribute. "I was 12. Robbie cared & looked after everyone around of him. Effortlessly. And made them laugh. Effortlessly. He was a big friendly giant on screen but even more so In real life. Love you mate — thank you for everything xx."

Beyond the Wand: The Magic and Mayhem of Growing Up a Wizard is available now wherever books are sold.