Prince Harry and his wife Meghan Markle have been warned by a former royal aide not to try and “live like Princess Diana” by relying “on billionaires for jets and bodyguards.”

Patrick Jephson, Diana’s former private secretary, spoke out to say that Harry and Meghan should learn from the difficult times Diana had after her ties with the royal family were cut. While Diana and Prince Charles separated back in 1992, they decisively split after her infamous 1995 interview with BBC’s “Panorama,” in which she opened up about her marriage troubles like never before.

“Well, there were three of us in this marriage, so it was a bit crowded,” she said, referring to the fact that Charles had been having an affair with his now-wife, Camilla Parker Bowles.

“The temptation for some of today’s Royal Family is to live like Diana after Panorama: no longer anchored to the humdrum concerns of British subjects, preferring the ego rush of the world stage and the shallow flattery of the international rich and famous,” Jephson wrote for Daily Mail.

“As perhaps her younger son Harry has discovered, you can escape the Palace but finding relevance and respect as just another celebrity is much, much harder,” he added. “So much easier to take the shortcut: keep lawyers and PR wizards on retainer and preach platitudes to the unenlightened masses.”

“Pretty soon, you have crossed the line from setting a good example—all a member of the Royal Family really has to do—to setting yourself up as an expert in public policy without the trouble of winning an election,” he continued. “It’s surely not what conscientious princes (and dukes and duchesses) intend, but it’s the risk they run when they lose touch with real people’s everyday concerns; when their personal enthusiasms take priority over the much less glamorous traditional royal work of healing and encouraging that needs to be done every day.”

Harry and Meghan shocked the world when they stepped down as senior royals earlier this year. They then briefly moved to Canada before relocating to Meghan’s hometown of Los Angeles, California. There, they spent months living in the $18 million mansion of Hollywood mogul Tyler Perry before buying a mansion of their own in the ritzy area of Montecito, where they have been living with their 1 year-old son Archie ever since.