Cameron Millar's penalty goal from halfway with two minutes left sealed a see-sawing victory at Forsyth Barr Stadium on the opening night of competition.
Joseph is one of two favourites for the All Blacks coaching job vacated by Scott Robertson's axing last month, two years out from the World Cup.
He has plenty of work to fix a shaky Highlanders lineout and some questionable kicking from his playmakers, but his vow to allow his team to play attacking rugby was evident.
The Highlanders opened the scoring after two minutes when Millar slotted a penalty from close range.
The Crusaders won the ball back twice early from errant lineout throwing, and another scrum win against the feed.
Despite their set-piece woes Caleb Tangitau extended the Highlanders' lead in the 15th minute with the first try of the season, beating two markers on the right flank to cross from 30 metres out.
The Crusaders started to settle into the match and responded five minutes later, Noah Hotham keeping the ball at the back of a ruck and running in down the right touchline.
Rivez Reihana added a penalty on the stroke of half-time to give the Crusaders their first lead of the game, 10-8 at the break.
Angus Ta'avao regained the lead for the Highlanders five minutes into his debut for the team, the prop coming off the bench at half-time and bustling over next to the posts.
Jonah Lowe added the Highlanders' third, Timoci Tavatavanawai sparking a break from inside his own half.
Crusaders coach Rob Penney turned to his bench immediately, All Blacks Codie Taylor, Fletcher Newell and Will Jordan among those introduced.
Jordan took 10 minutes to get on the scoresheet, crossing after a slick set play to reduce the deficit with 19 minutes remaining.
Taha Kemara added a penalty with eight minutes left to restore the Crusaders' lead, before Millar's penalty to seal a dramatic win for the home team.