The ruins of Findlater Castle cap the headland like a broken crown – the 15th-century stones half covered by grass and gorse bushes – while the North Sea swirls 50 feet below.

This is Clan Ogilvy’s long abandoned family seat, but seabirds reign over the place now. As I make the heart-quickening scramble across from the mainland, a jangling tune rises above the lashing waves: a solitary bagpiper is playing beside the crumbling fortifications. It’s the stuff of Outlander-fuelled dreams. And incredibly, aside from the musician and his wife (Americans retracing some distant family heritage, it turns out), I have the scene all to myself – even on a sunny spring weekend, when the sky’s as blue as the St Andrew’s flag.

It’s a similar story throughout my three-day drive along the Moray coast: near-deserted sand dunes, sleepy estuaries, harbour towns where locals outnumber second homers. Ospreys spend their summers nesting in this part of north-east Scotland and pods of bottlenose dolphins gather offshore. But tourist hordes? They don’t seem to have gotten the memo about Moray’s charms.

Hear the words ‘Scottish road trip’ and the North Coast 500 most likely spring to mind. It’s exactly a decade since the neglected North Highlands coast was cannily rebranded, turning the 516-mile circuit from Inverness into a bucket-list road trip – ‘Scotland’s answer to Route 66’. This clever bit of branding has undeniably delivered an economic boost. Yet many residents now complain it’s become too successful, with local infrastructure straining under an influx of #vanlifers bringing litter and dangerous driving, as well as contributing to a rise in housing prices.

Read more: 8 gorgeous Scottish islands to visit

A new visitor pledge aims to encourage more responsible behaviour, but it may also be time to try alternative stretches of the coast and spread out the travellers lusting after an epic Scottish road trip. There are, after all, more than 6,000 miles to choose from on the mainland alone. So, landing in Inverness airport, instead of joining the rental cars and minivans following the brown NC500 road signs north-west, I turn due east.

“Follow the A96” admittedly aren’t the most poetic directions, but my route hugs the lesser-visited side of the Moray Firth sea inlet, and nearly every turn off the main trunk road leads to soul-stirring shores. Forty minutes in, my first stop is Culbin Sands, a stretch of golden dunes and salt marsh where oystercatchers and godwits forage, fringed by 9,000 acres of woodland.

So far, the area’s living up to its ‘Sunshine Coast’ reputation; a friendly micro-climate makes this one of the driest and sunniest parts of Scotland. Still, be warned that the North Sea remains bracing year-round. Findhorn is the best place to take a dip thanks to the presence of Watershed Sauna, working up a sweat in the converted horsebox before hopping straight into the waves. Lucky bathers may well find a seal bobbing about nearby – a 200-strong colony congregates at the beach’s north end.

Read more: Best hotels in Inverness for Highland views and history

The backdrop to my drive is gently rolling farmland – which means plenty of roadside farm shops for picnic supplies – and pine forests threaded with cycling and hiking trails. Just before crossing into Aberdeenshire, the road sweeps down into Cullen beneath a towering Victorian viaduct. Granite cottages form a few neat rows, like an audience before a stage (there’s always one headline act: the sea), alongside antiques shops and cafes pretty much duty-bound to serve Cullen skink, the smoked haddock chowder that originates here.

Quainter still is Portsoy. It’s easy to imagine fishermen hauling their nets into the Old Harbour during the nineteenth century herring boom (if the place looks familiar, it was a Peaky Blinders filming location). As the ever-variable Scottish weather turns, the sea now looking as harsh and dark as the shale cliffsides, community-run museum The Salmon Bothy provides both shelter and a deeper dive into Moray’s fishing heritage.

The drizzle also derails my plan to sample the much-raved-about Portsoy Ice Cream Parlour, instead sending me through the cheerful red doors of The Port House Cafe for a warm cinnamon bun and Speyside Coffee Roasting Co brew. The maritime theme is writ large here, with a fishing boat for a counter and oars hanging from its oak-beamed walls. If you fancy setting sail for real, Guide Charters runs catamaran trips out of Macduff marina, scouring for puffins and porpoises.

From here, the road trip can divert inland to Speyside’s whisky distilleries and the Cairngorms’ dramatic loch-riven landscapes, before looping back to Inverness on the A9. Alternatively, continue along the coast all the way to Aberdeen on the North East 250 – one of several Scottish road trips now borrowing from NC500 nomenclature in the hopes of giving other regions a similar boost.

For now, this stretch of Moray coast remains magically unassuming, a briny gem hiding in plain sight between better-known neighbours. Although no doubt it’s only a matter of time before some enterprising group coins a catchy moniker for it. See you on the MC90 next summer?

The Scottish Outdoor Access Code has information on how to explore and camp responsibly in Scotland. For more road trip inspiration, local accommodation and sights, see visitscotland.com.

Read more: The best beaches in Scotland to visit this summer