It’s all a bit much for Edinburgh. This is really a small town; look north from Frederick Street and you can see clean across the Firth to the Kingdom of Fife, while from Edinburgh Castle you can see the Pentland Hills to the south. In fact, wherever you are in the city, its ancient geography confronts you: from the extinct volcanic mound that is Arthur’s Seat to the forbidding lump of basalt atop which the castle squats, to other remnants of geological upheaval, such as Calton and Corstorphine Hills. This is a landscape carved by fire and ice, and the city tells its story in stone. Unlike the brick-built cities of the south, Edinburgh’s houses are granite grey. Its tenement streets – particularly on those wet Scotttish days – are as imposing as cliffs, while the geometrical proportions of the New Town are the imposition of the self-confidence of the Scottish Enlightenment upon the recalcitrant nature of the land itself. All of Scottish history is here on display, in the juxtaposition of Old Town intricacy and New Town rationalism. No other city tells you its story in a more up-front fashion, but few other cities have as many secrets waiting, hidden, for those willing to take the time and turn from the high road, through a dark archway or down a hidden wynd.

Go. It may be touristy, but Edinburgh Castle (Castlehill, 225 9846, www.historic-scotland.gov.uk) is a nation in stone, its history revealed in the dominance of the basalt outcrop upon which it stands. The castle has stood here for 1,000 years. The oldest extant building is the tiny 12th-century St Margaret’s Chapel, dedicated to the canonised wife of King Malcolm III, himself immortalised in Macbeth.

The castle served as a royal residence until 1571, when Mary Queen of Scots’ supporters were besieged in the castle by the Regent, governing on behalf of her son, the infant King James VI (James I of England). The royal residence was then moved to the Palace of Holyroodhouse and the castle became a military stronghold. The best way to soak up the atmosphere inside is to join a tour led by a guide.

The royal castle and the royal residence of Holyroodhouse are separated, appropriately enough, by the Royal Mile. This is the ridge sloping from the heights of the castle down to the dell beneath the volcanic eminence of Arthur’s Seat, a green hill sheltering the palace.

In the Middle Ages, the Flodden Wall ran around the area known today as the Old Town. As the population grew, housing was built higher, over the long gardens on either side of the ridges. The often-dramatic descents on either side of the central street mean that some buildings that look only a few floors high on one side may tower many storeys on the other.

The charming closes, wynds (stone staircases) and tunnels leading from streets above to those below are the Mile’s most attractive features. Gladstone’s Land (477B Lawnmarket, 226 5856) is a six-storey house kept in 17th-century style; it offers a fascinating look at those times. For those wanting to live history, it has self-contained apartments for rent.

Mylne’s Court leads to the former Scottish Parliament Debating Chamber, while Lady Stair’s Close takes you to the house where Robert Burns lived, now the Writers’ Museum (Lady Stairs Close, Lawnmarket, 529 4901, www.cac.org.uk), with memorabilia from Burns, Sir Walter Scott and Robert Louis Stevenson.

Continuing the literary theme further down the hill, the name of Fleshmarket Close was used by Ian Rankin for his latest Inspector Rebus novel. Fans of Rebus should also visit the policeman’s favourite pub, the Oxford Bar (8 Young Street, 539 7119, www.oxfordbar.com).

Backtracking, there’s the sombre and magnificent St Giles, which dates from 1120. Formerly a cathedral, it’s now the High Kirk of Edinburgh. St Giles is by the former site of the Tolbooth Prison, where executions traditionally took place. The local custom is to spit as you pass… but it’s optional.

Just past the junction with the Bridges (North Bridge to the left, South Bridge to the right), Niddry Street dips steeply towards the Cowgate, damaged by fire in 2002; behind this is a warren of cellars built into the arches of South Bridge in the 18th century, and then forgotten. This shabby, spooky area is beloved of ghost-tour companies, whose adverts are everywhere.

Back on the High Street, you’ll find its name changing to Royal Mile and Canongate before it reaches the seats of power, old and new. First take a look at the Holyrood Building, the site of the new Scottish Parliament (Horse Wynd, 348 5000, www.scottish.parliament.uk), and decide for yourself if the money was well spent. There are guided tours if you want a detailed inspection.

Then cross over to the Palace of Holyroodhouse (Holyrood Road, 556 5100, www.royalresidences.com) to compare and contrast. The palace was originally built by James IV so that the family could be near Holyrood Abbey (now in ruins nearby). Despite being burned and looted by Henry VIII’s army in 1544, and further damaged by Cromwell’s gang in 1650, it remains a solid, elegant building and has lovely green gardens sweeping out behind it like a train after a wedding gown. It’s a favourite of the Queen’s, and when she’s in residence, her flag flies from the palace roof.

The New Town is actually quite old. It dates back to a 1766 plan by James Craig. His scheme was to turn a pleasant pasture known as the ‘long dikes’ into a desirable residential area and what would today be called a ‘prime greenfield site’. The wide, spacious streets, grassy squares and classical lines of the area contrast greatly with the comparative gloom of the Royal Mile.

Princes Street – Edinburgh’s high street – neatly divides the old area from the newer, and gardens occupy the space below the castle. The perspex-sided bridge over the railway line that runs through the gardens is a prime spot for train-mad toddlers, while the dark and lacy Scott Monument at the end of the Princes Street Gardens stands out against the gentler architecture nearby.

Next to the Mound, which stretches as far as the Royal Mile, the National Gallery of Scotland (624 6200, www.nationalgalleries.org) holds its Raphaels and Botticellis within stately walls. Together with the neighbouring Royal Scottish Academy, it has been restored and renovated, with a new underground space opening on to the gardens.

If the sun happens to shine, then head to Stockbridge for a peaceful stroll along part of the 21-mile Water of Leith walkway. Downriver the Royal Botanic Garden (Inverleith Row, 552 7171, www.rbge.org.uk) holds the huge temperate Palm House.

The secluded Dean Village and the impressively stocked National Gallery of Modern Art (75 Belford Road, 624 6200, www.nationalgalleries.org) reside upriver.

• Tourist information: above Princes Mall, 3 Princes Street, New Town (473 3800, www.edinburgh.org).