The sleek bullet train purred above fields, roads and watercourses on its stilted track, more than twice as long as Britain’s high-speed line. The termini were immaculate, the few stations en route as smart as those of the Jubilee Line extension in London. This was Taiwan’s High Speed Rail line between its capital, Taipei, and the second city in the south, Kaohsiung. I shouldn’t have been surprised; after all, per capita GDP in Taiwan has recently overtaken that of Britain. But Taiwan is hardly known as a tourist destination. It was to find out what lay behind its inclusion in a travel magazine’s “top ten countries for 2012” that I had joined a largely Australian group, travelling mostly by the country’s embarrassingly good railway and metro systems.

Most associate Taiwan with electrical goods and computers, perhaps as the refuge of Chiang Kai-shek after his defeat by Mao’s communist forces in 1949. Less well known is that it was colonised by the Dutch and Spanish before the Japanese took over in 1895, bringing the island into the 20th century literally and metaphorically before it achieved independence with peace in 1945.

The war resonated in a most unexpected way as we visited the Gold Ecological Museum at Jinguashi, where the mine formed the basis for an Allied prisoner-of-war camp from 1942 in a mountainous coastal area of dense forest and bush. One of our party, Alison, knew that her father had been a captive on the island during the war, but had no idea where until she read his name on the memorial to those who laboured in conditions of unimaginable cruelty. That rather put in perspective a display in the museum of the world’s largest pure 999 gold brick, at 222kg.

The first museum most visitors see, and unquestionably the country’s most important, is the National Palace Museum in the capital, Taipei; it has one of the world’s finest collections of east Asian art, which is not surprising since much of it was owned by Chinese emperors and kept in the Forbidden City in Peking.

Its survival is little short of a miracle, given that it was repeatedly crated up and carted round China before the advance first of Japanese and then of communist armies. Even if the collection had survived these onslaughts in its original home, it would probably have been destroyed during the cultural vandalism and mass destruction of the Cultural Revolution.

As it is, the 2,972 crates that were shipped to safety represent only 22 per cent of the objects removed from the threatened Forbidden City in Peking. Because only a small part of the National Palace Museum collection can be displayed, exhibitions are rotated, but the principal categories are jade carvings – some about 7,000 years old – ceramics, ivory, calligraphy, gems, bronzes and paintings. The workmanship is rarely less than exquisite. Some objects are made of bamboo and it was this Chinese symbol of longevity and resilience that inspired the architect C Y Lee to design what was briefly the world’s tallest building and is on every tourist itinerary: Taipei 101. High-speed elevators take just 37 seconds to reach the 89th of 101 floors for remarkable views from the observatory and an explanation of how the building has been proofed against typhoons and earthquakes, while the lower floors are devoted to shops.

Retail opportunities are not typically associated with monastic orders, but there is nothing typical about the gargantuan Chung Tai Chan Monastery in Puli Town, where tourists are escorted by saffron-robed monks sporting photo identity cards and earphones and by microphone-wired security guards. The $190,000 woodcarvings in the shop seem to be the least of their worries. If Mecca is turning into Las Vegas, as a recent report suggests, the Chung Tai Chan Monastery is more Canary Wharf meets Las Vegas.

The colossal structure, finished around the year 2000, dominates the surrounding countryside. Its gold-topped stupa on a 37-storey tower is flanked by sloping barrack blocks for the 1,600 monks, ending in wings with faux battlements, machicolations and arrow slits. As our Stuttgart-born guide talked about the monastery’s Zen master and Buddhist meditation, she led us into an immense hall with an entrance flanked by towering granite figures resembling the old kings of Gondor in Lord of the Rings.

The older temples we visited were much more reassuring in their obvious connection with local people and a harmonious place in the community. Every surface of the Fentien Temple near Chiayi is decorated in bright colours with landscapes, dragons, calligraphy and lanterns in honour of Mazu, a Chinese sea goddess. The roof is ablaze with vividly coloured glazed dragons dancing over the tiles. Worshippers buy spiritual money and burn it in a kiln-like fire opposite the temple in the hope of happiness, prosperity and long life. More specific individual requests are written on post-it notes and placed, for a fee, on rows of cones; the higher the post-it note, the greater the chance of success – and the higher the fee. The many mainland Chinese now coming to Taiwan see places such as the Fentien Temple as part of the heritage they have lost.

With a population of 23 million in an area not much larger than Belgium, Taiwan’s western coastal plain between Taipei and the second city of Kaohsiung in the far south is densely developed. To the east lie the hills and mountains that offer the visitor a very different experience. Regarded as one of the country’s most spectacular sights, the Taroko Gorge is the centrepiece of a national park, one of four alpine parks. The gorge is on Highway 8, one of a handful of roads linking east and west coasts and only completed in 1960, reaching a summit at 10,800ft. At the gorge’s narrowest point the sky is reduced to a narrow slit of blue between towering walls of perpendicular rock.

The park’s main hotel at Tianxiang overlooks the deeply ravined confluence of two rivers to become the Liwu River and is built on the site of one of Chiang Kai-shek’s 27 guest houses. Before breakfast I walked up to the Hidden Summit Pagoda on the hill to admire the 10 faces of the Bodavista looking in 10 directions. The only sounds were a woman sweeping leaves from the courtyard, birdsong and water against rocks. Rocks the size of three-storey houses interrupt the noisy flow of the beautifully clear Shakadang River, which is followed by one of the park’s most popular trails.

The air was filled with the sound of cicadas and the colours of myriad butterflies, some of 251 species found in the park. Huge spiders’ webs veiled the air between closely spaced trees, and above the path trees defied gravity by leaning over sheer cliff edges.

Logging of ancient cypress trees has given way to conservation in the Alishan National Scenic Area around 18 peaks to the east of Chiayi, with neat tea plantations covering the lower contours. The best way to reach the main hotel of Alishan House was on a corkscrew railway – built to extract logs but adapted to take tourists to the celebrated sunrise viewing point at Jhushan and the many forest walks among trees that were growing when William won at Hastings. Sadly, a typhoon has damaged some of the 49 tunnels and 77 bridges, causing a suspension of services. Once restored, it deserves to join the railway to Darjeeling as a World Heritage Site and become one of Taiwan’s “must do” experiences.