The waiting is over! Have the times finally caught up with Godot?

Godot keeps on coming. A play that on its London debut in 1955 was greeted with bafflement and derision (“another of those plays that tries to lift superficiality to significance through obscurity,” wrote Milton Shulman in the Evening Standard) is now big box office. The latest iteration of Samuel Beckett’s groundbreaker comes in September when Lucian Msamati and Ben Whishaw appear in James Macdonald’s production at that temple of luxurious elegance, the Theatre Royal Haymarket. So what has changed in close on 70 years to make Beckett’s play popular?

The short answer would be that both the theatre and the culture at large are very different now from what they were in 1955. One thing Beckett taught us was that plays need neither spacious plots, sumptuous sets nor multiple characters to hold our attention: as Stoppard once wrote, “Beckett redefined the minima of theatrical validity” and Sir Tom, along with Harold Pinter, Caryl Churchill and countless others, have been beneficiaries of Beckett’s pathfinding. We are also no longer taken aback by the idea that plays, like life itself, are inexplicable and incapable of resolution: if anything, we have grown wary of artists, politicians and philosophers who assume they have readymade answers to the conundrum of existence.

But has the way we stage Waiting for Godot also changed with time? Having seen a dozen or more productions over the past 50 years, I detect palpable differences and, while there is no such thing as a definitive version, Beckett’s own production for Berlin’s Schiller Theater taught us much. I saw it when it transferred to the Royal Court in 1976 and was struck by several things. One was its stress on the physical contrast and spiritual kinship of the two tramps: Vladimir was huge, ungainly, pigeon-toed while Estragon was short-legged, crab-gaited and moon-faced but they sported the mismatched halves of two suits which they swapped after the interval. Another striking feature was the austere beauty of the production: as the two tramps watched the rising of the moon at the end of each act, Beckett evoked a painting by Caspar David Friedrich that was one of the play’s inspirations. But it was the stoical grace of the production that stayed with me.

Great plays change according to time and circumstance and one thing I’ve noticed is how much the English-language version gains from being cast with Irish actors. Walter Asmus directed a famous Gate Theatre Dublin production that came to London’s Barbican in 1999 with an astonishing cast – Barry McGovern and Johnny Murphy as the tramps, Alan Stanford and Stephen Brennan as Pozzo and Lucky – that perfectly caught the play’s haunting musicality and reminded us of Beckett’s debt to the plays of Synge and Yeats. I’ve also never forgotten a Garry Hynes production which I saw at Galway’s Druid Theatre in 2016. I had begun to think that the play had lost its capacity to shock and surprise but the agonising silence as Aaron Monaghan’s Estragon wrestled with a recalcitrant boot and the terrified panic with which Marty Rea’s Vladimir viewed the prospect of their separation heightened the play’s tragic quality.

You can’t discuss Godot without invoking Peter Hall who directed the British premiere at the Arts Theatre in 1955. I never saw that version but, by all accounts, it lacked the visual spareness that is the play’s hallmark. But I did see Hall’s two later versions and, on each occasion, he made new discoveries. From his 1997 production at the Old Vic I remember the rage of Alan Howard’s Vladimir and Ben Kingsley’s Estragon at the their endless entrapment. Even better was Hall’s final version at the Theatre Royal Bath in 2005 where James Laurenson and Alan Dobie as the tramps and Terence Rigby and Richard Dormer as Pozzo and Lucky existed in a state of mutual dependence that suggested the best we can hope for, in our brief existence, is the solace of companionship.

The difficulty Godot poses is that it is both tragic and comic but too much emphasis on the latter element can unbalance the play. On its American premiere in 1956, with Bert Lahr and Tom Ewell, it was misleadingly billed as “the laugh-hit of two continents” and died a quick death. In contrast Mike Nichols’s 1988 New York production, with Steve Martin and Robin Williams, was a box-office smash but, according to Frank Rich, the relentless pace and lack of rapport between the two leads undermined the play. I had similar doubts about a West End production in 1991, starring Rik Mayall and Adrian Edmondson, which by milking every line for a potential laugh sacrificed Beckett’s sense of desolation. Even when two great actors, Ian McKellen and Patrick Stewart, did the play at the Haymarket in 2009 , I felt the production had a patina of cosy charm reinforced by the sight of the two stars singing an old Flanagan and Allen number at the curtain call.

I still await the new Haymarket revival with eagerness. The test for me will be whether it acknowledges the play’s comedy without lapsing into self-regarding sentimentality and whether it captures, as Beckett’s own production did, the derelict dignity of the two tramps and the human capacity for endurance in a seemingly senseless universe.

• Waiting for Godot is at Theatre Royal Haymarket, London , 13 September–14 December