If he hadn’t become a musician, Paul McCartney says, he would probably have been an English teacher. He has fond memories of his English teacher, Alan Durband, who studied with FR Leavis and taught the young Paul the value of close reading. When he wrote songs with John Lennon, the chords and melody came first. But the words mattered too. Where the straight-up, irony-free early lyrics wooed their audience through a flurry of pronouns – She Loves You, From Me to You, Please Please Me, etc – the later lyrics aspired to poetry.

Take Eleanor Rigby, which began as a song about the kind of old lady McCartney did chores for as a scout during bob-a-job week and who he thought of calling Daisy Hawkins until working with Eleanor Bron on the film Help! and spotting a shop sign with the name Rigby in Bristol. “The secret to successful songwriting is the ability to paint a picture,” he says, and the picture of Eleanor Rigby “picking up rice in the church where a wedding has been” perfectly captures her loneliness, just as the line “writing the words of a sermon that no one will hear” does with Father McKenzie (originally Father McCartney, till a trawl through the phone book turned up a suitable trisyllabic alternative). It’s a homespun English lyric – “the face that she keeps in a jar by the door” alludes to Nivea cold cream, a favourite of McCartney’s mum – with universal resonance: Allen Ginsberg and William Burroughs were among the song’s biggest fans.

Numerous biographies have traced the origins of Beatles songs. This is the McCartney version. Spread over two lavish volumes and more than 900 pages, and supplemented by memorabilia from the million-plus items in his archive (photos, posters, paintings, jottings and letters), the book came about through conversations with the poet Paul Muldoon: 50 hours of them, in 24 sessions between 2015 and 2020, covering 154 songs. On the face of it, the two Pauls have little in common: one a complex poet, the other a pop star. But they share an Irish heritage. And a few of McCartney’s rhymes (pataphysical/quizzical, Edison/medicine) wouldn’t look out of place in a Muldoon poem. At any rate the two hit it off. Though Muldoon has edited himself out of the text, you can sense him in the background, prompting and prodding. In effect the book becomes an autobiography, with Muldoon playing the part that Dennis O’Driscoll played in the interviews that became Seamus Heaney’s autobiography, Stepping Stones.

The biggest influence on McCartney’s music was the death of his mother, Mary, when he was 14. He used to deny that she lay behind the words of Yesterday (“Why she had to go I don’t know, she wouldn’t say”) but now accepts she must have been. He wrote more directly about her in the year she died, 1956, in I Lost My Little Girl, a song not released till 1991. And she is namechecked (“When I find myself in times of trouble/Mother Mary comes to me”) in Let It Be, a phrase she liked to use and one that also appears in Hamlet, which McCartney read at school. A midwife in life, she was also a midwife in her afterlife, helping to deliver some of his finest songs.

McCartney has similarly fond memories of his trumpet-playing father, whose love of crosswords he compares to his own approach to writing songs. When he sat down with Lennon – two guitars, two notepads, two pencils – they would have a song written within three hours: “After that your brain goes a bit.” You’d think there must have been sessions when nothing came off but he doesn’t remember any.

He talks a lot about Lennon, nostalgically and with affection (“I still have him whispering in my ear after all these years”), and is keen to emphasise that they ended on good terms; at their last meeting “we talked about how to bake bread”. Harsh words were exchanged when the Beatles broke up, with the acerbic John scornful of Paul’s taste for “silly love songs”, to which he retaliated by writing a song called Silly Love Songs. But till the breakup their differences were productive: “I could calm him down and he could fire me up.” They mirrored each other, John with his right-handed guitar, Paul with his left-handed one. And their tug-of-war rivalry produced brilliant harmonies. “We thought of ourselves as Lennon and McCartney from early on,” he says, a double act like Gilbert and Sullivan or Rodgers and Hammerstein.

The lyrics he wrote as a solo artist and for Wings are included here too. And many aspects of his offstage life are discussed along the way: his pacifism (which began after he met Bertrand Russell), vegetarianism, bird-watching, parenting, painting (which took off after a chat with Willem de Kooning) and unapologetic cheeriness (“it’s OSS: Optimistic Song Syndrome”). All kinds of music influenced him, Cole Porter as well as Little Richard: “No one thought it at the time but we were really big fans of the music that came out of our parents’ generation.” But the real revelation is how much he took from books – “intertextuality as they call it in posh circles”. Among the writers he alludes to are TS Eliot, George Orwell, James Joyce, Philip Larkin, Harold Pinter, Adrian Mitchell (“a good friend”), Eugene O’Neill, Lewis Carroll, Edward Lear, Sean O’Casey, Charles Dickens, LP Hartley and Louis MacNeice. And though the tone of the book is conversational, Muldoon’s editing ensures that it’s also quote-worthy: “Writing a song is like talking to a psychiatrist”, “The vignette is really my stock in trade”, “It’s not so much that I compose songs, they arrive”.

The most startling such arrival was Yesterday, the tune of which was in his head when he woke up one day and which seemed very familiar; only when he played it to others did he realise it existed only in his head. Getting it down, he used dummy words: what became “Yesterday all my troubles seemed so far away” began as “scrambled eggs, oh my baby, how I love your legs”. The backstories to the songs are often as interesting as the lyrics. With Ticket to Ride he and John were also thinking about a trip they’d made to Ryde, on the Isle of Wight; Blackbird, with its “broken wings”, was written after the assassination of Martin Luther King; “Hey Jude was originally Hey Jules and written for the young Julian Lennon after John had divorced Cynthia;the portrait of a community in Penny Lane took its bearings from Dylan Thomas’s Under Milk Wood while She’s Leaving Home “was almost like a shooting script for the Wednesday Play”.

The book won’t persuade the Nobel literature committee to honour McCartney as they did Bob Dylan, and though he once wrote a song about the Queen (“a person I admire very much”), he won’t be the next poet laureate. Stripped of the music, the words on the page can look random or banal. But at best he’s a wonderfully versatile lyricist: troubadour, comedian, elegist, social commentator, pasticheur. And anyone with even half an interest in the Beatles will find The Lyrics fascinating.