Tim Lake (Domhnall Gleeson) is a young man from Cornwall, England. He grows up in a house by the sea with his father (Bill Nighy), his mother (Lindsay Duncan), his absent-minded uncle, and his free-spirited sister, Kit Kat (Lydia Wilson). At the age of 21, Tim is told by his father that the men of his family have a special gift: the ability to travel in time. This supernatural ability is subject to one constraint - they can only travel to places and times they have been before. After his father discourages Tim to use his gift to acquire money or fame, he decides that he will use it to improve his love life.

The following summer, Kit Kat's friend Charlotte (Margot Robbie) visits to spend her vacation with Tim's family. Tim is instantly attracted to her and at the end of her stay, decides to tell her how he feels. She tells him that he should not have waited until the last day, that perhaps if he had told her earlier, something could have happened between them. Tim travels back in time and, the second time around, tells Charlotte in the middle of the vacation how he feels. In this instance, Charlotte uses the exact opposite excuse, saying that it would be better if they waited until the last day of the vacation and then something could potentially happen between them. Heartbroken, Tim realises that Charlotte is not attracted to him and that time travel will not empower him to change her mind.

After the summer, Tim moves to London to pursue a career as a lawyer. He is put up by his father's old acquaintance, Harry (Tom Hollander), a misanthropic playwright. After some months, Tim visits a Dans le Noir establishment, where he meets Mary (Rachel McAdams), a young American woman who works for a publisher. The two flirt in the darkness of the restaurant and afterwards, Mary gives Tim her phone number. Tim returns home to find a distr