In the literature on autobiographical narrative, self, and identity construction, many researchers have taken narrative coherence as an important feature that reflects and shapes identity and sense of self. Commonly, this feature is defined and assessed in isolation, as if at stake were an autonomous text. We argue this approach is too narrow to represent things as complex as narrative, self, and brain. We explain this argument in discussing narratives by individuals with serious neuropsychological challenges: people who, due to illness or disability, cannot fully rely on their neurocognitive and narrative resources for their identity construction. We offer a broader view of the issue of coherence in autobiographical narrative that goes beyond a decontextualized concept of narrative, especially, by including (i) the intersubjective context in which stories are told, (ii) the larger autobiographical context of their narrator, and (iii) the wider socio-cultural context in which narratives and narrators are situated. Using narrative excerpts from adults with acquired brain injuries and neurocognitive disabilities, we point out how what is seen as (narrative) coherence of one’s brain, mind, and self changes when these contexts are taken into account.