I think you're referring to a fic in esorlehcar's Down to the End 'verse. The precise sentence goes "The physical aspect is in some ways almost beside the point; Sam's pretty sure the last few years would have gone exactly the same without it." and is from the fic titled Practical Men of the World (http://esorlehcar.livejournal.com/372671.html). I think.