First, your premise is that Kavanaugh didn’t do it, and that explains and justifies his unhinged, hyper-partisan attacks against those who dared to air Ford’s allegations. That premise obviously may be unfounded.
But no one is saying someone should be found guilty just on someone’s say so. The allegations must be tested. And because there was a whitewash and the FBI was hamstrung, all we have is the testimony of Ford and Kavanaugh, and other things we have learned.
Ford answered all questions put to her, and Kavanaugh was absurdly evasive, choosing instead to respond to questions with lengthy pre-packaged, repetitive speeches.
We also have accounts of Kavanaugh’s friends and contemporaries saying that Kavanaugh often got blasted on booze, and was aggressive and belligerent when he did so.
Adding to that, Kavanaugh clearly lied about many things (all of which the GOP is simply ignoring, BTW).
Compare their testimony and Kavanaugh’s clear lying about many things, along with Kavanaugh’s history of blotto drinking and belligerence, and it’s fair to say that Kavanaugh very likely did it. Not just because “someone said so,” but based upon all these things.