beside the point

Also, beside the mark or question. Irrelevant, off the subject. For example, Whether you had insurance is beside the point; the accident is your fault. These terms came into common use in the mid-1800s. Also see neither here nor there.