How do we know that silver/reflective things are silver/other colors?

i thought of this while looking in a mirror... how do we know that its silver in color when it makes a reflective image that has the same color as anything it reflects?