Starting with a new automotive relay is the easiest way to confirm that's the problem and they're only a few dollars. If that doesn't work, we'll get into Analog VM 101. They aren't magic but neither are they intuitive.
The first steps - you can do this now, learn it and it'll help you forever. Again, I don't know what you know, so some of this is very basic but 'must know' stuff.
1. Find the AC and DC positions on the dial, we'll be using DC
2. Turn the dial to the lowest setting above 12 (volts). My small VM has 10, 50, 250 and 500, so I'd set mine to 50 (volts)
3. Plug the Red probe into the red port on the meter or (+), Black into black, COM or (-).
4. While watching the meter, touch the black probe to the negative terminal on the battery and quickly touch the red to positive. What we're looking for here is that the meter deflects upward and doesn't bounce backward - that would tell us the leads are backward or we're on the wrong battery terminals. Always start here, this is how we know if we're touching 12 volts (battery voltage)(+) or ground (-).
5. Now you can hold the black probe on a known Ground anywhere and touch wires looking for 12 volts (or Hot or Positive or voltage). Or you can touch the Red probe to a KNOWN voltage and 'look' for ground with the Black probe. This is what we'll be doing with the relay - and why I explained all this.
An example would be touching the black probe to a known ground, touching the red probe to Key On power, as you turn the key on and off, the meter needle should bounce with the key position showing that you have power. If it doesn't, that's not a Key On source or you've picked a bad ground (which is what I did when I tried to use the contactor mounting bolt).