Newbie qn re O2 PAYG & a cheap phone

Having looked at the various PAYG deals on offer, it seems that O2 has
the best tariffs for my needs. The only problem is that the phones
that o2 offer with the PAYG package are more expensive than I need. I
only want a cheap phone with a monochrome screen and no fancy extras.

If I buy a "new or used" cheap phone on Amazon.co.uk or eBay, with an
o2 PAYG sim, will that give me what I want?

Thank you,

Jim