In most cases no, most access points are capable of doing this though. There are a few exceptions to this rule though so it may pay to check up on the router specifications and see if it supports wireless client mode.
A point worth mentioning is that the Linksys WRT54G and many other Broadcom-based routers can be flashed with third-party firmware such as
DD-WRT which allows you to use the routers as a bridge or client, and would achieve what you require.