But there is one way to get out of the center for cheap: it is to use the regular Kolumbus busses! Here are 3 ideas for you.
1) Over the Bridge
It is almost impossible not to notice this bridge. Did you know you could walk on it? I generally recommend to take the bus number 4 from city center in the direction of Rosenli and to stop at the first stop after the bridge. Then you can either go down to the level of the sea (there is a small path through the rocks) or simply walk back on the bridge and enjoy the view of the sea and the mountains.
There are many beaches in Stavanger. Sola beaches being the most well-known ones, but there are others, even easier to access by bus. For example, if you take the bus number 3 to Viste Hageby and stop at the last stop, you will only be at 5 minutes to the sea (ask the driver for directions). It is really easy and there are way less people than at Sola beaches.
Some other possibilities: take line 2 or 3 in direction of Sandnes and stop at Vaulen. Take the Stasjonsveien and it will lead you directly to the sea. Or you could even try the beaches at Mariero. No sand, a lot of rocks but the sea is there.
3) To Rennesøy
This is a fantastic trip and one of my favorite. Take the bus 10 to Rennesøy. It will drive for quite some time (through tunnels and everything) and arrive at Rennesøy (the second or third island). It then drives around the island and back to Stavanger (so you use the bus going in the same direction when you go and when you come back).
The advantage is that the sea is everywhere in Rennesøy and there are generally no tourists. Just stop where you want or even stay in the bus (it would then just be a tour of the island).
Personally I like to stop before the Coop (can ask the driver, I am not sure about the name of the stop) as I am then able to buy some drinks or food, or anything I would want.
You don't need to pay much to enjoy Nature in Stavanger and this is one advantage of this city. The seas is omnipresent abd you just have to find "your spot".
PS: I must apologize for the lack of pictures. I had taken many for each trip but my PC crashed and I lost all the memory. The photo of the bridge is an old one I had. Sorry for that.