Bergen is the most beautiful place in Norway! I was traveling around the country last summer and Bergen was the best part of my journey. Although it's the second city of Norway, it's peaceful, clean, and it seems more like a little town than a city! I recommend to visit the Bergen Bryggen -- incredible, the pretty little streets with its wood houses -- and go up Mount Floi with the Floibanen (incredible sightseeing of the whole city). Yes, Bergen is expensive, as are all the places in Norway. I ate sandwiches and salads with ingredients I bought in the supermarket to save money, but a few times I ate out for "cheap" price -- in Scotsman (the cheapest beer in Bergen, with good ambience and fine food) and in Peppe's Pizza (enormous dishes for reasonable price, I recommend to share a dish for two people, because my friend and I never could finish a dish by ourselves, and it has a nice pavement café). It rains a lot but I think it gives the city a dreamlike atmosphere, like if you were in a fairytale. You have to go there! You'll enjoy your visit.
Bergen is very pretty, surrounded by hills and with a beautiful harbour. I suggest you bring a raincoat though, cause it rains 275 days a year (Lonely Planet statistic). It is pricey, so be prepared. If coming from Oslo take the 'Norway in a Nutshell' option and get off the train at Myrdal, train down to Flåm, ferry to Gudvangen, and catch the bus back to Voss and get back on the Olso-Bergen railway. You can buy a pass which includes all transport (visit www.norwayinanutshell.com). We found a cheap pub which did dinner and a Pepsi for 50kr (cheap by Norwegian standards), it's called the Scotsman and the food is ok for the price. Tip: Shops are closed on Sundays, including supermarkets, and the place is pretty dead.
Bergen is the 2nd most populous city in Norway (there are about 260,000 residents). The city dates back to the 11th century, and is very well known for its great outdoors activity possibilities and culture. The city was formerly Norway's administrative capital.
Bergen is served by its own airport (Bergen Airport Flesland, BGO). There is a railway that connects Bergen with Oslo, and there are some scenic views on the Bergen end of that trip. If you're on a tight budget, you might want to take a bus to Bergen, although it is not the most time-effective option. There are also some ferry services.
Hostels in Bergen are of good quality. Like most of Norway, the standard of living is very high and the prices are also a bit on the higher side. You'll be looking at about 25 euros for a dorm bed, and more for double / twin rooms. Bergen hostel owners and staff speak English and are a great source of information and tips when planning your trip in and around Bergen.
The Bergen city center is fairly compact so your best bet is to just walk around to see the sights. There is a public transport system (which includes buses and light rail, for example) available, should you need to utilize it.
There are plenty of sights to see when you stay at a Bergen hostel: the downtown area is great to start from. If you're interested in art and museums, you'll find plenty of those in Bergen.