In recent months I've become a dedicated Banshee user, and finally decided to drop Rhythmbox (bit dated / behind with the times) and Songbird (too slow for the size of collection I have). As of around v1.5.2 in Banshee they had fixed the slow load-up, and added a filesystem watcher. With those two things fixed, I made the switch.
I currently run the latest version of Banshee via the PPA, but recently had the problem of Banshee crashing a lot. I gather it's from incompatible data between an older version I was running and the latest version. It was upon using v1.5.4 that if I tried to access certain tracks it would just crash.
I ran Banshee via terminal and checked the logs, but couldn't get much info back. After reading around I found rebuilding the Banshee database fixed the issue, and allowed me to keep my library intact. This can be done by running:
sudo apt-get install -y sqlite3 &&
sqlite3 ~/.config/banshee-1/banshee.db ".dump" > dump &&
mv ~/.config/banshee-1/banshee.db banshee.db.backup &&
cat dump | sqlite3 ~/.config/banshee-1/banshee.db;
It basically dumps out your old database and recreates it. Thanks to gabaug for the fix.