GTX 580 > 580 SLI > 580 Tri-SLI > 580 SLI > 670 >780
By my counts, the 780 is the first card, so far, that actually feels like a substantial upgrade from a pair of 580's, which are only a tick or two faster than your 480's. With the 780, while having reduced heat, noise and power consumption is a great consortium of benefits, not having to deal with SLI (even though it arguably works MUCH better than Crossfire in my own personal experiences) was the greater advantage. Although you may see less fps here and there in some games, the overall gaming experience is significantly smoother, without needing to worry about driver bugs (even though they may be few and far between).
At the end of the day, it really comes down to if you can afford it, and if you really want it. While it is a big purchase, if this is your (main?) hobby, and you have a good, steady income, then it's just money - you'll get more in due time. This is especially true if you are looking to pair up 780's in the near future.
Justifying a purchase is really an emotional response - no amount of numbers or facts will deter you if that's what your heart truly desires. You only live once.
(Maybe that in a weird way is your justification right there.)