Photoshop CS6 & Lightroom 5 hardware requirements

I have grown tired of getting the "not enough RAM message" from Photoshop, so I guess I must start considering an upgrade. In this respect, since a)my budget is somewhat limited and b) I am not sure which "parts" matter the most (more on this below) I thought I should post a question here in case the more tech-savvy guys can help.

I want to be able to run Photoshop CS6 and Lightroom 5 smoothly. What this means is to be able to handle large files (both in MP and in size). Imagine 36MP raw files from a D800 or 100MB tiff files from my MF analogue Mamiya (scanned of course).

I am no Photoshop expert at this point meaning that so far i have not gone above 4-5 layers when processing a shot but I plan on changing this. This in turn means that I could end up with 50 layers on Photoshop for one image one day. I need to be able to do this without problems on the above mentioned files.

So, as said my budget is somewhat limited so I would like to spend "where it counts".

1. RAM
2. CPU
3. Video RAM
4. GPU
5. HD speed

My current setup is an older Inter core 2 Duo at 3Ghz with 4GB or RAM and the cheapest NVidia g-card I could find that supported two screens. HD-wise nothing special, just a normal 500GB hard drive running @7200, unfortunately no SSD available. My questions:

1. Which of the above 1-5 do you believe is the most crucial?
2. On my setup what needs definite upgrading?
3. CPU, do more cores make a difference?
4. Is the message I am getting related to RAM or Video RAM?
5. Would an SSD make Photoshop and Lightroom run smoother?

Anything else you can think of much appreciated. As well as if you have any questions for me that would help identify the problem better pls let me know and I will revert asap.

Thanks!