Multiprocessor support in CS4

SS
Posted By
Steven_Scotten
Nov 3, 2008
Views
551
Replies
19
Status
Closed
I know this is a much more complicated topic, but I find myself surprised at how little CPU usage I get when doing things I believe to be purely CPU-intensive. Activity Monitor rarely reports Photoshop using more than 100% CPU on an 8-core Mac Pro.

I know it’s not realistic to expect things to run eight times as fast because I have eight cores, but I’m doing a rasterize operation that’s taken over twelve hours already. Every time I look at the CPU monitor and see only one processor being used, it makes me a little sad—never mind feeling stupid for spending the extra bucks on an 8-core Xeon.

Scratch disk is not my bottleneck—utilization on the (yes, dedicated) scratch volume has remained steady for the last couple hours. I still have 4GB of RAM reported free by Activity Monitor.

Plus, rasterization is the sort of process I would expect to be highly CPU intensive compared to disk and memory usage.

The file I’m rasterizing is pretty complex—over a half million endpoints. I don’t expect it to be quick. I’m just wondering if anyone knows about the state of multiple processor usage development in Photoshop.

How to Improve Photoshop Performance

Learn how to optimize Photoshop for maximum speed, troubleshoot common issues, and keep your projects organized so that you can work faster than ever before!

SW
Scott_Weichert
Nov 3, 2008
Snow Leopard (OS 10.6) is supposedly designed to allow the OS to make MUCH better use of multiple processors. I wonder if Photoshop is just doing all it can do with the current OS.
SS
Steven_Scotten
Nov 4, 2008
Some applications do seem to be written specifically to take advantage of multiple processors. I thought that Photoshop was one of those.

It’s a subject I wish I knew more about. I can only report what the little green bars in Activity Monitor tell me, then dream up wild, inaccurate guesses as to why or how.
WG
Welles_Goodrich
Nov 4, 2008
Steven,

Two resources which will help understand multiprocessing and the tangential subject multi-threading are here…

<http://en.wikipedia.org/wiki/Multi-threaded>
<http://en.wikipedia.org/wiki/Multiprocessing>

Leopard in conjunction with Xeon processors has very good support for multi-threaded apps written to take advantage of the capacity. I use several well written 3D apps which use all four cores of my 2006 Mac Pro at 100% when rendering. I believe many of Photoshop’s processes are multi-threaded simply becuase I can perform tasks on very large images and watch as Menu Meters indicates that all four cores are being utilized roughly equally.
P
PECourtejoie
Nov 4, 2008
For the story about Photoshop and multicore:

< http://blogs.adobe.com/jnack/2006/12/photoshop_and_multicore .html>
R
Ram
Nov 4, 2008
Thank you for digging that up, Pierre. I knew I had read it, but couldn’t remember where.
CC
Chris_Cox
Nov 4, 2008
Photoshop does make extensive use of multiple processors — but most things you might think are CPU intensive, really aren’t. Photoshop spends most of it’s time waiting for memory (RAM) and adding extra processors doesn’t help (or slows it down).

Some rasterization is threaded, and some is not. Some parts of rasterization (especially for EPS and PDF) are not good candidates for threading.
P
PECourtejoie
Nov 5, 2008
Chris, as you are here, there are discussions on a French Mac site about the fact that Photoshop CS4 does not use multiprocessors to compress when saving the images (Despite alleged Adobe claims, but let’s put this part aside???).
I suspect that it is an OS task, but I might be wrong.

Another user claims that Adobe (and Microsoft) refused to work with Cocoa, and demanded another technology (Carbon) to continue developping on OS X, and then dragged their feets to port Photoshop, instead of developping immediately on Cocoa.

I linked to several blog posts that outlined the immaturity of Xcode, the fact that even Apple does not use Cocoa 64 bits for FCP, the Finder, iTunes… That all the API were not present at the time, etc.
Also, what good would it be to wait without a valid reason?

I wondered if Carbon was not rather a safety net for Apple to make sure that others would envision developping for their new OS, rather than to throw in the towel; especially as Apple then was in the same shape than it is now…

What is your view on this, so that I can translate it, and if needed, correct the innacuraties of what is claimed there?
CC
Chris_Cox
Nov 5, 2008
Most types of compression don’t benefit much from threading (and some could be threaded, but get worse compression). We did some experiments with threading the compressors — best case was a 2% speedup, the worst case was a 15% slowdown, and the resulting size suffered.

And yes, most of the time when saving is waiting for the OS and your disk.

The claims about Carbon and Cocoa are missing a LOT of facts. Carbon was the only way Apple could make a business case for moving to OS X. Carbon was an evolution of the OS 9 APIs and made the port reasonable (still not easy, but reasonable). Carbon and Cocoa were supposed to live side by side, serving different audiences.

We adopt and ship new technologies when the technologies are ready (ie: we can’t ship without a compiler that works and an OS that works). We are generally aggressive in new technologies, when those technologies offer a benefit. If we appear to drag our feet – there’s usually a good reason (or a few hundred good reasons).
SS
Steven_Scotten
Nov 5, 2008
Chris–when you say "waiting for RAM" do you mean Photoshop is waiting for RAM to become available, or for scratch space to be allocated, or literally waiting on the throughput of the data bus and the six nanoseconds or so it takes to cycle?

I’m not being facetious, I get that there’s a lot of data to cycle through that memory. Even so, I’m wondering how a process that isn’t accessing the hard drive (not very often anyway; Activity Monitor says I’m sitting at 0 reads per second and 0 writes per second and my scratch drive hasn’t changed it’s available space) and hasn’t used up the RAM on my machine can be anything other than processor intensive. Even if it’s busy rearranging the memory it’s already allocated, that’s processor work.

I was under the assumption that a PostScript interpreter works on 2D data the way that a raytracer works on 3D data: by picking a pixel at a time and calculating the value of that pixel based on the formula provided by the Postscript data. I used to split raytracing tasks among multiple machines… this one renders the first 400 lines, this one renders the next 400 lines and so on. I had ignorantly assumed that the same sort of division would be possible with rendering PostScript.

I’m at about 48 hours in on this render and the progress bar is at about a third of the way across. I guess the real lesson here is that using Photoshop as a RIP is sort of like trying to hammer a nail with a putty knife. I should do some tests with GhostScript and consider investing in real RIP software.
AW
Allen_Wicks
Nov 5, 2008
I still have 4GB of RAM reported free by Activity Monitor.

Not what you should be looking at. Instead watch how many Page outs are being generated during rendering. If Page outs are increasing at a rapid rate more RAM would be of benefit.
SS
Steven_Scotten
Nov 5, 2008
Sorry, didn’t mention Page outs.

Page outs: 0 bytes. I’m trying again after a system lockup that forced a cold boot.

I didn’t mention it because I don’t remember ever seeing any increase in Page Outs when using Photoshop until *after* my free RAM drops under 1GB.

I opened a 16GB file and that generated almost 12MB of Page Outs after dropping my available RAM to about 380MB. My system only has 10GB of RAM. I’ll get more eventually, but for now I gotta make do.

As a side note, I tried it again without antialiasing. Took about 90 seconds instead of over 48 hours. Trying the run again with antialiasing on. It’s been about ten minutes so far, and I still haven’t seen Page Outs budge. I’ll keep checking back tho.
CC
Chris_Cox
Nov 6, 2008
If I say "waiting on RAM", I mean that the CPU is stuck waiting on something to come from RAM into the CPU, or out of the CPU into RAM. DRAM latency is a lot more than 6 nanoseconds. And you do not have to fill all available RAM to be limited by the speed of RAM.

Just because something is CPU intensive does not mean it can be threaded.

No, a PostScript interpreter is a lot more like a ZBuffer than a ray tracer — it’s a RIP. Raytracing is embarassingly parallel – each pixel can be independent. Area buffers (when threaded) either end up repeating a lot of work, or need coherency (work on large chunks) to maintain their performance.

48 hours on a PDF or EPS render? AntiAliasing shouldn’t add that much time.
SS
Steven_Scotten
Nov 6, 2008
48 hours on a PDF or EPS render? AntiAliasing shouldn’t add that much time.

I agree. It explains why I have so much time on my hands to think about stupid things like how many of my cores are being used… =^(

It’s a big job; the AI file is made up of about a half million bezier segments and I’m trying to render it at 50 inches by 50 inches at 600 ppi grayscale.

The Progress Bar seems to go slower the farther along it gets. It got about 5% across in about 10 minutes. After an hour and a half it got to about a tenth of the way. Now seven hours in it’s about a fifth of the way. Before it got shut down yesterday it wasn’t even halfway after 48 hours (I don’t remember… it was maybe a third of the way?).

Well, it’s progress anyhow. Fifteen years ago I could blow up a RIP with a 25K Illustrator file. Today it takes a 25MB file. =^)
P
PECourtejoie
Nov 6, 2008
Steven, one would think that the guys at San Jose would love to see your files, and study them to solve your problems!
SS
Steven_Scotten
Nov 6, 2008
Well, they (and anyone else) are welcome to this one.

<http://splicer.com/files/pictures/spiral_simplified.ai>

For now I’m pretty much set. I tried last night with the GIMP and this morning I had my image. It took somewhere between two and seven hours to render (can’t tell exactly, since I was asleep.) I can’t say that I care for the GIMP that much, but with Ghostscript built in, it was an easy alternative.

…..and I never saw GIMP use more than one processor either 😉
CC
Chris_Cox
Nov 6, 2008
… I was just about to ask for a copy of the file for testing.

Thank you!
SS
Steven_Scotten
Nov 6, 2008
Thank you! I’m interested to hear if you find anything of note.

GIMP just rasterized the whole thing to 1200ppi (60,000 x 60,000) in about five hours, with "high" antialiasing. Photoshop wouldn’t even attempt to open it at that size, so this is good for me, but the best thing is that I got to keep working in Photoshop the whole time. Now *that* is making use of multiple processors! 🙂
CC
Chris_Cox
Nov 7, 2008
We’re testing it now. Without antialiasing took about 8 minutes. With antialiasing… is still running.

Yeah, that might be a problem in the antialiasing code. I just don’t know why it shows up in your image and not on our other test images — or if it’ll even show up for other users. It’s probably some special case in the rasterizer that the rasterizer team will have to track down.

As for size – Photoshop should rasterize and work up to the document limit of 300,000 x 300,000 pixels. Of course, that might take longer….
SS
Steven_Scotten
Nov 7, 2008
Thanks Chris–

In the "Import PDF" dialog box, the natural size comes up: 50.149 x 50.143. If I change the resolution to 1200ppi (a little over 60,000 x 60,000 pixels) and hit "OK" I get an alert box: "A number between 0.001 and 26.666 is required. Closest value inserted." When I hit OK to that, I get 26.666 x 26.663 at 1200 ppi.

Unless there’s another way than File->Open to rasterize an AI file in PS, PS won’t let me build larger than 32,000 pixels on a side.

If this helps: the AI file was originally built by importing postscript code into AI CS4. That code was found here: < http://www.graphicdesignforum.com/forum/showpost.php?p=38781 3&postcount=7>

I updated some of the value definitions (number of windings of the spiral, width of the lines, distance between the lines) but didn’t otherwise change the code.

I manually corrected some of the curves in Illustrator, applied a "simplify path" on the outer parts of the spiral arms, then made 18 duplicates, adjusted the widths and the color, and saved.

If there’s any other information I can provide, please let me know. I appreciate your attention to my file.

How to Improve Photoshop Performance

Learn how to optimize Photoshop for maximum speed, troubleshoot common issues, and keep your projects organized so that you can work faster than ever before!

Related Discussion Topics

Nice and short text about related topics in discussion sections