Without going into why you are doing this, if you have a truly linear image, you should be able to use the Color Settings dialog box to make a gamma 1.0 working color space using the primaries from sRGB or whatever other space you want. Assign that new linear gamma profile and then convert to a gamma 2.2 profile. There will also be a loss converting from a gamma 2.2 to linear and back that can account for some loss, especially if you have add dithering option checked.
Peter,
Thanks for the response. Unfortunately I am a little lost trying work through your suggestion…
I opened the Color Settings dialog and basically did the following:
I turned off all the Color Management Policies, left the Coversion Options Engine set to Adobe (ACE) and switched the Intent to Absolute Colorimetric. I also turned off the three check boxes below that (Use Black Point Compensation, Use Dither, and Compensate for Scene-referred Profiles).
I then checked Blend RGB Colors Using Gamma… and set it to 1.0 based on your email.
Now I’m lost. I’m not sure what you mean by "Assign that new linear gamma profile and then convert to a gamma 2.2 profile"
Could you elaborate?
As to why I am doing this, I work for a visual effects studio and our 3D rendering system expects all input textures to be in linear space (no baked in gammas or color corrections). All our outputs (from the renderer) are then viewed through proprietary viewing tools that apply a film-look to the image. The underlying image is still in what we are calling linear space.
One type of film-look is gamma 2.2 for shows going direct to video – which is what I am testing at the moment. For this reason I need to generate textures that have no gamma encoding to the pixels – which means any reference images we have typically need to be converted with a gamma .454.
My issue seems to be that PS does its color correction (or at least levels and curves) in something other than 16 bits. As best as I can tell, it is using an 8 bit representation of the image when doing these adjustments – though that seems really surprising to me and there must be something that I am doing wrong.
Thanks again for your help so far.
While we are on the topic of color and bit depth, can anyone tell me how to get PS to open a 32 bit image without doing what seems to be a creative interpretation of the black and white points? I’ve tried changing the settings in the color settings dialog to be similar to what I mentioned in the above post with no luck.
For my purposes, I need PS to accept that black is 0 and that white is 1 and that the image is linear.
Thanks!
Ben
"I turned off all the Color Management Policies, left the Coversion Options Engine set to Adobe (ACE) and switched the Intent to Absolute Colorimetric. I also turned off the three check boxes below that (Use Black Point Compensation, Use Dither, and Compensate for Scene-referred Profiles). "
I’d leave CM on, as you really can’t turn it off anyway. Set rendering intent to Relative, which is the same as Absolute except that it maps white to white. BPC probably has no effect on these types of profile conversions, but it’s generally a good idea to keep it checked – unless it’s causing you a problem.
"I then checked Blend RGB Colors Using Gamma… and set it to 1.0 based on your email."
This is not the same as using a 1.0 gamma image color space. It affects the way some filters blend.
"Now I’m lost. I’m not sure what you mean by "Assign that new linear gamma profile and then convert to a gamma 2.2 profile"
Could you elaborate? "
You need to open up Color Settings in Ps and make sure that the advanced options checkbox is checked. Then in the RGB Working Space dropdown menu, click and hold that and scroll UP to an option called Custom RGB. It will automatically open with the coordinates for whatever RGB space was loaded in Color Settings at the time. Now you’ll be able to choose a different gamma and even primary coordinates if you wish. Name it something that makes sense to you and click OK. That new altered space will now be loaded. Now immediately click the dropdown and choose the Save RGB option, which will now save your new gamma 1.0 color space in your ColorSync profiles folder. It can now be accessed like any other profile you would use.
Now what you do is open the image in question, and use the Ps Assign and Convert dialogs to manipulate the color spaces as needed.
"As to why I am doing this, I work for a visual effects studio and our 3D rendering system expects all input textures to be in linear space (no baked in gammas or color corrections). All our outputs (from the renderer) are then viewed through proprietary viewing tools that apply a film-look to the image. The underlying image is still in what we are calling linear space. "
That makes sense.
"One type of film-look is gamma 2.2 for shows going direct to video – which is what I am testing at the moment. For this reason I need to generate textures that have no gamma encoding to the pixels – which means any reference images we have typically need to be converted with a gamma .454. "
You should have the tools available now to do what you need.
"My issue seems to be that PS does its color correction (or at least levels and curves) in something other than 16 bits. "
I’m not sure anyone outside of Adobe knows the exact math involved, but there has been some discussion about what referred to as 16 bit per channel really being 15 bit +1 internally, while some calculations (I seem to remember it being mode changes) being computed using 20 bit math.
"As best as I can tell, it is using an 8 bit representation of the image when doing these adjustments – though that seems really surprising to me and there must be something that I am doing wrong. "
You’re always using an 8 bit representation to view the image. The actual tools will work in higher bit precision but the data has to be higher bit to begin with, not 8 bit converted to 16. You can set the Info Palette to read in 16 bit data, but it just confuses most people as we’re so used to reading everything from zero to 255.
Read Peter’s last paragraph until you have memorized it. A jpg isn’t even really 8 bit to start with.
I wonder if there’s a simple way to make or fake a 2.2 gamma adjustment in ACR…
Peter,
Thank you so much. That was a very thorough explanation and it really helps me moving along the path to solving this issue. I have to do some studying up on color management before I can comfortably apply this info, but you have given me a lot to start with.
My understanding of color-spaces is very limited due to the nature of my job (source images generated by computer – and therefore quite unambiguous – and a single output to film). Now I find that I have to educate myself a little more thoroughly if I want to bring PS into that world (it makes a bunch of assumptions that I need to at the very minimum understand and quite possibly override).
As to my original problem of not reproducing the "quality" of my conversions in PS to match those in Shake… some additional investigation shows that the information is actually in the file, but that PS is displaying it (or possibly interpreting it during load) in a fundamentally different way than Shake. Again, this is a call for me to educate myself more on the science behind PS’s color management.
I think I already knew that PS displays 8 bits even when the underlying data was a higher bit depth, but reading it here confirms it. Even though my original jpg image was 8-bits to start with, when I promote it to floating point prior to applying the gamma .454 curve in shake, I *think* I effectively shift the color values into a range beyond what the original 8-bit image could handle. I still have at most 256 discrete steps per channel, but the actual values (assuming the file is still linear) would have fallen "between" the available values in the original 8 bit file. This, I think, makes the image a true 16-bit image even though the actual color fidelity of the data has not been improved.
@Lundber02,
jpg’s are not 8-bit? Are they 7 bits and a signed integer? I rarely work with them unless I have reference from a digital camera. Can you elaborate?
Thanks again to everyone who responded… I feel like I was exploring a one room apartment and just stumbled on a little door that opens to a mansion. I have a lot to still figure out.
some additional investigation shows that the information is actually in the file, but that PS is displaying it (or possibly interpreting it during load) in a fundamentally different way than Shake.
I think what you are experiencing is the simple fact that L* in Photoshop is not linear in the LAB reference tables and Shake is. The reason why Adobe has done this is to make Relative Colormetric function with more natural "looking" pixel transformations, but it’s becoming apparent that it will need to be changed in future versions of PS.
See if you can use ACR to do what you want with regards to color space transformations because it does use linear gamma in its L* tables.
Oh, and one more thing… as I start to figure this out I may post back here from time to time so that anyone else who is in my same situation might have some points of reference.
jpgs use local pixel averaging and throw away information.
You can not have a "true 16 bit image if you start with an 8 bit. That should be self evident.
It may sound odd but it has been shown that Up-Bitting to 16 bits (from an existing 8-bits) can result in far smoother gradient Masks than you can get on your original 8-bit file.
@Lundberg02:
I agree that the images I create using shake (going from 8 bits to float, performing a gamma .454 move, and then down sampling to 16 bits) are not "true" 16 bit images in the sense that I have gained any color fidelity (I haven’t). What is true, I think, is that in performing these actions I have an image that can no longer be accurately represented in a standard 8-bit linear file format (even going from float to 16 bits degrades the image even though I had originally started with an 8-bit image).
Could you explain what you mean by local pixel averaging? Are you referring to the compression? Like I said I’m fairly new to using jpg file formats and am trying to wrap my head around them.
@Mike:
Thanks for the hint. I have to learn about L* tables now but it gives me something to work with. I also just started reading "Real World Color Management" by Bruce Fraser, Chris Murphy, and Fred Bunting. I’ve actually owned it for a while but am only now diving into it. Hopefully it will help.
Ok. I googled jpeg and found the following wiki article (which seems fairly comprehensive):
<
http://en.wikipedia.org/wiki/JPEG#Color_space_transformation>
There is a mention that the jpeg standard suggests converting the rgb values in YCbCr which *may* result in some image degradation (I don’t know because we are getting way out of my area of expertise here). Alas there is no mention of "local pixel averaging". As best as I can tell, local pixel averaging looks at a certain number of adjacent pixels and then generates a single value based on some algorithm or another. I don’t *think* that this would reduce the bit depth of an image in that there are still, in an 8 bit image, 256 discrete values per channel. Whatever algorithm is being used would seem to still have access to all of these values as it averaged (or whatever) the pixels together.
Ultimately, what matters to me is that a jpeg image can be treated as though it were a true 8-bit color file format and I think that this is still the case.
By the way, I’ve started reading that "Real World Color Management" book and so far it is excellent. Highly recommended if you are new to color management but feel you want a truly in depth presentation.
Ok, so I have dug into this a bit more (along with the excellent help of "Real World Color Management") and I am a bit (though not much) closer to figuring some of this stuff out. I’m posting here in case this information is useful to anyone else trying to work through color management in a similar way to us. Plus, the act of typing it out helps me kind of cement it in my own mind, so there. 🙂
Please keep in mind that I am working this out more or less on my own and so what I describe here could well be completely wrong…
In our workflow, rgb is absolute. I.e. a red pixel value of, say, .7 is always the same regardless of the source. There is no need for any sort of an input profile because we are not dealing with any analog to digital conversions here. The images we deal with come from one of two sources… 1) a cg renderer in which there is no analog to digital conversion or 2) a background "plate" image which has been already color corrected and is saved with its rgb values baked into their "correct" settings (this step may quite well have used an input profile, but it occurs before our color pipeline gets a hold of it). In either case, for our workflow, the rgb values themselves are sacrosanct, and should never ever be modified or displayed differently unless we specifically act to modify them.
Photoshop, I believe, operates under the assumption that the rgb values coming into the app are *not* absolute, but are instead the result of an uncorrected analog to digital conversion (i.e. a digital camera or a scanner, etc.). Consequently, it attempts to address the inherent ambiguity in the supplied rbg values by applying input profiles to them. In this way PS hopes to shift the individual rgb values closer to what the "original" color is supposed to be. This makes sense for the majority of cases, but unfortunately is not usable in our workflow at all.
In both cases (our workflow and the PS color management workflow) there is a profiled, calibrated display AND an output profile designed to mimic the actual output media (film in our case, print for many PS users).
So, now my job is to figure out how to use the color management system in PS to mimic our simplified view of color. Basically, PS should do nothing to the incoming image whatsoever, and only use an output profile to mimic whatever media we are targeting. And that’s as far as I have gotten. I have played around with ACR but can’t seem to figure it out well enough to determine whether I can "turn off" any input profiling. I’ll keep exploring and report back whatever I can.
it has been shown that Up-Bitting to 16 bits (from an existing 8-bits) can result in far smoother gradient Masks
Of course but that is only because the masks are made in 16bit. up sampling does nothing for the image.
Bernard,
"In our workflow, rgb is absolute. I.e. a red pixel value of, say, .7 is always the same regardless of the source. There is no need for any sort of an input profile because we are not dealing with any analog to digital conversions here."
RGB is never absolute. It HAS to have a reference. You have to know, when you specify a pixel value, what context it’s being referred to. You don’t need an analog to digital conversion either. When you’re specifying pixel values for your film recorder, you need to know that a specific red is made from a specific mix of red, green and blue values are going to make that color. You can either do that by trial and error or you can do it by using lookup tables or profiles to define the "language" of your output device.
"So, now my job is to figure out how to use the color management system in PS to mimic our simplified view of color. Basically, PS should do nothing to the incoming image whatsoever, and only use an output profile to mimic whatever media we are targeting. And that’s as far as I have gotten"
If I were you I would profile your film recorder and then you would have an ICC profile that described the characteristics of your output device, and you would be able to use that in Ps to accurately target color generated in Ps for your specific output. I’ve only profiled a couple film recorders and the 35mm recorders are a pain in the ass, but they are possible to profile, and as long as the process of exposure and development remain stable, the profile works great.
Peter,
Thanks for the quick response. I think I may have phrased it badly when I said that for us rgb is absolute. What I should have said was that any rgb value coming from any source is treated identically. This means, regardless of origin, a red pixel value of .7 will always be considered identical to any other file with a red pixel value of .7. We sort of live in a mythical world where all of our "color capture devices" (cg renders, digital files that have been color corrected prior to coming to us) work identically to each other. In other words, it is as though we had a single input profile that we use for every single file that comes through our color pipeline. (I should add, that this is all to the best of my knowledge).
The other side of the equation – what a red pixel value of .7 means in our output devices – You are absolutely correct (and I should have been more clear about) that this is most definitely not absolute. For this we have a fairly complicated color pipeline (in Shake and some other tools, but not PS) that does some wacky 3D transforms on our image files to present them to us in a manner that simulates various film stocks. It’s all a bit over my head but luckily there are some people here much much smarter than I who are handling that end of the pipeline. We have also contracted out the creation of several ICC profiles that tries to simulate this same color pipeline in PS.
So what’s the issue? Well, I’m quite sure you’re sorry you asked, but here it is 🙂
I cannot reproduce in PS what I see in Shake.
As far as I understand it, Shake takes the raw rgb values in a linear file and then applies our LUT to them which simulates our film-look (all of which is displayed on a calibrated and profiled monitor). In order to simulate this under PS I have to be sure that the source file is linear and then apply the "same" LUT in the form of one of our ICC profiles. Linearizing these files, though, is where things start to fall apart. Let’s say I take an 8-bit file (with no assigned profile) into Shake, promote it to float, apply a gamma of .454, and then drop it back to 16 bits and save it out. I now have a 16 bit linear file (assuming the original had a baked gamma 2.2 – a safe assumption for many of the files we deal with). In shake I can open this file up and apply a gamma of 2.2 to it and, visually, see that it looks identical to the original. In PS if I open this file and apply a gamma 2.2 to it (via levels – changing the central value to 2.2) I get an image that is considerably darker with some very crushed blacks. If I play with the levels control I can see that the detail is still in these areas, but somehow being displayed differently. This is what originally led me to the whole idea of input profiles and wondering whether PS was interpreting this 16 bit linear file in some other way. That, in turn, has led me to learn more about color management and so far, it has been a fair bit of fun (I am a such a nerd). Thanks for your help.
But this may not be a color profile issue. Even in a single document I cannot seem to go from gamma 2.2 to linear and back. To simplify, try this.
1) Open a jpg file in PS (I’m particular to: <
http://adorablay.files.wordpress.com/2007/02/cat-hat.jpg)>
2) change the bit depth to 16 bits
3) apply a levels adjustment layer using a gamma of .454
4) apply a second levels adjustment layer using a gamma of 2.2
the resulting image is nothing like the original image (if you actually did this on the file I listed above, then look at the reflections in the eyes in particular). I would have expected that, visually, the resulting image would have been almost identical to the original. Especially since I did the gamma adjustments in 16 bits (in Shake I can even do this trick in 8 bits without any serious loss of visual quality). So maybe it isn’t a color issue… but I cannot figure out why I can’t do a round-trip like this.
Sorry for the extremely long email but it helps me to learn to type things out long-hand, plus I’d be really interested if anyone has any insight into the last bit here.
Thanks!
Ben
I wonder if using levels is part of your problem, as the curve it applies with the midtone slider actually has invisible clampdown points to help preserve both highlight and shadow detail, as opposed to a true gamma curve.
LUT in the form of one of our ICC profiles. Linearizing these files, though, is where things start to fall apart. Let’s say I take an 8-bit file (with no assigned profile) into Shake, promote it to float, apply a gamma of .454, and then drop it back to 16 bits and save it out. I now have a 16 bit linear file (assuming the original had a baked gamma 2.2
Your Shake system HAS to have some sort of color space assignment to your image – PERIOD. Your Shake system shows you the correct preview on the monitor and assigns a color space correctly for that preview and if it converts the image using that profile or something else is what is unknown and you have to find out how your Shake system actually functions before you can get Photoshop set up correctly to emulate another system.
Also – How do you know that you are getting linear files supplied to you after your assumed color correction is done by an outside source? Is there an industry standard for your Shake system?
Without knowing the inner workings of said system – it’s difficult to advice on how to set up Photoshop to emulate.
Sounds like Peter has the answer, but Mike is also correct.
Peter,
That makes some sense to me. I’ll try to build a gamma curve using curves (though that may have similar issues??). One interesting thing, if I do the test that I mentioned above (8-bit image with two levels adjustment layers above it) in 32 bit space (i.e. convert the image to floating point space) I can do a complete round trip. In all likelihood the math behind levels is significantly different under float than 16-bit, so that may explain it.
What I’d like to do is apply a true gamma 2.2 curve to the image and see if I can get back to the original look while still having the underlying data be "linear". Do you know if there is such a thing as a simple gamma 2.2 ICC profile that I can download? Or is such a thing not possible?
Mike,
You are quite correct. Shake does have to have some sort of a baseline to determine what actual color any particular combination of rgb is. Unfortunately I am a bit lost trying to figure out just what that is. I do know it is the same regardless of the input source, so if I can figure it out I should in theory be able to duplicate it under PS since PS has such a sophisticated color management system.
As to knowing whether there is an industry standard… I suspect there is, and that it is the same "profile" that I am trying to track down to emulate in PS. Once our color guys are back from siggraph, I’ll try to pick their brains again and report back (they know our color pipeline, but not PS so much).
Thanks to everyone for their help so far. Once I learn more I’ll post back here.
Bernhard –
You need to find out if
A. Shake’s white point is the same as the color space you are assigning in Photoshop.
B. Confirm that Shake has TRUE linear gamma. If this is correct – this will create a bunch of work for you to make Photoshop match it.
Ok, a little more info.
We have a proprietary image viewing app here and when I open an 8-bit jpg using that tool, it looks exactly the same as when opened in Shake (and in Preview.app incidentally). I was able to talk to the developer and she told me that all the viewing tool is doing, color-wise, is sending the raw rgb values in the file directly to the frame buffer (via an openGL system call). I’m assuming that the actual values shown are affected by the display profile running on the OS.
This same tool can optionally use our color management system which uses the display profile and a target profile to create a 3D LUT and passes the rgb values through that. Even in this case, the raw rgb values are still being used as is (i.e. the only profiles being used are the target profile for a particular film stock and the display profile for the particular monitor/video card connected to the workstation). In other words, there is no input profile per se. I assume there are a whole bunch of assumptions that underlie this model, but they are the same assumptions each time.
So, now I am going to try to figure out how to get PS to mimic this workflow. I think Shake (and our other tools) just uses the monitor’s whitepoint (if we turn on our viewing LUT – soft proofing – that is changed, but Shake itself does not set a custom white point I don’t think). As to whether Shake has a true linear gamma, I’m not sure I understand what a "linear" gamma is. Is it a mathematically pure gamma curve? More research to do and more mind-numbingly long posts to come.
Aha.
I have found what I needed (so far). The following web site has a decent description of the issue I have been having using levels to try and do a "round trip" from gamma 2.2 to linear to gamma 2.2 again.
<
http://www.aim-dtp.net/aim/evaluation/gamma_error/index.htm>
I’m still trying to understand the actual reasons for it (though I suspect it is what Peter and Mike referred to in their earlier posts with the clamp down points in the curve underlying levels and the L* curve not being linear <- though I am still struggling to understand how PS utilizes L*).
Here is a graph of the different operations in PS and the resulting curves (from that same site)
<
http://www.aim-dtp.net/aim/photoshop/v6/levels_dialog/levels -errors.gif>
In any event, this site has a downloadable set of curves that are for a huge range of different gammas. Using these curves, I can do a complete round trip with no visible loss in image quality.
I have also discovered that if I specify the Monitor RGB profile I get a result that matches my other tools (which makes sense, as I suspect this results in a "unity" translation as it is coming from the same color space as it is going out to).
The final thing I need to figure out now is how to get the gamma curve into an ICC profile (do I need to take the Monitor RGB into account?) so that I can see my image in gamma 2.2 but have the underlying pixels live in linear space. Once I figure that out, I will post here again.
In the mean time, thanks so so much to everyone who answered my many questions (and especially Peter and Mike). It has been extremely educational.
For ———– sake , learn what the Monitor profile is.
<
http://www.brucelindbloom.com/>
Go to the calculator tab on Bruce’s site and look at L* axis of the LAB Munsell model. See how L* is represented as a straight lumance value? Well, Photoshop’s lumance is not perfectly linear so when you do color space transformations – you are getting slightly off values relative to your Shake transformations.
OR better said – Your working space may be slightly off lumance in Photoshop.
The site is a really good reference for getting really geeky techno info in your research projects.
Thanks , Mike. Didn’t know about the Bradford. Chris Cox probably has a pair of glasses with the Bradford built in.
Mike,
Thanks for the links. I checked out the bruce lindbloom site and it is very cool. Over my head for the moment, but I can at least see what you mean by the straight luminance value.
I’ve also downloaded the first of the ICC whitepapers on the site you suggested. It’ll be a bit of a slog, but now I’m in deep enough that I want to learn more. I’m going to work my way through as many of these papers as I can (though there will be a point where I just hit the limits of my mathematical abilities I think). It really is a case of how deep does this rabbit hole go? Deep, I think.
Lundberg,
Both Mike and Peter have taken their valuable time and, with nobody forcing them, helped a complete color newbie out as he struggled to figure out a color problem. They answered his dumb questions and guided him (and inspired him) to learn more about color. They didn’t have to, but they stepped up to the plate anyway. Even as I continue to work to learn more regarding the details they have given me, I am very very appreciative.
You act as though I somehow came into your office and forced you to read through my dumb questions. I’m terribly sorry if I offended you by doing that.
So, since you were so vociferous in suggesting that I learn what a Monitor Profile is, perhaps you could take a look at the following explanation of how I think a linear workflow would work (including the use of monitor profiles) and let me know (in polite, constructive language) if I have it right or not. Of course, you are under no obligation to do so, but here is a really good chance to actually contribute.
1) At my firm, our entire color workflow – using tools other than PS – assumes that rgb values in images are sent directly to the computer’s frame buffer. It also assumes that these images are linear. This is the workflow I am attempting to emulate. Whether this is a good workflow or not is not relevant. It is what we use.
2) In PS, by setting my working space to my Monitor Profile space I am essentially doing a unity transform (assuming no losses from the conversion to and from the pcs <- still learning about this process). I.e. any rgb value coming in from the source file will be translated into the exact same value before being sent to the computer’s frame buffer.
3) Ideally, I’d like to paint in gamma 2.2 space (for example) while leaving the underlying data in a linear format. For this I would need to have an additional gamma curve (2.2 or .454 <- I get confused as to which one, but it’s easy to test). Since I cannot apply two ICC profiles, I would need a single profile that somehow combined both my Monitor Profile *and* a gamma 2.2 curve. This is what I was alluding to in my earlier post.
Did I get it right?
Well, that was to be expected. I fed the troll and I got what I deserved (should have checked his other comments on this forum first, and I would have learned what he really was). My mistake. 🙂
Thanks again to everyone who knew what they were talking about. Peter and Mike especially. You’ve inspired me to delve a lot deeper into this subject than I ever thought I would or could.
And with that, I think this thread is done.
Anytime Bernhard…
I hope you can create some sort of workflow that will allow you to accomplish what you need.
take care.
mo
Read Real World Color Management and then you will understand the short answer. Until then, don’t judge me.
