Coffee Science for CoffeePreneurs by CoffeeMind

Why Rate of Rise is a bad reference point for optimizing flavour in coffee roasting

• Morten • Episode 17

Share episode

As a consultant and trainer, I’m constantly confronted with the concept of Rate of Rise as a superior reference point for improving roast profiles. With our scientific background and strong commitment to only use concepts that are explicitly helpful for coffee roasters to create successful products for customers we have to speak up about the misleading nature of how Rate of Rise often is obsessed by by students all over the world. It is the single most discussed concept that I have to handle during all my courses and dismantle in my students minds before we can get to the userful concepts that actually makes a difference in their lives and are actionable from a business strategic perspective. The content of this blogpost is the result of discussion this with hundreds and hundreds of students and clients so I think it is relevant and useful to make a separate blogpost about it even though it was also dealt with from many different angles in our podcast series about Coffee Science.  It will show you how our thoughts are not just ‘our opinion’ but deeply embedded in the scientific tradition all the way back to Plato and through the centuries of refinement and improvement of scientific thinking! Where the science podcast series was structured from the point of view of science this article is a good example of how science can be used practically on a single technical subject that has caused quite a lot of confusion. This article is a condensation of the approach I have when the subject of Rate of Rise is discussed during courses and consultancy sessions. To save time and get this subject over with before these sessions we will systematically refer students and clients to this podcast before the event. 

Links

See the blogpost with illustrations here:
https://coffee-mind.com/why-rate-of-rise-is-a-bad-reference-point/

Or see it with video on Youtube here:
https://youtu.be/zrscIUhii-U

Why Rate of Rise is a bad reference point for optimizing flavour in coffee roasting

This text is simultaneously a podcast episode and blog post on our website and I will refer to it as ‘the article’ and you can find the links in the show notes and in text context in the blogpost where several illustrations are supporting the text, so if you listen to this as a podcast please also visit the blogpost to deepen your understanding of the content.

If Rate of Rise (RoR) is a concept unfamiliar to you I would recommend you to read this blogpost first: Roast profile analysis

I’m aware that this subject is a bit controversial as it seems that many educators are operating as if this is best practice for education and product development. As a consultant and trainer, I’m constantly confronted with the concept of Rate of Rise as a superior reference point for improving roast profiles. With my scientific background and strong commitment to only use concepts that are explicitly helpful for coffee roasters to create successful products for customers I have to speak up about the misleading nature of how Rate of Rise often is obsessed by by students all over the world. It is the single most discussed concept that I have to handle during all my courses and dismantle in my students minds before we can get to the userful concepts that actually makes a difference in their lives and are actionable from a business strategic perspective. The last 5 years I have experienced this all over the world (from the US, Europe, Golf region, Asia and Australia) so the content of this blogpost is the result of discussion this with hundreds and hundreds of students and clients so I think it is relevant and useful to make a separate blogpost about it even though it was also dealt with from many different angles in our podcast series about Coffee Science. If you have not heard the podcast yet, please find it now also on our website.  It will show you how our thoughts are not just ‘our opinion’ but deeply embedded in the scientific tradition all the way back to Plato and through the centuries of refinement and improvement of scientific thinking! Where the science podcast series was structured from the point of view of science this article is a good example of how science can be used practically on a single technical subject that has caused quite a lot of confusion. This article is a condensation of the approach I have when the subject of Rate of Rise is discussed during courses and consultancy sessions. To save time and get this subject over with before these sessions we will systematically refer students and clients to this article before the event. 

Of course the following points makes sense only in the light of the overall approach we have in CoffeeMind to roasting education. I do acknowledge all my colleagues out there have different experiences and different approaches and that some of our concepts only makes sense in our context. Also of course I’m aware that I’m not right all the time, but the identity of CoffeeMind is to be the ones asking the question: “Wait a minute… does this make sense at all?”. Sometimes it seems that the general global education community (and universities as well) is more eager to add more opinions, approaches and complicated concepts than to weed out all the old nonsense that has been piled up in the global coffee roasting community over the years. 

So here you have it: The following article is a critical approach to the concept of using Rate of Rise as a quality indicator in coffee roasting given CoffeeMind’s particular approach to teaching coffee roasting which as been developed since 2005 through teaching more than 1600 individuals in small groups of 6-8 and conducting more than 30 research projects of which 12 are now scientifically published and consulting more than 30 small and bigger coffee roasting businesses around the world since 2007. That does not make us right in everything we say. I’m just mentioning our experience background and the reason we are so deeply embedded in and passionate about the scientific implications for coffee roasting education. So in the following please think less about who is right or who is wrong but which concepts make sense in themselves and why in the light of the purpose behind using the concepts in the first place. Hopefully this article and the derived discussions will bring us all forward (including CoffeeMind and the blind spots we have that we still haven’t realized) where the ‘who’ question is a waste of everybodys time.

As the beginning it seems suitable to start with the question:

Where is the concept of Rate of Rise useful in coffee roasting?

The Rate of Rise can be used

  1. for predicting where the roast is heading just like the speedometer on your car.  As it represents the fundamental speed of the process at any given moment you can use it in your car to estimate if you will be early, late or on time for the meeting that you are heading for. Similarly you can use the Rate of Rise to see if you will hit a waypoint that you were aiming for during a roast such as first crack or if you are roasting faster and slower than initially planned and therefore deviate from your plan. 
  2. to ‘flavor categorize’ your different roast profiles. Even though you already have the information just specifying the development time and end color of a roast, which would be the most aligned with Ocham’s Razor, I think it could make sense to quantify the average Rate of Rise after first crack and use this number to categorize you different profiles. This number is highly machine dependent so you can only use it to compare your roasts on your machine. This way of categorizing your different roasts is aligned with our research which has shown that a shorter development time (with a higher Rate of Rise) will give you a more fruity and acidic flavor and a longer development time (will have a lower Rate of Rise) gives you less fruity and less acidic and more chocolate nutty flavor relative to the short development time. This was demonstrated in our scientific paper on the subject, which is available for free on the journal website with the catchphrase title: “The Effect of Roast Development Time Modulations on the Sensory Profile and Chemical Composition of the Coffee Brew as Measured by NMR and DHS-GC–MS”. 


So my point is not that the concept does not make sense in itself and indeed I consider it one of the most important and useful concepts in mathematics (see Differential calculus). But aligned with my model for a good theory it is dangerous (and highly confusing) not to restrict yourself to using the simplest possible concepts with the fewest assumptions (Aristotle and Ochams Razor) explicitly relevant (what I call ‘Form Follows Function’ in the Podcast) for what you are trying to do. It is dangerous (confusing, decision inhibiting and time wasting) to use an overcomplicated and highly assumption burdened concept by default.

Constantly declining Rate of Rise is a natural behavior in a roaster anyway

Even if you kept the flame setting constant throughout the roast the system would show a pretty stable declining Rate of Rise with very few fluctuations on the curve. The reason being that the constant flame with a constant airflow would at some point reach a steady state ending in a pretty constant temperature of the air going into the roasting beans. And since temperature always goes from a hot substance to a colder substance with the speed of energy exchange proportional to the temperature difference between the two substances. Said plainly the hot air heats up the colder beans and as the air succeeds heating up the beans the temperature difference between the air and the beans declines and hence the heating of the beans slows down which gives you a progressively slower roast leading to a constantly declining Rate of Rise. With a few fluctuations. And since our data shows that small fluctuations leading to small differences between roasts can’t be detected even by coffee professionals it seems that the focus on the small fluctuations of the RoR curve is below sensory threshold even for professionals (please see these sensory data presented in this passage in this YouTube webinar).


Unfortunate practical consequences of using Rate of Rise as a reference point to optimized Rost profiles.

Really, what science says and does does not matter if it does not have any practical implications for daily life and decision making as a coffee roaster! The only reason we are so passionate about science is that  - if applied correctly - it has deep, liberating and purpose fulfilling implications for how you make decisions in the light of the purpose of your organization and therefore your life (because your life quality depends on your success or not to make your organization a picture of your passion and dreams. Too many are struggling with making their organization aligned with their deepest dreams and struggling with clumsy concepts not aligning with the overall purpose of the organization is NOT helping here!). So first I would like to point out some practical implications.


Improvising with the gas settings during roasting to keep it on track

People who use Rate of Rise as a reference point for optimizing roast profiles are often seen improvising with the gas settings during roasting to smoothen the RoR curve which is very unfortunate because it shows that they don't understand the fundamental thermodynamics of the machine in the first place. If they did understand the thermodynamics of the machine they didn't have to improvise during roasting as they would just execute the plan they have developed during experimentation that provides deep knowledge of the thermodynamics of their own machine. In CoffeeMind’s way of teaching this is a step you learn during the “Control” step in our competency roadmap (see the below illustration) which you can do in the online e-learning course Roast Profile Design Basics for only €30. 


Improvising with the gas setting during roasting is not only a sign that you don't understand the thermodynamics of your roaster, it also has the unfortunate consequence of compensating too late for a bad decision made earlier during the roast. If you realize that your roast deviates from your plan you should have made an adjustment at least a minute or two earlier to avoid it and compensating after the fact doesn't really help and will not give you the same result as if you did the reduction earlier. This is why we in the CoffeeMind roasting training tell people to NEVER IMPROVISE but follow the iterative and circular process of ‘Plan-Execute-Learn’. 


If you don’t follow the bad plan to the bitter end you don’t know the magnitude of error in the plan which is the basis and the specific material for your learning so that you can improve the next plan for the next roast until you have created a rock solid and simple plan for each of your profiles. If you know the thermodynamics of the machine you always get the result you set out to do. If you don’t the deviations are probably too small to taste (if you keep the color the same!) in which case micromanaging the flame wastes your time without giving you a different result. If you overcompensate with the flame you might burn your beans and neither of these cases lead to a better result. Remember that small variations in the roast profile typically cannot be detected by anybody, not even skilled coffee professionals as long as the color of the coffee is the same.

It doesn't help you predict customer preferences because it advocates and assumes that there's one optimum


With lack of specific predictions of flavor you can’t use it for product development

Another big problem is that the theory doesn't offer a clear intensity model in between the roast profile data and the consumer so even if it is preferred it's not clear why it is preferred from a flavor intensity perspective.


Difficult to compare and translate profiles between machines

From an education and a consultancy point of view the Rate of Rise is difficult to compare between machines as you get very different shapes of RoR curves due to probe type and placement where simple and fundamental Concepts like Time, Temperature and Color are more easily transferable from machine to machine. Of course you still can’t transfer roast 1:1 from roaster to roaster but using more fundamental concepts you are very quickly close and final tweaks can be made based on experimentation and sensory analysis.

Conceptually

As a theory the claim of the superiority of a constantly declining Rate of Rise is not well structured. At the heart of a theory is the notion of cause and effect and in research design it is called input parameters and output parameters. 


Cause   | →  | Effect
Input  | →  | Output
Independent variable | →  | Dependent variable


Simplicity is king

In a good theory it's very important that both the input parameters and output parameters are resting on the simplest possible Concepts with fewest assumptions such as good old Time, Temperature, Energy, Color. 


We already have the data!

Sticking with simplicity it is striking that Rate of Rise is not a deep separate piece of information in a roast curve since it is just a quantification of the shape of the bean temperature curve that we see already! So why make such a fuss about it? Looking at the curves it is also striking that the big jumps we see on the Rate of Rise curve is a magnification of the small bumps on the bean temperature curve. In itself the bumps on the bean temperature curve does not look major nor alarming but when making the calculations the noise of the signal on the bean probe is amplified combined with the noise created by the different complicated computer calculations involved with calculating the Rate of Rise live while roasting. This lends itself to overestimation of small fluctuations just because it is visually bigger on the Rate of Rise curve. You don’t taste the curve - you taste the coffee! And only if big impacts are applied to the coffee does it show up in the sensory properties of the coffee at all. It’s like putting a magnification glass on a small spider: it does NOT make it more dangerous!


 


The input variable

Even though RoR is a derived value of Temperature and Time there are too many unfounded assumptions about using it as a rock solid ‘Cause’ / ’Input’ / ’Independent variable’ in a theory. The temperature that a probe shows is a bit of an artifact already as it does not show ‘the real’ temperature of the coffee anyway and for me it is primarily a relative reading telling us how much energy we have added where points like 1st crack and color are the absolute indicator of how much energy has been added to the coffee. So putting too much emphasis on the shape of the progression of a relative and artifact burdened reading such as the bean temperature is an extremely flimsy parameter to use as ‘Cause’ / ’Input’ / ’Independent variable’ in a theory. It is simply not a theory resting on solid ground from the get go.


The output variable

And since the theory of the superiority of constantly declining Rate of Rise has no specifically defined flavor consequences defined from a sensory perspective it seems like also the output parameters of the theory is also too vague to provide a good basis for a theory. An example of a specific outcome prediction of a theory could be if it predicts if a roast sample becomes less or more acidic or less or more bitter.

The underlying Concepts for a good Theory should be the simplest possible and most specific concepts with the fewest assumptions otherwise it's difficult to be specific and therefore practical in the use of the theory. As I also mention in the podcast (aligned with Popper’s Critical Rationalism) a theory that does not predict something specifically in the first place can survive for a long time since it can never be caught in being wrong, because it never predicted anything specific in the first place.

Looking at the causal chain of events during a roaster the heat source (1) is the first cause of everything as coffee roasting is about applying heat, so this parameter is number one. The flame heats up the medium between (2) the heat source and the beans. And the beans is heated up as the 3rd link in this chain. The Rate of Rise of the beans is a derived value of the 3rd link in the chain coupled with all the artefact of bean probes in general so the probability this value will give us deep insight or deep intuition about the system seem a bit far out.


Linear variables

A good visualization of “a good theory” is a simple linear function where there is a specific relationship to each point on the line. There are not two points on the Y axis for a point on the X axis. There is specifically only one point on the Y axis for each point on the X axis.

 


Categorical variables

Only in extreme cases with extreme differences it is beneficial to use categorical variables such as done in the original SCAA Roasting Defects curriculum (which included the categories ‘baked’, ‘scorched’, ‘faced’, ‘tipping’, ‘underdeveloped’) and when we did our extreme difference mapping in our first piece of research mapping out sensory properties of roast profile modulation Common roasting defects in coffee: Aroma composition, sensory characterization and consumer perception. And since then I have started to avoid using those categories at all as they are to relative and not useful from a product development perspective where these clumsy concepts can stall your thinking as the only relevant question is: How would I like it to taste? And which type of green coffee should I choose and how can I modulate the flavor to meet that sensory objective? As long as you don’t go extreme you don’t need to worry about the extreme defects and you can focus on just experimentation and evaluation iteratively until you have achieved your flavor objective.

Scientifically

Where the conceptual approach to a theory is more mathematical the more practical scientific approach is a bit more diverse as there are several sciences and in our case we have to look at how sensory science deals with theories, concepts and research design.


Pangborn’s Razor

The theory of the superiority of the constant declining Rate of Rise violates the fundamental distinctions of sensory science has developed by Rosemary Pangborn at UC Davis at least 50 years ago which I've called Pangborns Razor in earlier material to illustrate the usefulness of this distinction as a way of shaving away nonsense or unclear Concepts in Disguise of complexity just as Occam's razor in theory of science. Pangborn advices a series of questions with corresponding experimental setups where you clarify one step at the time in a certain sequence



Is it worth investigating at all?: Pangborn’s Discriminative test

The intuition behind Pangborn’s Razor is that before investing too much attention in an elaborate theory of flavor differences the very first criteria for even spending time with the theory is to test if the claims of the theory makes any perceivable sensory difference at all!


So according to fundamental distinctions and sensory science you have to be very aware, critical and skeptical about theories dealing with only small differences in the input parameters. This first criteria is surprisingly often not really mentioned at all or dismissed with reference to personal experience and that hundreds of students and clients have been convinced. From a scientific point of view this is a highly biased setup! I also convinced hundreds of students about my theories before I started doing sensory scientific data at the university (read this blogpost Your Perfect Coffee Roastery) and I think of CoffeeMind’s approach to coffee roasting as a pre- and post-Pangborn phase after which I have a completely different perspective on what I expect to be a big difference and a small difference and an extreme skepticism for claims of the relevance of small differences and I transformed through the pain of publicly being wrong.


Ceteris paribus - ‘everything else being equal’ principle

Scientifically it also violates the everything else being equal principle which is a fundamental setup of an experiment where you make sure that one and only one parameter changes between the different input samples in the set-up to see how this one input parameter changes the output parameters. This is the most often violated scientific principle in experiments done by non-scientist who are not aware of this fundamental principle of research design. This happens for example when people are doing drum speed or airflow experiments without making sure that all samples end at the same color in which case the samples will end up with different colors. Later at the cupping table, when a preference is found, the superiority of the preferred cup is wrongly associated with either drum speed or airflow or any other experimental factor where really the preference is related to the color of the sample without them knowing it. Colour is in this scenario a confounding factor on the decisions on the cupping table and since the person is not aware of this a wrong conclusion is made.

Another Problem with the theory of the superiority of a constantly declining Rate of Rise is that it cannot be tested as a theory where the everything else equal principle is applied because you cannot treat a sample with a constant declining Rate of Rise and compare it to a non-constant declining Rate of Rise of the same color and development time as they would end up with either different development time and/or different color due to the constraints of the dynamics of the curves in a live roaster. This is a consequence of the theory not being formulated properly in the first place using only first principles of thermodynamics and chemistry which would be plain time temperature and color. 


Pangborn’s Descriptive and Affective step

And from a sensory science perspective a big problem about the theory about the superiority (Affective claim) of the constant declining Rate of Rise is that you assume as universal preference maximum without being specific (Descriptive step) about the insensitive involved in making it a maximum at all. And even if the intensity was mapped out, the assumption that there is a universal preference across consumer segments is violating Pangborn's Razor.



Variation of input and output in experimentation

At the heart of an experiment is variation of one and only one input parameter to map out how the variation of this parameter varies the output parameter. To follow the idea of a constant decline Rate of Rise is an advice the does not encourage you to do your own experiments with flavor modulation because it assumes one optimum. In CoffeeMind we want to empower people to do flavor experiments so we don’t not paint people into a preference corner during education. The question of optimizing flavors should be left to the specific business model of a specific product development process.


Only specific (and big) things matters

It's worth noting that a good Theory provides a linear relationship between the input and output parameters such as more or less acidic more or less bitter where category differences require big differences. In extension of this you might argue that I did this myself when arguing that you could use the Rate of Rise to categorize you roast by specifying the average Rate of Rise after 1st crack but remember, that I only do this because in the research we where  looking at pretty extreme differences in development time which was spanning from 1,5 to 6 minute development time between samples and I only suggest to categorize the extremes where it seemed in our data that the sensory descriptors modulated pretty linear when going from one extreme to the other extreme. And the linear tendency we mapped out lends itself perfectly to product development and customer preference mapping completely aligned with Pangborn’s model for intensity mapping and subsequent consumer segmentation (not assuming that one flavor fits all).

Getting a sense of what is a big and a small difference is where the reluctance to even think of the Rate of Rise course to be relevant comes in.

CoffeeMind’s general framework

In CoffeeMind we have noticed a general trend in the coffee community to drag students down rabbit holes with very complicated unclear concepts that simply drowns their minds to the extent that they miss the big picture and their purpose of it all. This is why we in CoffeeMind have created a competency roadmap (Figure XX) for start-up roasteries with really simple and specific steps that are finite in scope and very practical in the application so you can go to the next step after finishing a step. This way it's very clear how each step fits into the next and aligns with the overall purpose of the organization. Or our framework has the following steps

  1. Personal purpose of having a company in the first place
  2. Business vision derived from personal purpose
  3. Control (thermodynamics of your machine where it is installed)
  4. Flavour Experimentation
  5. Intensity mapping
  6. Quality Control
  7. Consumer mapping

Ongoingly the organization is rearranged through continuous improvement all the way back to the passion and purpose of the founders of the company to keep the company filled with passion and purpose. Notice how we have applied the fundamentals of a good theory by isolating each step as an ‘Input → Output’ system in itself. The personal passion and talent is an output into the input of the business vision which is defined by the customers the organization is committed to serve. The business model needs predictability which is why Control of the production equipment is the first competence needed in the organization. This consistency output is the input to the Experimentation step which can’t be done properly without sufficient control of the machine. So in the roasting equipment the Input is the settings of the heat source and the output of the roast curve. But we don’t stop here as the output of the Experimentation step is the Input into the Evaluation step and the output of the evaluatio step is the flavour intensities mapped out. The input into the consumer step is the output of the experimentation step namely the flavour intensities. And the output of the Consumer step is the liking of the intensities in the different consumer segments! All is connected but nicely separated for each step so it is manageable and success is achievable without missing the big picture (getting stuck in rabbit holes which seems to be the trend) of the organization and your own life quality as a CoffeePreneur!

Only by following a simple, rigorous and practical education framework like this can we make sure that we educate coffee roasters to be able to align their methods and efforts with any business model they might find themselves embedded in in the future (regardless if it is their own or if they are employed or even if they want to impress family and friends as home roasters! Don’t serve the same coffee to your hipster friends as your mother in law!) as they are not painted into a preference corner by the educators who educated them in the first place. This is completely aligned with and in the spirit of Pangborn’s distinctions and therefore the gold standard of science for product development and where to spend your energy on the different processes and questions in the company! In CoffeeMind experimentation and not unification coupled with expected diversity of preferences is at the heart of coffee roasting education.