"Point Four Four per Floor" –– QFE to QNH to QFF

Several years ago we developed a jingle for making elevation corrections to barometer readings for quick conversions from station pressure measured at some elevation to sea level pressure needed for weather work.

It goes "Point four four per floor," which is intended to remind us the that pressure drops 0.44 mb for every 12 ft we rise above sea level––we are calling 12 ft a floor, which is more or less right for buildings with multiple floors. Needless to say, we chose the word because it rhymes, not because it matches some architectural standard.

If your barometer was 120 ft above sea level and it read 1012.5, then you would increase that by 4.4 mb to read 1016.9 to get the equivalent sea level pressure. More generally, the correction is just (H/12) x 0.44 mb.

This jingle will serve many needs we have in this department.

For example (using info from an earlier note):

The barometer on my boat is, mounted 6 ft above the water. It reads 1015.1 mb.  The tide is 13 ft, MHW = 8 ft, MLW = 2 ft, so mean sea level is (13-8) + (8-2)/2 ft, which equals 8 ft below the surface of the water. So this barometer at this moment is 14 ft above sea level, and thus I need to correct it by: (14/12) x 0.44 = 0.5 mb. The correct SLP at this moment is 1015.6 mb.  If this moment happened to be a synoptic time (i.e. 00, 06, 12, 18z) then I could wait a couple hours for the next weather map that covers this time, and I should see that pressure on the map at my location. Or i can compare this with a local buoy or lighthouse report, which is given every hour (google "nws ndbc").

This jingle will almost always work for computations like that one, even on up to several hundreds of feet, over a broad temperature range.  But this jingle is definitely an approximation, and it is time to address this more specifically so we know what affects the accuracy and how to compensate for variances.

It is an approximation for two main reasons. One, the density of the air decreases with increasing elevation, so it cannot be 0.44 forever as we go up.  Also the conversion of a pressure measured at a high elevation to the best equivalent value at sea level pressure (SLP) at that location and time depends on the outside air temperature as well as the elevation. Worse than that, it also depends on other properties of the atmosphere at the moment and it even depends on the local geography, but the main factor beyond elevation is the air temperature.

The conversion dependence on these factors is a complex one, and even meteorologists from different countries do not agree on the best way to compute it. The best rules for the highlands of Norway are not the same as the best rules for highlands of New Mexico or Colorado.  For an introduction to the conversion process and the factors involved, see Appendix A3, Reducing Station Pressure to Sea Level Pressure in The Barometer Handbook. More details on the issues and challenges are presented in the WMO document CIMO/ET-Stand-1/Doc. 10, Pressure Reduction Formula, Nov, 2012.

To begin, the jingle is an approximation to an approximation, and to clarify that thought we need to use the specific terms for the types of pressures we are dealing with. The short definitions are: QFE = station pressure, QNH = sea level pressure figured from elevation alone (called altimeter in aviation weather),  and QFF = best value of the equivalent sea level pressure taking all known factors into account.

There is a unique and unambiguous way to find QNH from QFE. We assume that QFE decreases with increasing elevation exactly like the pressure drops with elevation in the International Standard Atmosphere (ISA). This average atmosphere was developed originally by aircraft designers who needed some standard to work with. There are tables and formulas online that compute the pressure of the ISA as a function of altitude.  A slick online computer is at www.digitaldutch.com/atmoscalc, which even lets you create a printable table to your design.

Essentially all electronic barometers on the market that offer the option to display SLP as a function of elevation are presenting QNH and they are using the ISA formulation. Indeed, most of the time we hear or see the phrase sea level pressure the implication is that the elevation dependance is this simple one based on the ISA. If air temperature is not mentioned in the same sentence, then it has to be that one.

And the results (ie QNH) are actually very close in many applications, compared to the best we could do (QFF) if we made a lot of corrections. We have to go to extenuating conditions (high elevations or unseasonal temperatures) to see notable difference, or we have to care for a precision that may be difficult to justify.

So our jingle of point four four per floor is just an approximation of how the ISA pressure drops with altitude over the lower layer of atmosphere, which as noted is itself an approximation of how QFF might drop with elevation. Note I say, "might drop" with elevation.  Throw a big temperature inversion in there and even the best computers will have trouble coming up with a good sea level pressure.

Table below shows how the jingle falls off in accuracy as the elevation increases. Also shown are the ISA air temperatures as a function of elevation. As you go up in the atmosphere, the pressure and the temperature drop.

Elevation (ft)    Jingle factor     ISA air temp (Fº)
0                       0.44                 59
100                   0.44                 59
500                   0.44                 57
750                   0.44                 56
1500                 0.43                 54
3000                 0.42                 48
6000                 0.40                 38

So the first thing we see is our jingle is good up to 750 ft, and still ok within a couple percent to 1500 ft.          

But the underlying assumption of using this conversion (QFE to QNH as an approximation of QFF) assumes the temperature of the air as a function of elevation matches that of the ISA––in some sense.

This brings up an interesting distinction in language that is related to our problem here. The ISA describes the properties of the air as a function of altitude, which is assumed to be distance up in the air, whereas we are dealing with our elevation, which means a height above sea level while still on land. A cloud is at some altitude; a hiker is at some elevation.

In short, we have little reason to think that the air temperature at our elevation is what it would be if there were no land below us.  We know the weight of the air above us, that is QFE which we measured, so a determination of QFF is tied to making some estimate the average temperature of the imaginary air column below us, so we can figure the weight of that, which added to QFE is QFF. 

For practical corrections at sea we have some advantage.  First we are on a uniformly flat surface (water or ice), as opposed to in a mountain range, and second we are not going to be very high.  Thus the NWS Observing Handbook No.1 can include a table for navigators to use to make the correction for elevation and temperature in one step.  Clearly there has to be some empirical nature to the result, because there is not agreed upon theory, and I would guess that in principle the correction depends on the pressure as well, but the main point is everyone uses the same corrections, and that is what the models need for consistency.  Below is the NWS table with some inserts added.

The white background are the actual numbers from the NWS Handbook Table. Thus if the elevation is 110 ft and the outside air temp is 86º F, the correction to get from QFE to QFF would be + 3.8 mb. The yellow number beside it means that of this 3.8, 0.2 is due to the temperature, which caused a lower than normal correction.  Likewise, if the air temp had been a chilly 4º F, the correction at 110 ft would be 4.6 mb, of which 0.6 mb was due to the air temp, and this caused a higher than normal correction.

These yellow numbers come from just computing what the elevation correction alone would be using our 0.44 jingle, and then subtracting it.  In a sense, the yellow numbers reflect the difference between QNH and QFF.  We see from this that air temperature is not a big factor for low elevations, even with rather extreme temperatures, but since we do have this good approximation, we are better off using it than not.

At higher elevations, however, these corrections are quite large. We will post here some results on that as soon as completed.  These we have to compute.

Our  next step then––the truth meter––will be to compare our results with the differences seen in metar reports for various elevations and temperatures around the county.

Mean Sea Level, Tides, and Barometers

Recently on thinking about the precise elevation of my desk for making barometer corrections, i happened to notice for the first time that a navigational light that I see every night is about at eye level from my desk. As luck would have it, in that same desk is a precision Kuker-Ranken hand held telescope level, which confirmed what the eye could see, namely  that we are indeed very near the same elevation as that light.

Now comes the question at hand.  The light shows on a nautical chart at a height of 29 ft.  But charted elevations are the heights above mean high water (MHW). Checking the chart we find that the MHW is 10.5 ft––that is the only place you find that; it is not in tide tables––and navigators then know that when the the tide is 0 ft, this light will then be 39.5 ft (29+10.5) above the water, and if the tide were 11.5 ft, (1 ft higher than MWH) then the light would be 28 ft above the water.

But for barometer work, we need to know the height of the light above mean sea level ( MSL), which is not the same as zero tide. Zero tide in this area corresponds to the MLLW.

There are various ways to define MSL, but an easy, usable one is the average of MLW and MHW, which is called the Mean Tide Level (MTL). Thus we could write

MSL = (MLW + MHW) / 2.

We can usually get MLW from some charts as well. The local chart here gives MHHW, MHW, and MLW. If your chart does not have what you need, you can go to the tide tables to get it.

In this case we have MLW = 2.8 ft, so we can call MSL the height of the water when the tide  = (10.5+2.8) / 2 = 6.7 ft, which means MSL is (10.5 - 6.7) = 3.8 ft below MHW.

So our light is 29 ft above a tide level of 10.5 ft, which means it is 29 + 3.8 = 32.8 ft above MSL––which to a good approximation must be the elevation of my desk above MSL. 

Fig. 1. MSL defined in terms of halfway point between MLW and MHW.

So we have a sort of interesting arithmetic that MSL = ( MHW + MLW ) / 2, which means the increase in the elevation reference of a light from MHW to MSL = ( MHW - MLW ) / 2.

( On the other hand, we do have an accurate barometer here, and we could look up the height of the tide, then just measure the pressure at the desk compared to that at the water surface, and figure the elevation using our jingle “four point four per floor,” i.e. a drop of 0.44 mb with each elevation increase of 12 ft, defined here as “a floor,” but that means getting up from this desk, which so far I have not had to do. )

All Nautical Charts reference lights and bridge clearances to MHW, but remember they still reference the elevation contours and spot peak heights on the charts to MSL. This latter matter does not enter into routine navigation very much, but it does indeed matter when you are using a sextant measurement of the height of a low isolated near-coastal peak above the water to determine an accurate distance off, especially in areas with a high tide range.

Setting a ship’s barometer to read Sea Level Pressure

This topic has a direct application to setting  a ship’s barometer to read sea level pressure (SLP). This if often the Captain’s choice so they do not have to make corrections that depend on their loading or on the stage of the tide.

Referring to Figure 2, if the ship’s barometer read the correct station pressure, then the Captain could report the SLP by correcting that reading by the elevation labeled h-water. This elevation must then  be determined from h-deck or h-load line, corrected as needed for actual loading (dH). 

Fig. 2 Ship at dock.

But we do not know without a reference standard if the ship’s barometer is correct or not. Ship’s barometers are often checked by the local Port Meteorological Officer (PMO). The PMO can take a well known standard barometer on board and hold it at the elevation of the ships barometer, and any difference noted is the error in the station pressure of the ship’s barometer.  This correction can be recorded and applied to the reading before the elevation correction is applied.

That is a standard procedure used to check the ship’s barometer, but then to use this barometer reading to report SLP, the Captain or officer in charge must know the h-water at each reporting.  The dH is usually well known or can be seen on the side of the hull, but sometimes, the height of the instrument on the ship is not known to a high precision.

We keep in mind at this stage, that the goal of pressure reports is to give the SLP accurate to ± 0.1 mb if possible, which corresponds to an elevation uncertainty of (12/0.44) x 0.1 = 2.7 ft.––which we cannot write without wondering about the effects seas on the pressure reading, but that is another topic.

The goal of Fig. 2 is to show one approach to either measuring the actual height of the ship's barometer, or to set the barometer directly to SLP, which then removes all factors but dH. 

For example. Suppose the reference barometer reads 1020.0 at height h-ref = 10 ft above the water, standing on the dock.  Then we carry the ref barometer up to location of the ship's barometer and now it reads 1017.5 mb.  So there is a 2.5 mb elevation change between the dock height and the ship's barometer height.  This corresponds to (2.5/0.44) x 12 ft = 68.2 ft. So the ship's barometer is located 68.2 + 10.0 = 78.2 ft above the water level. But this waterline can change, so we need to know the dH, and the structural elevation of the barometer is 78.2 ± dH as now measured. For the sake of example, let us say the ship is floating 2.0 ft above its reference load line, so the ship's barometer's structural elevation is 76.2 ft above the load line.  

Thus the elevation correction to the water level for a less loaded  ship is 76.2 + dH, and for a heavy loaded ship it is 76.2 - dH.

So we have found the elevation needed to convert to SLP at sea (away from notable tides), but if our goal is to actually set the instrument to read SLP, we have to apply these corrections accounting for the present dH and any present errors in the instrument.

Suppose the ship's barometer happens to read 1016.3 at this time. The first thing we know is the instrument is not reading the right station pressure.  We have sitting beside it a calibrated reference barometer that reads 1017.5. 

So step one, is raise the ship's barometer to read 1017.5. Then we know it is reading the proper station pressure... and it could be that is the end of the task if we do not want SLP, ie just leave it at that, and the Captain makes the correction at sea for 76.2 ± dH.  Notice that we have to say at sea with no tides. If the report is made from inland waters with notable tide, then the right elevation to use is 76.2 ± dH + (Tide - MHW) + (MWH - MLW)/2.

This fact also complicates meeting a Captain's goal of setting his barometer to read SLP when at a dock that has high tides.  We have to explain carefully what we are setting.

Let us assume the goal is to set the instrument to read SLP when at sea with no tide, but we must do this now at a time when the tide is high.

Suppose the height of the tide at this moment was 15.0 ft, in a place where MLW = 3 ft and MHW = 13 ft. Therefore, MSL = a tide of 8 ft, so MSL is 7 ft below the present water level. At this moment, then, the ship's barometer is located ( 76.2 + 2 + 7 ) 85.2 ft above MSL.  This corresponds to a pressure drop of (85.2/12) x 0.44 = 3.1 mb.

If we raise the 1017.5 by 3.1 mb to get 1020.6 mb we would indeed be reading SLP from this instrument. But for this instrument to read the proper SLP it must be located 85.2 ft above MSL. When the ship heads off to sea where the tide is zero, and the load is maybe neutral (then we are off by two factors, the missing tide and the missing dH.)

So one option might be, set the instrument to read SLP when the tide is zero and the load is neutral, then do a manual correction when these criteria are not met, such as right now!

To do this, we just have to undo tide, assuming the water level will then be MSL at sea, and set dH=0. We already know what we need. The height of the barometer above the load line is 76.2 ft.  This corresponds to a correction of (76.2/12) x 0.44 = 2.8 mb.  Thus we take the correct reading of 1017.5 and raise it 2.8 mb to read 1020.3 mb.

This will not be the correct SLP now, but when the ship goes to sea on a level load line, the pressure read from the barometer without further correction will  be SLP.

Then the ship can just use a short table of baro corrections as a function of dH, and that is all that is needed at sea.

In this example, when they pull into this dock at this moment and the barometer reads 1020.3  they know they are now 7+2 above MSL so they must correct by (9/12) x 0.44 = + 0.3 mb, to get what we know is the right SLP of 1020.6 mb

So the summary of this method is: set the barometer to SLP from an elevation above the load line, then correct as needed with

MSL correction =  ± dH + (Tide - MHW) + (MWH - MLW)/2.

In areas with low tides and ships with near constant load lines, this is not a factor, but in parts of Alaska, for example, the tides have 30 or more feet of range, so this could be an important factor. The tide values used here are typical of Puget Sound.

Examples: Removal (barometer to waterline) = 5 ft, MLW=2 ft, MHW=10 ft.  and dH =0.

Then if tide = 13 ft, then MSL correction is (13 - 10) + (10 - 2)/2  = 7. Mean seal level is 7 ft below the surface, and full correction is (5 + 7) = 12 ft.

If tide = 0, MSL correction = (0 - 10 ) + 4 = -6. Mean sea level is 6 ft above the surface, so the total correction is (5 - 6) = - 1ft.

Other examples and related discussion has been added in this article:  Point Four Four per Floor – QFE to QNH to QFF.

Answers to Geocaching Exercise

Spoiler alert!

Do not read this if you plan to work the  practice exercises we posted in an earlier article called 

Piloting Practice with Geocaching — You Can Do It Online.

Horizontal Sextant Angles

(1) home plate

(2) 6 ft right of pitcher's plate

(3) 6 ft behind pitcher's plate

Here is a Google Earth screen cap of the location involved.  The objects referred to are the trees, marked here with yellow circles. The center of the tree trunks are your plotting targets.

We will come back and annotate this with the vector solutions.... as soon as at least one person asks for it with a comment!

Line of Soundings Navigation

(1) Corner of Central Park West and West 69th St. (40° 46.456' N, 73° 58.43' W).

Here is a screen  cap of the location involved seen in the nationalmap viewer.  The route to be found is the one marked on the image.   This is the north end of the park.  With little practice you will find that this is a unique location and easy to home in on.

The trick is find the approximate location from bulk results, then home in and print, then plot on another paper the elevations at the right separations, then just slide this along the chart keeping it oriented at 298T.  This may seem daunting at first, but with little practice you go more or less straight to it.

New Starpath Kindle and iBooks now online

Starpath textbooks function well in several ebook formats. They offer full color graphics, zoom for detailed images, highlight text, search and find.

They offer a functional and economic alternative or supplement to our printed books. See www.starpath.com/ebooks for various options.

The Amazon Kindle Store and the Apple iBookstore offers free samples and good previews of the texts.