Takeoff rounding error: 3 checks before pricing

Key takeaways

• Note the reading limit beside each measured length.
• Compare relative errors before and after conversion.
• Check the factors behind areas and volumes.
• Keep working values unrounded until the quantity you price.

A takeoff sheet that writes a length with an extra decimal place looks more exact than one that rounds it to the nearest whole foot. The number of digits you write says how precise you think the measurement is, and the bid carries that claim into every line it touches. If your reading of the plan is good to about an inch, writing the length to a hundredth of a foot promises more precision than that reading supports. Multiply that by a unit price and the extra digits become a total that looks confident and is not.

Most estimators know this in principle and still round by habit. They keep whatever the calculator shows, or whatever the software prints. The fix is not complicated. Before a length goes into a price, run three checks.

The checks use a worked example from the National Institute of Standards and Technology (NIST), which covers converting a measured length from one unit to another. The same logic then carries through to areas and totals. The conversion numbers in this post come from that example, and the arithmetic is shown so you can repeat it with your own lengths.

Before you start: what the digits promise

Significant digits are the digits a reading actually supports. If the reading is good to about an inch on a 36 ft run, the digits that describe the run down to a single inch are meaningful. Digits past that point describe nothing you measured. They are arithmetic leftovers.

This matters most when a number passes through several steps. A value that looks precise at the start can look precise at the end, even when its uncertainty has grown along the way. The three checks are a way to keep the uncertainty visible from the reading to the price.

Check 1: turn the reading limit into a relative error

Start with the reading, not the number on the sheet. Ask how far off the reading could be. A scale rule, a digital measuring tool, a zoomed PDF, and a tape each have a limit, and that limit depends on how you read the mark and how the drawing was printed or displayed. Write the limit as an absolute amount, such as one inch or one foot, and keep it beside the length. For setup, see takeoff scale setup and stated limits.

Then divide the limit by the measured value. The result is the relative error, or RE: the fraction of the length that could be wrong. NIST works this exact case. The length is known to within one inch, and the NIST guide states: "Then in this case, since l is known to within ± 1 in, the RE of the numerical value "36" is ± 1 in/(36 ft × 12 in/ft) = ± 0.23 %." That is about a quarter of one percent of the run. You can get the same result for any length by dividing the limit by the value, in the same units.

Working in RE rather than raw inches matters because the same limit means different things at different sizes. One inch on a 36 ft wall is a small fraction of the wall. One inch on a 4 ft piece is a much larger fraction. A single rule such as "two decimals for every length" cannot serve both. Work out the RE for the long runs and the short pieces separately, and let each item set its own digit count.

In practice, write the reading limit and the RE in the margin of the takeoff, next to each length. It takes seconds. When the totals come together, the RE column shows which items control the result and which are noise. A long run with a small RE is usually safe at a modest digit count. A short piece with a large RE is where a careless digit can mislead, even though it looks trivial on paper.

If you doubt the limit, test it. Measure the same run twice, from different starting points on the sheet or with a different tool, and see how far the two readings disagree. If they differ by more than the limit you wrote down, the limit was too small. Reassess it before pricing; agreement between two readings alone does not establish the limit.

Include whatever moves the mark: the thickness of the line you measured, whether the sheet was scaled when it was printed, whether you measured to a centerline or an edge, and whether the designer rounded the dimension on the plan. If you cannot name the limit, make a conservative guess and say so in your notes. A conservative limit is far more useful than a precise-looking number with no basis.

Check 2: watch what a unit conversion does to the digits

Many takeoffs cross units. The plans are in feet and inches, the supplier quotes in metres, or the unit price applies to each square metre. The arithmetic of conversion is simple. The trouble is that rounding after conversion can change the relative error of the number you keep, and the output gives no warning when it does.

NIST's example shows the trap. Take the 36 ft length from Check 1 and convert it to metres. Round that to two significant figures and you get 11 m.

The source then states the general lesson: "This example therefore shows that when selecting the number of digits to retain in the numerical value of a converted quantity, one must often choose between discarding information or providing unwarranted information." Round too hard and you throw away precision the reading had. Round too little and you claim precision the reading cannot support. Neither choice is free, and the conversion itself will not tell you which one you made.

NIST's way out is to keep one more digit. It gives the converted length as "l = 11.0 m." The RE of that value is ± 0.45 %, about twice the ± 0.23 % of the original. NIST describes the original ± 0.23 % RE as lower than, but comparable to, the ± 0.45 % RE of 11.0. It then concludes: "Therefore, the result l = 11.0 m is still given as the converted value."

The other extreme is just as wrong.

So the check is simple to state. After converting, compare the rounding RE of the retained value with the original reading's RE. If it is much larger, you rounded too hard, so keep another digit. If it is much smaller, you are carrying digits the reading cannot support, so drop one. "Comparable" is a judgment, not a formula.

The same trap appears in area conversions, where the factor is squared. Converting square footage to square metres uses the squared length-conversion factor, and the rounding decision has to be made again on the converted figure, not carried over from the length.

A few habits make this easier:

• Convert from the unrounded value. Keep the full figure in your working cell and round only what you write down.
• When a supplier or a piece of software does the conversion, compare its digit count with yours before you accept it.
• Record both units and both REs in the notes, so a reviewer can follow the conversion without redoing it.

Watch for round trips too. If a supplier's metric quantity is converted back to feet for your bid, each conversion can add its own rounding. Compare the final figure with the one you started from, and let the less precise of the two govern the digit count.

Check 3: trace the error through to the total

Lengths seldom go straight into a bid. They multiply into areas, volumes, and counts, and those feed costs. Each multiplication passes error along. For a product of measured factors, the relative errors can add. In the worst case, the RE of a wall area is about the sum of the RE of its length and the RE of its height. An area is therefore never more precise than its roughest factor, however many digits the calculator prints.

Walk the chain for each line. List every factor that goes into the quantity, write each factor's RE, and let the roughest one set the digit count for the line. A waste allowance behaves the same way. It is an estimate rather than a measurement, and it limits the precision of the line just as a roughly known height does. Carrying six digits through a chain whose height is only roughly known is precision theater.

Counts need the same care. A count of items from a layout is a whole number, but if the count depends on a spacing you measured, its uncertainty follows that spacing. Ask what the count is derived from before deciding how to treat it.

Software often builds shapes from points, and the same reasoning applies there. Bluebeam's volume measurement guide describes the basic method: "Select each point in your shape to define the area." That page covers volume rather than area, but the logic holds. Each point you place is a reading with its own limit. A curve traced with many points has many chances for placement error to drift, and a shape of a few straight edges has fewer. If you placed the points roughly, the area is roughly known, however many decimals the tool prints.

Finally, round once. Keep working values unrounded through the chain and round only the quantity you will price. When several lines are summed, their errors can add in the worst case, so check the contribution of each line's uncertainty to the total. Choose the bid's digit count from that uncertainty, not from the software's display.

Illustrative example: compare long and short runs

Suppose a reading limit is 1 ft. On an illustrative 20 ft run, the same limit gives an RE of 5 percent. The shorter run has the larger relative uncertainty even though both readings have the same absolute limit. Record the limit beside each length before choosing how many digits to carry.

Common mistakes to avoid

A few habits cause most of the trouble:

• Carrying the display's digits. A spreadsheet or takeoff tool shows as many decimals as its settings allow. The display says nothing about how good the reading was.
• Rounding the conversion too hard. As Check 2 showed, rounding the converted value can increase its RE substantially. Keep the working figure unrounded.
• One digit rule for every length. The same limit is a different fraction of each item, so a single rule fits one item and misfits the rest.
• Trusting an area without its points. A tool's total hides how the shape was drawn. Check the points, and ask whether each one could be off by the reading limit.

Where unit prices raise the stakes

When a price applies to each unit, the quantity drives the bid total. The estimate multiplies straight into the total, so the digits you carry also set how confident the total looks to the person reading it.

One public rule addresses quantity variation directly. The Acquisition.gov page for Subpart 11.7 of the Federal Acquisition Regulation states: "Construction contracts may authorize a variation in estimated quantities of unit-priced items." That sentence is a permission written for federal acquisitions. It does not tell you how far quantities may move on a private job, and your own contract decides that. Read the clause before you assume it applies to you.

The practical point is about digits. If quantities may vary, the estimated quantity in the bid is a planning figure, and it should show the precision of the estimate, not a spurious one. A note that gives the measurement method and the RE lets the reader judge the number without being misled by its decimals. Review the price inputs separately with the construction material cost line checks.

A worksheet for every length

Run these steps on every length and area before you price it:

1. Write the reading limit for each length, in the unit you measured it.
2. Divide the limit by the value to get the RE.
3. Convert from the unrounded value, then compute the rounding RE of the retained value.
4. If the converted RE is much larger than the original, keep another digit. If it is much smaller, drop one.
5. For each area, volume, or count, list every factor and its RE. Let the roughest factor set the digit count of the line.
6. Round once, at the quantity you will price. Keep working values unrounded until then.

The order matters. You cannot judge a conversion until you know the RE of the original, and you cannot judge an area until you know the RE of its factors. Doing the steps in this order means each decision rests on a number you already checked.

Some items do not need all six steps. A lump-sum allowance is not a measured quantity, and an item priced by the job rather than by its length has no measured-length RE to protect. Give those a short note saying they are estimates, and spend your effort on the measured lengths that drive the cost.

Your next step

Choose one measured length in today's takeoff and run the worksheet on it. Keep the original reading, its limit and any converted value together so a reviewer can follow your rounding decision.

Common questions

What is relative error?

Relative error is the reading limit divided by the measured value, using the same units. It expresses the limit as a fraction of the length.

Should every takeoff length use the same decimal places?

No, the digit count should reflect each reading's uncertainty. A fixed reading limit represents a larger fraction of a short piece than of a long run.

When should I round a quantity?

Round the quantity you will price after carrying unrounded working values through the calculation. Keep the reading limit in your notes alongside that quantity.