Back to the Table of Contents

Numbers and Their Application - Lesson 9

Scientific Notation, Significant Figures, etc.

Lesson Overview

Accuracy vs. Precision

Accuracy is a measure of rightness.              Precision is a measure of exactness.

Versus (vs.) is Latin for against or facing. Accuracy and precision, although similar in meaning, have a very subtle difference important to mathematics and science in general and statistics specifically. You can have one without the other, neither, or, best of all, both together. As you can see below, precision has to do with repeatability, how well your results can be reproduced. Here is an example involving e. It is an important number we will study further in lesson 14. It is also on your graphing calculator is several places.

e = AccuracyPrecision
27NO NO
2.18281828NO YES
2.72YES within 1 ppk NO
2.718281828YES YES

A popular chemistry book illustrates all but the third case with a dart board and darts as follows.

DartsAccuracy Precision
Randomly spread far from the bull's eye NONO
Clustered in the upper left cornerNO YES
Clustered near the bull's eyeYES NO
Clustered inside the bull's eyeYES YES

A common measure of precision is the standard deviation or uncertainty. We will discuss standard deviation more in the upcoming Statistics lectures. Uncertainty is the magnitude of error that is estimated to have been made in the determination of results. It is now common to state results in the form measurement (uncertainty) units. As an example consider the results from the author's dissertation available at http://etd.nd.edu, click on search, enter Calkins as last name, and click now on the search button below. We reported there our 2005 results of the cesium D1 transition centroid frequency as: 335 116 048 748.2(2.4) kHz. Basically, the 2.4 kHz is saying we are about 68% confident that the true value is with ±2.4 kHz of the reported value of 335.116 048 7482 THz (about 894 nm or in the infrared).

Scientific Notation

In science, numbers large and small are commonplace and a shorthand notation call scientific notation was developed to simplify their specification and utilization. It is based on place value and base ten. Recall that 101=10; 102=100; 103=1000 and 3×100 = 300 = 3×102 or 103× 81= 8.1×104.

A number is in scientific notation if it is in the form: Mantissa ×10characteristic,
where the mantissa (Latin for makeweight) must be any number 1 through 10 (but not including 10),
and the characteristic is an integer indicating the number of places the decimal moved.

The manissa might sometimes be called a coefficient. The term mantissa is more commonly applied to the decimal portion of a logarithm.

Examples of scientific notation:
92,900,000 miles becomes 9.29×107miles (earth-sun distance)
Planck's Constant:  .000000000000000000000000000000000663 Js becomes 6.63×10-34 Js
3141592653 is approximately 3.1416×109
6,600,000,000,000,000,000,000 tons is 6.6×1021 (6.6 sextillion) tons or the "mass" of the earth.

Note the use of the EE key on calculators and an E on computer printouts in reference to scientific notation. 3.14E9 is the same as 3.14×109. D may also be seen indicating use of double precision (typically 64 instead of 32 bits of precision). An easy way to remember when changing number into scientific notation is: if the mantissa is a smaller number in magnitude than your decimal value, then the characteristic will be a positive number. If the mantissa is a larger number than your decimal value, then the characteristic will be negative. Keep this hint in mind as you change from scientific to decimal notation. Example: 5.43×10-3 = 0.00543, since the characteristic is negative, you know the decimal number is smaller than 5.43, so you move the decimal left. Another example: -0.000002 = -2×10-6.

Operations with Scientific Notation

When adding numbers in scientific notation, the characteristics must be the same. For example:
2.3×105 + 4.55×103 The easiest way is to decrease the larger characteristic by rewriting the mantissa!
230×103 +4.55×103 = 234.55×103 After rewriting and adding, rewrite in scientific notation.
2.3455×105 [approximately equal to] 2.3×105 Results rounded according to rules given below.
Notice what happens when you add the following together: 8.23×1017, 4.67×1012, and -1.05×10-12!

The same method is used when subtracting numbers in scientific notation! Here, however, if they are close in value loss of significance may result–the answer may be nonsense! When multiplying numbers in scientific notation, add the characteristics and multiply the mantissas. Division is similar, divide the mantissas and subtract the denominator's characteristic from the numerator's characteristic. Always convert the answers back into proper scientific notation form.
8.1×10-3 × 2×105 = 16.2×102 = 1.62×103.        1.08×1017 ÷ 1.2×1010 = 0.9×107 = 9×106.

A variation on scientific notation is engineering notation. In engineering notation the exponent is a multiple of three, reflecting the fact that the standard multiplier in the metric system is 103=1000. It is thus more common to speak of meters, kilometers, millimeters, nanometers, and femtometers than is to speak of decimeters and dekameters. Unfortunately, some units such as centimeters and Angstroms are entrenched which complicates our conversion to SI (see below).

Numbers written in scientific notation are assumed to be measurements, thus approximations.
Therefore, the rules outlined below must be applied.

Significant Figures, Rounding and Truncating

The significant figures (digits) in a measurement include all the digits that can be
known precisely plus a last digit that is an estimate.

The rules for determining which digits in a measurement are significant are:

  1. Every nonzero digit in a recorded measurement is significant. 24.7 m, 0.743 m and 714 m all have three significant figures.
  2. Zeroes appearing between nonzero digits are significant. The measurements 7003 m, 40.79 m, and 1.503 m all have four significant figures.
  3. Zeroes in front of (before) all nonzero digits are merely placeholders; they are not significant. 0.0000099 only has two significant figures.
  4. Zeroes at the end of the number if a decimal point is present and also zeroes to the right of the decimal are significant. The measurements 1241.20 m, 210.100 m and 5600.00 all have six significant digits.
  5. Zeroes at the end of a measurement and to the left of an omitted decimal point are ambiguous. They are not significant if they are only place holders: 6,000,000 live in New York—the zeroes are just to represent the magnitude of how many people are in N.Y. But the zeroes can be significant if they are the result of precise measurements. A vinculum over the least significant zero is often used.
The significant figures in a number in scientific notation is the number of digits in the mantissa. The number 4×105 has only one digit in the mantissa, so it has one significant figure. 9.344×105 has 4 significant figures. Thus the number 1200 which is unclear as to how many significant figures it has is more clearly expressed as 1.200×103 as having 4 significant figures or as 1.2×103 as having 2.

When calculating with significant figures,
an answer cannot be more precise than the least precise measurement.

This means for...

It is very important to round rather than truncate your results: [pi] [approximately equal to]3.1416 not [pi] [approximately equal to]3.1415. You are often instructed to round to so many significant digits or to such and such a level of precision. There are variations, but the standard rule would round anything from $0.50 up to $1.49 all to $1. One variation would round $0.50$ down and $1.50$ up based on the evenness/oddness of destination digit. A common mistake to be avoided is "double rounding," for example, rounding 1.46 first to 1.5 and then to 2. More on that will be discuss in the Introduction to Statistics, lesson 3.

Various Common Units

The National Institute of Standards and Technology, formerly the National Bureau of Standards, is our nation's official source of standard weights and measures, as well as other standards, such as for programming languages. The metric system (Système International or SI) has a long, interesting history and is in use the world over. A notable exception is in common (non-scientific) uses in the United States. SI differentiates between basic and derive units and hence is often called the MKS system for meter, kilogram, second, the fundamental three of the seven basic units. Listed below is a hodge-podge of units and the most important conversions.
  1. English units of volume:
    3 teaspoons=1 tablespoon (useful for child medicine dosage, not just cooking)
    8 tablespoons per stick of butter–4 sticks per pound (Historically, a pound was cut in quarters.)
    2 cups per pint, 2 pints per quart, 4 quarts per gallon, 16 fl oz per pint (a pint's a pound the world round—works only for water. That is, a fluid ounce of water weights about a ounce.)
    231 cu in per gallon (US liquid—there are also Brit and US dry gallons)
    There are 160 Brit oz per Brit Gal., there are .9607594 Brit fluid oz per US fluid oz.
    There are 1.16 US liquid gallons per US dry gallon.   8 US dry gallons per bushel, 4 pecks per bushel.    42 US gallons per US petro barrel (31.5 US gallons per US liquid barrel). 2 US liquid barrels per hogshead.
    A cord is 4'x4'x8'—be sure to get that and not a third of that ("rick") when buying wood!
    Concrete is specified in cubic yards (27 cu ft per cu yard–why?).
    There are many more "English" units of volume, with a rich history but most are fortunately falling into disuse. I have never had to use: Grains, Scruple (20 grains), Minim (20 scruples), Drachm/Dram (60 minims; 1/8 or 1/16 oz), Gill (5 Brit oz), Bucket (4 Brit gallons), Firkins (9 Brit gallons), Bag (3 bushels), Seam (8 bushels), or Butt (2-4? barrels or 2 hogsheads). Since fresh water on ships was stored in a butt, and people congregated and gossiped there, the term scuttlebutt now refers to gossip, not just the fountain!

    Note: 33.8 ml/fl oz and 3.785 liters per gallon are useful crossovers to metric.

  2. Common "English" units of weight include: caret (200 mg), ounce (12 apothecaries/troy or 16 avoirdupois per pound!), pound, and ton (2000 pounds per short ton, 2240 pounds per long ton, 2204 pounds per metric ton). Mostly fallen into disuse are: pennyweight (20 per troy oz), slug (32.174 avdp. pounds), hundredweight (20 per ton). Pounds are, of course, abbreviated as lb!

    28.349523 grams per ounce and 2.2 pounds per kilogram are useful crossovers to metric.

  3. Common units of time are: the pico-, nano-, micro-, milli-, seconds. There are 60 seconds per minute (angle or time!), 60 minutes per hour (or degree), 24 hours per day, 7 days per week, 14 days make a fortnight, 365.24 days per year more or less. There are sidereal, calendar, and tropical years as well as calendar and lunar months. We also speak of decades, centuries, millenia, age of the earth (4.5 billion years), or universe (about 14 billion years=a Hubble time).

    NIST is responsible for defining the second, currently via the cesium fountain clock and cooperates internationally to generate world time known as Coordinated Universal Time (UCT). However, the US Navy is responsible for maintaining and distributing this time and uses several dozen cesium clocks and about one dozen hydrogen masers to do this. They are researching the use of a cesium fountain clock to help stabilize and steer the hydrogen masers. The second is metric. The 21st century/3rd millennium started January 1, 2001. Also, the designations 12 am (technically noon, Chicago style midnight) and 12 pm should not be used.

  4. You are responsible to know and understand the metric prefixes of: giga, mega, kilo, milli, micro, nano, and pico. You should be very aware that giga(G), mega(M), and kilo(K) can have slightly different meanings especially when used in a computer related context. There K refers not to 1000, but to 1024 or 210. M might refer to 1,000,000; 1,024,000 (3.5" floppies!); or 1048576=220. G might refer to 1,000,000,000; 1,073,741,824=230; or possibly some number in between! The terms Kibi(Ki), Mebi(Mi), Gibi(Gi) have been suggested.

  5. Common "English" units of length include the inch, foot (12 inches per foot), yard (36 inches per yard), mile (5280 feet per statute mile—a nautical mile is about 6076 feet (Int) or 6080 feet (Brit)). My father still speaks in rods (16.5 feet), which is also a pole or perch. Physicists speak of lightyears (5.8785x1012 miles or 9.46x1012 km). This is the distance light travels in one year. Light travels at 299,792,458 meters per second (3x108 m/s). This value is c. Hands (4"), mil (.001"), and points (about 1/72") are still commonly used. Falling into disuse are furlongs (8 per mile), leagues (3 naut. miles), fathom (6 feet), chains (80+/- per mile), and cables (720 feet).

    2.54 cm/in= 39.37 inches per meter and 1.609 km per mile or .62 miles per km
    are more useful crossovers to metric.

    Feet are often abbreviated as single quotes and inches as double quotes. (I am 5'6".) These same quote symbols are used for angle measurement in minutes, seconds, and thirds. (A right angle is 90°0'0"0'''.)

Unit Conversions

Converting from one type of unit to another is a common occurance in science. It is just another incidence of multiplying by our multiplicative identity (1)! For example, to convert 0.62 miles into feet we multiply by the identity 5280 feet/1 mile. The units of miles in the numerator and demominator cancel and we are left with 3273.6 feet. (More than 3 significant figures were retained, since 5280 is an exact value.) Two additional and useful conversions are given below as further examples.

BACK HOMEWORK ACTIVITY CONTINUE