Oscilloscope Measurement Fundamentals

Many oscilloscope users take advantage of only a small fraction of the powerful features available to them. In addition, selecting the right measurement from a catalog of possibilities and accurately interpreting the results can lead to confusion and mistakes. This series of articles is intended to help users understand oscilloscope measurements more completely in order to avoid common pitfalls.
Digital storage oscilloscopes vary greatly among vendors in terms of form factor (stand-alone, PXI, VXI, PCI, etc), resolution (8-bit, 12-bit, 16-bit, etc), acquisition rates (1 MS/sec, 1 GS/sec, 40 GS/sec, etc), functionality (advanced triggering, deep memory, self-calibration, etc.), and more. One aspect that separates true oscilloscopes from most PC-based, modular digitizers is the ability to make measurements in hardware on an onboard processor. The available measurements also differ from one oscilloscope to another, although this paper will cover a large segment of them. In addition, the algorithms used to complete the measurements may differ slightly among vendors. This paper will focus on the measurements and algorithms used in ZTEC modular oscilloscopes, but most of these concepts are universal.
Oscilloscope measurements can be sorted into the following three categories:
- Vertical-Axis
- Horizontal-Axis
- Frequency Domain
Vertical-Axis Measurements
Vertical-axis measurements analyze the vertical component of the applied signal. These measurements most often describe a signal in terms of a voltage level. However, they can also correspond to current, power, or any other physical phenomena converted to voltage via a probe or transducer. Some common vertical-axis measurements include Amplitude, Peak-To-Peak, Average, and RMS measurements.
Vertical Resolution and Accuracy
The resolution and accuracy of an oscilloscope can affect measurements greatly, so its important to understand these limitations. An oscilloscope with an 8-bit analog-to-digital converter (ADC) has 28 (256) levels available while a 16-bit ADC has 216 (65536) levels. Thus, a 16-bit oscilloscope has 256 times more resolution than an 8-bit oscilloscope. Since only finite levels are available to represent the signal, there is a quantization error of 1 least significant bit (LSB). To find the minimum detectable voltage change (code width), divide the input range by the number of levels. Figure 1 depicts a 16-bit oscilloscope digitizing an 8 Vpp square wave with a 100 mV ripple voltage. In this case, the oscilloscopes code width is (10/65536) 150 uV which allows it to produce a good representation of both the large and small signals. An 8-bit oscilloscopes code width would be only (10/256) 39 mV, so it could not show the 100 mV component adequately. Changing the input range setting to 250 mVpp would improve the performance, reducing the code width to (0.25/256) 1 mV.

Figure 1: Signal with Large & Small Components
The dynamic range of an oscilloscope refers to how well the instrument can detect small signals in the presence of large signals and is expressed in decibels (dB). It is limited by the quantization error and all other noise sources such as background noise, distortion, spurious signals, etc. The equation for computing the dynamic range is:
Vmax is the maximum voltage that must be acquired and Vres is the minimum resolution that can be seen. A good rule of thumb is that every bit of resolution equals 6 dB of dynamic range. An 8-bit instruments theoretical maximum dynamic range is 48 dB, but it is significantly less once all limitations are considered.
Accuracy refers to the oscilloscopes ability to represent the true value of a signal. An oscilloscope with high resolution, does not necessarily translate into giving an accurate result. Accuracy and resolution are related though, because the achievable accuracy of an instrument is limited by the resolution of the ADC.
The factors that reduce the accuracy of an oscilloscope can be mostly lumped into high- and low-frequency errors. Noise is generally the cause of high-frequency errors, while low-frequency errors are caused by drift stemming from temperature, aging, bias currents, etc. High-frequency errors can usually be removed by oversampling and averaging. Low-frequency errors often require the calibration of the instrument, either internally or through a factory calibration.
Relative vs. Absolute Measurements
An oscilloscopes accuracy is often specified in terms of gain accuracy and offset accuracy. Gain accuracy is related to how well it handles high-frequency noise and can be called its relative accuracy. Offset accuracy is related to how well it handles the low-frequency issues and can be referred to as absolute accuracy. Figure 2 shows a real and measured 1 Vpp sine wave. Notice that the measured Amplitude is 0.99 V which equates to a gain error of 0.01 V or 1%. The measured signal is also offset 0.02 V for a 2% offset error.

Figure 2: Gain & Offset Errors
Vertical-axis measurements can either be relative or absolute in nature. Relative measurements compare two voltages within the same signal. Amplitude is an example of a relative measurement because it returns the difference between the high and low voltage. The Amplitude of a 1 Vpp sine wave will be the same when it is centered at zero or has an offset of 5 V. Therefore, relative measurements are unaffected by the offset error. Absolute measurements are a representation of their real-world value and are affected by gain and offset errors. The Average measurement is an example of an absolute measurement.
Amplitude vs. Peak-To-Peak
Two vertical-axis measurements that are often confused are Amplitude and Peak-To-Peak. This is understandable because they are identical for all types of signals, except a pulse signal. Figure 3 shows the difference between the Amplitude and Peak-To-Peak (PTPeak) for a pulse signal. Peak-To-Peak returns the difference between the extreme Maximum and Minimum values, while the Amplitude returns the difference between where the pulse settles at the top (High) and bottom (Low) of the signal. The other measurements shown--Rise Overshoot (ROV), Rise Preshoot (RPR), Fall Overshoot (FOV), and Fall Preshoot (FPR)--are only valid when measuring pulses.

Figure 3: Vertical Axis Measurements [1]
The measurements shown in Figure 3 are computed on the oscilloscope processor using a histogram. Figure 4 shows how the pulse signal in Figure 3 is represented in an 8-bit oscilloscope histogram. The samples are sorted into one of 256 bins, each corresponding to a voltage range. The algorithms simply look for the bit value with the most points for the Low and High measurements and the absolute largest and smallest bit values for the Maximum and Minimum. This allows for an extremely fast computation, but the measurements resolution is limited by the quantization error (1 LSB) of the ADC. The accuracy also suffers due to a single samples susceptibility to noise.

Figure 4: Histogram Processing of a Sine Wave
Root Mean Squared (RMS) & Average
The Direct Current (DC) RMS, Alternating Current (AC) RMS, and Average measurements are methods of characterizing the vertical level and power using the entire waveform.
The Average function is the mean vertical level of the entire captured waveform. It can be calculated by taking the sum of all of the voltage levels and dividing that by the number of points as shown:
![]()
The DC RMS and AC RMS measurements return the average power of the signal. The DC RMS returns the entire power contained within a signal including AC and DC components. This can also be described as the heating power when applied to a resistor. The AC RMS is used to characterize AC signals by subtracting out the DC power, leaving only the AC power component. The equations for the RMS measurements are as follows:


Figure 5 shows these results on a 4 Vpp square wave with 0.5 V of offset.

Figure 5: Average & RMS
All three of these measurements are capable of more accuracy than the Amplitude and Peak-To-Peak measurements described in the previous section. The reason for this is that every single point in the waveform is included in the calculation of the Average and RMS measurements. This naturally cancels out noise that may be present in the signal. Additionally, when measuring the Average or RMS values, the more points that are acquired in the waveform, the better the accuracy of the measurements become. The upper bound of the accuracy is determined by the number of bits in the onboard processor. Some oscilloscopes use a 16-bit processor, so these measurements are limited to 16 bits of resolution because the largest number that can be stored on the chip is 16-bits. However, the 64-bit processor on the ZT4611 modular oscilloscope allows users to attain up to 64 bits of resolution. The tradeoff for the higher accuracy is longer computations since more points must be analyzed.
When only a few cycles of a waveform is acquired, it becomes critical to acquire only the full cycles or otherwise the results contain an asymmetric error. Figure 6 shows the same signal as Figure 5 except that an additional (high) half cycle was acquired. The Average and RMS values are offset because of this.

Figure 6: Average & RMS with Partial Cycle
There are a few ways to avoid this circumstance. The best way is to acquire a longer waveform that includes many cycles so that the offset is effectively minimized. This method requires more time and more onboard memory to store the waveform. Another way to solve the problem is to make use of the Cycle RMS or Cycle Average measurements. These calculate the RMS and Average including only the points from the first cycle of the waveform. The third way to solve the problem would be to use a gated measurement. Gated measurements allow the user to choose the points that are included. This can be done by selecting a start and stop time or a start and stop point. Both the Gated By Time and Gated By Points methods require the user to know the period of the waveform to solve the problem shown in Figure 6.
Horizontal-Axis Measurements
Horizontal-axis measurements involve analyzing the horizontal time axis of the applied signal, and include measurements such as Period, Frequency, and Rise Time. The value returned is usually in time, but can also be expressed as a ratio, radians, or in Hertz.
Horizontal Resolution and Accuracy
The horizontal-axis resolution is limited by the sample rate of the onboard clock. A board with a 1 GS/sec acquisition rate can only achieve a time resolution of 1 / (1 GS/sec) = 1 nsec. Much like the vertical axis, the horizontal-axis accuracy can be reduced by high- and low-frequency errors.
High-frequency errors consist of clock jitter or phase-noise, but these are usually minute when considering that clocks used on most oscilloscopes have errors of 100 parts per million (ppm) or less. An error this small is insignificant when compared to the accuracy of the vertical axis. Occasionally, when completing horizontal-axis measurements, it may appear that clock jitter or phase-noise is causing incorrect readings. However, it is usually the lack of vertical-axis accuracy or noise that causes the incorrect time measurement. This will be further discussed later in the Edge Measurement section.
Low-frequency errors can be a problem and consist of drift associated with temperature, aging, etc. Annual factory calibrations must be completed to guarantee the accuracy of the clock over a long period of time.
Horizontal Waveform Measurements
The majority of the horizontal-axis measurements are fairly straight forward. They are shown in Figure 1. The Period measures the average time for a cycle to complete using the entire waveform in the capture window. The Frequency is the inverse of the period and is measured in Hertz. The Positive Pulse Width measures the time from the first rising edge to the first falling edge, while the Negative Pulse Width does the opposite. The Positive and Negative Duty Cycles are then calculated by taking the ratio of their corresponding Pulse Widths to the Period. All of these measurements are calculated based on the Middle voltage level which is simply halfway between the High and Low values. The time of the first maximum and minimum levels can also be retrieved using the Time of Maximum and Time of Minimum measurements.

Figure 1: Horizontal-Axis Measurements
When acquiring Period and Frequency measurements their accuracy can be very much affected by the sample rate. Both of these measurements are calculated by counting the number of samples that occur between Middle crossings. If a 10 MHz signal is being sampled at 100 MHz, this will result in exactly ten samples per period. The samples at the zero crossings may be very near the borders. If one is missed, this results in only nine samples being detected which returns a Period of 9 * (10nsec) = 90 nsec and a resulting Frequency of 11.1 MHz. This resolution is obviously not very good. It could be improved by acquiring long waveforms to capture many cycles and average out the resolution error. Another solution would be to sample the signal at 1 GHz or greater. Overall, for more accurate Frequency and Period measurements, it is best to sample at a far greater rate than the signal and capture many cycles. Cycle Average and Cycle Frequency measurements can be used to measure only the first cycle if desired. Also, the gated methods described in the vertical-axis section can also be employed. All of these methods are still susceptible to the resolution errors described above.
Phase measurements make most sense when acquiring two or more waveforms to determine how many radians or degrees a waveform is shifted in relation to another. However, the phase can be measured on a single periodic signal. This can be confusing, but it is simply calculated by comparing the starting point of the waveform to the rising edge Middle crossing. Figure 2 shows one signal with a positive 90 degree (1.57 rad) phase shift and another with a 270 degree (4.72 rad) phase shift.

Figure 2: Phase Measurement
Edge Measurements
A subset of Horizontal-Axis measurements is Edge Measurements. All of these measurements are made in relation to the Reference High (REF HIGH), Reference Middle (REF MID), and Reference Low (REF LOW). These references are user-selectable and are different than the High, Middle, and Low levels discussed in the previous sections, which are not user-selectable. By default, the REF HIGH, REF MID, and REF LOW are set to 90%, 50%, and 10% of the Amplitude (High Low). However, all of these percentages can be adjusted to suit the applications needs, or input in terms of absolute voltages.
With a firm understanding of the references, the meaning of the edge measurements becomes clear. They are shown in Figure 3. The Rise Time (RTIMe) measures the relative time for the leading edge of a pulse to rise from the REF LOW to the REF HIGH. The Fall Time (FTIMe) measures the same thing on the falling edge. The Rise Crossing Time (RTCRoss) is the absolute time when the waveform rises above the REF MID, measured from the start of the waveform. The Fall Crossing Time (FTCRoss) measures the same thing on the falling edge. All four of these measurements are edge selectable, meaning that the user can choose which number edge to characterize within the capture window.

Figure 3: Edge Measurements
One possible problem when taking edge measurements are inaccurate crossings due to noise on the vertical axis. Figure 4 shows a signal with and without vertical noise and how that could affect a horizontal measurement. The noisy signal crosses the voltage thresholds at slightly different points than the smooth signal, causing a shorter Rise Time Measurement. Another problem with a noisy signal is the potential for false crossings. This occurs when noise causes a signal to dither near the crossing points in several recorded crossings. Both of these problems can be avoided by either oversampling and averaging or by using the Smooth function before taking the measurement to reduce the noise. The algorithms used on ZTEC oscilloscopes incorporate hysteresis at the crossings which helps avoid detecting false crossings. This does result in a minimal detectable edge, however.

Figure 4: Noisy & Smooth Rise Times
Relative vs. Absolute Measurements
Much like the distinction made between absolute and relative voltage measurements made in the vertical-axis section, there are absolute and relative time measurements as well. For example, the Period of a waveform compares two points on the same waveform, so its often unnecessary to relate this to a real-world or absolute time. Therefore, this is considered a relative time measurement. An example where the absolute time would be important is measuring the Time of Maximum (TMAX), which returns the timestamp of the first maximum voltage level in relation to the start of the acquisition.
Frequency Domain Measurements
Frequency domain measurements involve translating a time-domain waveform with a fast Fourier transform (FFT), and then measuring the noise and distortion characteristics in the frequency domain. Frequency domain measurements provide magnitude and phase characteristics versus frequency.
Frequency Resolution and Accuracy
Using the FFT to quickly transform a signal into its frequency components is powerful, because it reveals signal characteristics that cant be seen in the time-domain. The FFT used within ZTEC oscilloscopes returns complex IQ data which is then converted to magnitude and phase data. Figure 1 shows the result of calculating the FFT of a signal and a few of the measurements.

Figure 1: Frequency Domain Measurements
ZTEC oscilloscopes provide four FFT windows that can be applied as well. Windows are used to increase the spectral resolution in the frequency domain. The Rectangular Window provides the best frequency and worst magnitude resolution. It is almost the same as no window. The Blackman-Harris Window provides the best magnitude and worst frequency resolution. The Hamming Window provides better frequency and worse magnitude resolution than the Rectangular Window. It provides slightly better frequency resolution than the Hanning Window. The Hanning Window provides better frequency and worse magnitude resolution than the Rectangular Window.
Like some of the vertical- and horizontal-axis measurements discussed previously, the accuracy of the FFT can be improved by analyzing longer waveforms. Due to the nature of the calculations, the resolution is limited to half of the resolution of the onboard processor. In the case of the ZTEC ZT4611 oscilloscope, which uses a 64-bit processor, the accuracy would be limited to 32 bits of resolution. The FFT algorithm is binary in nature, so for the best performance it is wise to select a waveform size that is equal to 2N.
Frequency Domain Measurements
Once a signal has been converted to the frequency-domain, five valuable measurements can be performed as explained in the following paragraphs. All of these measurements assume that the input signal is a perfect single-frequency sine wave and that all other frequency components are assumed to be harmonics or noise. All except the ENOB (bits) are expressed in decibels relative to carrier (dBc). THD is the only negative value.
The Signal-to-Noise Ratio (SNR) is the ratio of the RMS amplitude of the fundamental frequency to the RMS amplitude of all non-harmonic noise sources. SNR does not include the first nine harmonics as noise. In Figure 1, the SNR would be computed by dividing the magnitude of the fundamental by the sum of the magnitudes of all of the other frequency components, excluding the 2nd through the 10th harmonics. SNR is commonly used when only the narrow-band around the fundamental frequency is of concern and the harmonics will not have an effect on the system under test.
The Total Harmonic Distortion (THD) is the ratio of the RMS amplitude of the sum of the first nine harmonics to the RMS amplitude of the fundamental. In Figure 1, this would be calculated by summing the magnitudes of the 2nd through the 10th harmonics and then dividing that by the fundamental magnitude. THD is a concern when using active components such as amplifiers and mixers where the harmonics need to be minimized to reduce distortion.
The Spurious-Free Dynamic Range (SFDR) is the ratio of the RMS amplitude of the fundamental to the RMS amplitude of the largest spurious signal. This spurious signal can be a harmonic or noise frequency component. In Figure 1, the SFDR would be computed by dividing the magnitude of the fundamental by the magnitude of the 2nd harmonic, since it is the largest spurious signal. SFDR is used when there is a dominant spurious signal in relation to the other noise and distortion components.
The Signal-to-Noise and Distortion (SINAD) is the ratio of the RMS amplitude of the fundamental to the RMS amplitude of the sum of all noise and distortion sources. This is equivalent to the sum of the SNR and THD. In Figure 1, this would be calculated by dividing the magnitude of the fundamental by the sum of the magnitudes of all of the other frequency components, including harmonics and noise. SINAD is used in broad-band applications where all harmonics and noise will affect the signal.
The Effective Number of Bits (ENOB) is another way of expressing SINAD. It provides a measure of the input signal dynamic range as if the signal were converted using an ideal ADC. For instance, the ENOB of an 8-bit oscilloscope is often somewhere in the 6-7 bit range due to the noise and distortion affecting the instrument. The ENOB is calculated using the following equation:
![]()
High-Speed ADC Test Example
The specifications and test procedures of a high-speed Analog to Digital Converters (ADCs) are generally expressed in the frequency domain. The frequency measurements on a ZTEC oscilloscope can be used to mimic a more expensive spectrum analyzer to complete these tests. One test that is often used is a two-tone or multi-tone distortion test. This is completed because intermodulation distortion can occur when the ADC samples a signal composed of more than one sine wave. Figure 2 shows the FFT of an acquired ADC data record undergoing a two-tone test. Once the FFT is created, measurements such as THD and SINAD can be used to characterize the performance of the ADC.

Figure 2: FFT of Two-Tone Distortion Test
This concludes "The Fundamentals of Oscilloscope Measurements". Hopefully, these articles have provided our readers with a little deeper understanding of the waveform measurements available from an oscilloscope. This understanding can help users leverage the power of oscilloscopes more effectively and avoid potential pitfalls.