Fiber Quality Metrics: What Each Number Means and Where It Breaks the Link
The physical-layer and signal-quality parameters that decide whether a coherent link runs error-free — attenuation, OSNR, Q-factor, BER, dispersion, and PMD — with the formulas behind them and the thresholds where each one stops a service.
Introduction
A coherent 400G channel that carried traffic cleanly for two years starts throwing post-FEC errors one afternoon, and the OSNR reads 21 dB — comfortably inside spec. The number that moved was differential group delay, spiking from 2 ps to 8 ps on a stressed splice. No single metric would have caught it; the link health lived in the relationship between several of them. That is the working reality of fiber quality metrics: they are not a checklist of independent readings but a connected set, where attenuation feeds the power budget, the power budget and amplifier noise set the OSNR, OSNR sets the Q-factor, and Q-factor sets the bit error rate that forward error correction then has to clean up.
This guide walks the full chain in the order signals actually traverse it. It separates the physical-layer parameters that describe the fiber and components from the signal-quality parameters that describe what the receiver sees, gives the formula and the evidentiary class for every number, and marks the threshold where each metric turns from a margin into a fault. The four calculators near the end let you move the inputs and watch the thresholds move with them. The treatment assumes coherent transmission as the default — which changes how dispersion and PMD behave compared with the direct-detection era — while still explaining the foundations an engineer entering the field needs.
Physical-layer metrics: the fiber and the components
These parameters describe the medium and the parts, independent of the signal riding on them. They are what a fiber characterization report measures and what a link-budget spreadsheet sums.
Attenuation
Signal power falls along the fiber from two mechanisms: material absorption, where silica and residual impurities convert light to heat, and Rayleigh scattering, where sub-wavelength density variations frozen into the glass redirect light out of the guided mode. Rayleigh dominates in the low-loss windows and scales as 1/λ4, which is why 1550 nm (roughly 0.18–0.22 dB/km on G.652.D) is quieter than 1310 nm (roughly 0.32–0.35 dB/km). The loss is linear in length, so it adds predictably across a span.
α is the fiber loss coefficient in dB/km; L is span length in km. Connector and splice losses add on top as discrete terms.
Attenuation reads as a budget, not a penalty: every 3 dB lost halves the received power, and the running total against the transmit power and receiver sensitivity is what sets reach. The same accounting underlies the treatment of fiber attenuation and its component losses.
Chromatic dispersion
The refractive index of silica depends on wavelength, so the spectral components of a modulated signal travel at slightly different group velocities and the pulse spreads in time. The coefficient for G.652 fiber at 1550 nm is about 17 ps/(nm·km), and it accumulates linearly with distance.
D is the dispersion coefficient in ps/(nm·km); L is length in km. A 100 km G.652 span accumulates about 1700 ps/nm at 1550 nm.
The mechanism that turns accumulated CD into a penalty is the spread relative to the symbol period, so the tolerance scales as the inverse square of the baud rate: quadruple the rate and the dispersion tolerance drops by roughly 16×. For a chirp-free 10G NRZ signal the 1-dB-penalty tolerance is about 1200–1400 ps/nm, which is why uncompensated direct-detection 10G runs out near 60–80 km. The boundary moved with coherent detection: a coherent DSP estimates and reverses accumulated CD electronically, routinely handling many thousands of ps/nm, so for modern links CD is a DSP setting rather than a reach limit. It still matters as a parameter the receiver must track and as a contributor to nonlinear interactions.
Polarization mode dispersion
A real fiber is slightly birefringent: its two polarization axes have marginally different group velocities, so energy split between them arrives with a differential group delay. Because the birefringence varies randomly along the fiber and drifts with temperature and stress, PMD is a statistical quantity that grows with the square root of length, not linearly.
DPMD is the PMD coefficient in ps/√km; L is length in km. Modern fiber is well below 0.1 ps/√km; older plant can be far worse.
The square-root scaling and the statistical spread are what make PMD dangerous: a link can sit at a safe mean for months and then a temperature swing pushes the instantaneous DGD into a tail that closes the eye. First-order PMD is compensated in coherent DSP, which is why high-baud coherent systems tolerate DGD that would have been fatal to 40G direct detection, but the compensation has a limit and second-order PMD is harder to undo.
Optical return loss and reflectance
Back-reflections from connectors, mechanical splices, and fiber ends send power toward the source, where it degrades the transmitter and, in coherent systems, raises in-band noise. Two sign conventions describe the same physics and are easy to confuse. Reflectance is the reflected fraction at a single interface, expressed as a negative number (an APC connector is around −60 dB or better; an open glass-air PC face is about −14.5 dB). Optical return loss is the aggregate reflected power for the whole link, expressed as a positive magnitude you want large — above 27 dB for a working link, and 45 dB or better for coherent systems, which is why angled-physical-contact connectors are standard there. The distinction is laid out in the comparison of return loss and insertion loss.
Takeaway: Physical-layer metrics add up as a budget you can compute before the link exists: attenuation in dB, dispersion in ps/nm, PMD in ps, reflectance and ORL in dB. Coherent DSP has turned CD and first-order PMD from hard reach limits into managed parameters, but the numbers still have to be tracked because they bound the DSP and feed the nonlinear and noise budgets.
Signal-quality metrics: what the receiver sees
These parameters describe the signal after it has crossed the fiber and the amplifiers. They are measured at or near the receiver and they decide whether the data decodes.
OSNR — the master noise metric
Optical signal-to-noise ratio is the ratio of channel power to amplified-spontaneous-emission noise power in a fixed reference bandwidth, conventionally 0.1 nm (12.5 GHz at 1550 nm). Every optical amplifier adds ASE, and across a chain of identical spans the noise accumulates as 10·log10(N): doubling the number of spans costs 3 dB of OSNR, independent of how good each amplifier is.
Pch is launch power per channel in dBm; Lspan is per-span loss in dB; NF is amplifier noise figure in dB; N is the number of identical spans. The constant 58 dB sets the 0.1 nm reference at 1550 nm.
The formula names every lever: more launch power helps until fiber nonlinearity erases the gain, lower span loss and lower noise figure help directly, and the span count punishes you logarithmically. A measured 400G coherent module at its FEC threshold reports a receiver OSNR around 20–21 dB (a field-measured value), which is why amplifier spacing and count are the first things an OSNR budget fixes. The choice of per-channel versus composite power in this calculation is a common source of error, covered in the note on composite versus per-channel power.
Q-factor
The Q-factor is the eye opening normalized by noise: the separation between the mean levels for a one and a zero divided by the sum of their noise standard deviations. It folds amplitude, noise, and distortion into a single number that maps directly to error probability.
I1, I0 are mean levels for marks and spaces; σ1, σ0 are their noise standard deviations. Often quoted in dB as 20·log10(Q).
Because Q is an amplitude-domain signal-to-noise ratio, a power penalty of X dB lowers it by a factor 10−X/20. That is the right way to fold in an impairment such as a poor extinction ratio — not to subtract it from a noise margin. Techniques for improving Q-factor work by attacking exactly these terms.
Bit error rate
BER is the ground truth: the fraction of received bits decided wrongly. For Gaussian noise it follows the Q-factor through the complementary error function, with no fitting constant.
Exact for additive Gaussian noise. Q = 6 gives BER ≈ 1×10−9; Q = 7 gives BER ≈ 1.3×10−12.
Two BER numbers matter on a coherent link: pre-FEC BER, measured before error correction and allowed to be high (the FEC threshold for an OpenZR+ oFEC engine sits near 2×10−2), and post-FEC BER, which must be effectively zero for error-free service. The gap between them is the coding gain, discussed below. The interplay of OSNR, BER, and Q is set out in the reference on OSNR, BER, and Q-factor.
Extinction ratio and received power
Extinction ratio is the power contrast between a one and a zero. A finite ratio leaves residual power in the zero level, which closes the eye and costs a fixed power penalty.
r is the linear extinction ratio (P1/P0). A 10 dB ER (r = 10) costs about 0.87 dB; a 6 dB ER costs about 1.65 dB.
Received power has its own two-sided bound: below the receiver sensitivity the signal is too weak to decide, and above the overload point the receiver saturates and the error rate climbs again. A coherent receiver typically wants its input within a window such as −18 to −10 dBm; the lower edge is the sensitivity, the upper edge is overload.
Watch forAn apparent gain at a coherent receiver — received power higher than expected — is as much a fault as low power. Overload distorts the constellation and raises BER, and a high-reflectance event upstream can push power back into the transmitter. Treat the power window as a band with two walls, not a floor.
How the metrics connect
The parameters above are not independent gauges; they are links in one causal chain. Attenuation and amplifier noise figure set the OSNR through the span formula. OSNR, together with the modulation format and any linear or nonlinear penalties, sets the effective Q-factor. Q sets the pre-FEC BER through the erfc relation. Forward error correction then converts a tolerable pre-FEC BER into a post-FEC BER of effectively zero. Break any link in that chain and the symptom appears at the end as errors, which is why diagnosis means walking back up the chain rather than reading one meter.
What FEC buys, in decibels
Forward error correction adds parity to the data stream so the receiver can correct errors without retransmission — essential on long links where round-trip delay makes retransmission useless. Its value is quoted as net coding gain: the OSNR or Q improvement at a fixed output BER, which is not the same as the overhead percentage. Modern coherent FEC delivers a large gain for a modest overhead, and the industry has converged on a few interoperable codes.
| Code | Overhead | Net coding gain | Where it is used |
|---|---|---|---|
| GFEC — RS(255,239) | ~6.7% | ~6.2 dB | Legacy 10G OTN (ITU-T G.709) |
| CFEC (concatenated) | ~15% | ~10.8 dB | OIF 400ZR coherent pluggables |
| oFEC (block turbo) | ~15% | ~11 dB | OpenZR+ and Open ROADM, extended reach |
| Proprietary SD-LDPC | ~20–25% | 11–12+ dB | Vendor long-haul and submarine |
In one lineNet coding gain is the metric that matters, not overhead. An oFEC engine turns a pre-FEC BER near 2×10−2 into a post-FEC BER below 10−15 for roughly 15% overhead and under 3 µs of added latency.
A 600G channel shows a rising post-FEC error count. Post-FEC errors mean the pre-FEC BER has crossed the code threshold, so the first check is pre-FEC BER against the threshold — it reads 3×10−2, above the engine’s 2×10−2. That points to a collapsed Q, so OSNR is next: it has fallen from 27 dB to 24 dB. The OSNR drop traces to a span whose amplifier gain sagged after a pump aged, raising the effective noise figure. The fix is at the amplifier, three links up the chain from the symptom. The post-FEC errors were real, but they were the last domino, not the cause.
Where each metric breaks the link
Every parameter has a band where it is a margin and a threshold where it becomes a fault. The bands below are typical for coherent systems and shift with baud rate, modulation order, and FEC strength — a 64QAM channel needs far more OSNR than QPSK at the same baud rate, on the order of 14 dB more to go from four to twelve bits per symbol.
| Metric | Healthy | Marginal | Fault |
|---|---|---|---|
| OSNR (0.1 nm) | > 5 dB above required | 0–5 dB above required | below required |
| Q-factor | > 7 (16.9 dB) | 6–7 | < 6 |
| Pre-FEC BER | < 1/10 of threshold | up to threshold | above threshold |
| Post-FEC BER | 0 | — | any nonzero |
| Received power | mid-window | near sensitivity or overload | outside window |
Attenuation faults
Excess loss erodes the OSNR through the span term and the received power directly. A dirty connector adding 0.8 dB or a degraded splice adding 0.3 dB looks trivial alone, but several of them across a route quietly consume the margin a designer set aside, and the link fails on the day a repair adds one more. The mechanism is cumulative and the boundary is the OSNR budget, not any single component.
OSNR degradation
OSNR falls as spans are added (10·log10(N)) and as any amplifier’s noise figure rises with age. The boundary is the required OSNR for the chosen format and FEC; cross it and pre-FEC BER climbs past the code threshold and post-FEC errors appear. This is the most common reason a long chain cannot support a higher-order modulation that a short one can. Real-time OSNR, Q, and BER trending is the basis of predictive maintenance approaches that catch the slope before it crosses.
Dispersion and PMD faults
For direct detection, uncompensated CD closes the eye once broadening approaches the bit period, and the penalty rises steeply with distance. For coherent systems the DSP absorbs static CD and first-order PMD, so the fault mode shifts: it appears when accumulated CD exceeds the DSP’s configured range, or when DGD spikes into a statistical tail beyond the equalizer’s tracking. PMD’s intermittency is the trap — a link inside its mean PMD budget can still fail on the rare high-DGD event. Both impairments are tracked per channel in modern monitoring, as described in DWDM channel monitoring with OCM and OSA.
Service-affectingA fiber cut takes everything; severe OSNR collapse below the required value drives pre-FEC BER past the code threshold within seconds; a DGD spike on a stressed splice can do the same intermittently. These are the conditions that justify protection paths and sub-50 ms switching, not slow maintenance windows.
Fixing the impairments
Each impairment has a primary remedy that attacks the mechanism and a secondary one that buys margin. The right choice depends on whether the system is direct-detection or coherent, because coherent DSP has absorbed several jobs that used to need dedicated optics.
Dispersion: optical versus electronic
Dispersion-compensating fiber carries a large negative dispersion (roughly −80 to −150 ps/(nm·km)) to cancel the accumulated CD of the transmission fiber. It is passive and broadband, but it adds 5–10 dB of insertion loss and is fixed once installed — loss that then has to be made up with more amplification, which costs OSNR. Electronic dispersion compensation in a coherent DSP reverses CD in the digital domain at no optical loss, adapts to the link automatically, and handles many thousands of ps/nm, which is why new builds remove DCF entirely. The trade is power and a couple of microseconds of processing latency.
OSNR: lower noise or shorter spans
A lower-noise-figure EDFA (4–5 dB single-stage, 3–4 dB dual-stage) raises OSNR directly through the NF term. Distributed Raman amplification uses the transmission fiber itself as the gain medium, so the signal is amplified before it has decayed to its noisiest point; the effective noise-figure improvement is several dB, at the cost of high pump power and added complexity. The boundary is the nonlinear threshold: pushing launch power for OSNR eventually trades ASE noise for nonlinear noise, so the optimum is a balance, not a maximum. Raman’s reach benefit is treated in the discussion of the clean-fiber zone in Raman links, and it is a defining feature of repeatered submarine systems.
Errors: stronger FEC
Where OSNR cannot be raised further, a stronger code extends reach by tolerating a worse pre-FEC BER. Moving from legacy GFEC (~6 dB net coding gain) to oFEC (~11 dB) is worth roughly 5 dB of system margin — equivalent to several extra spans or a step up in modulation order — for about 15% overhead. The catch is the framing overhead and the latency, both of which scale with code strength.
| Impairment | Primary remedy | Cost / boundary |
|---|---|---|
| High attenuation | Add amplification; clean and re-terminate | Each amplifier adds ASE; nonlinear limit on power |
| Chromatic dispersion | Coherent DSP (EDC) | DSP power and latency; DCF adds loss if used |
| PMD / DGD | Low-PMD fiber; coherent equalization | Second-order PMD and DGD tails remain |
| Low OSNR | Lower-NF or Raman amplification; fewer spans | Pump power, cost, nonlinear threshold |
| High pre-FEC BER | Stronger FEC (oFEC, SD-LDPC) | Overhead and latency |
| Nonlinear noise | Optimize (not maximize) launch power | Trades against OSNR; large-Aeff fiber helps |
Takeaway: In a coherent system the first questions are noise figure, span count, and FEC strength — not dispersion compensation, which the DSP now handles. The order of attack follows the causal chain: fix OSNR where you can, then let the code convert the remaining pre-FEC BER into error-free service.
Designing to the budget
A link design is two budgets reconciled: a power budget that gets the signal to the receiver above sensitivity, and an OSNR budget that keeps it decodable after the amplifier chain. Both are computed before any fiber is lit, and both carry an explicit margin for aging, repairs, and temperature.
The power budget
Sum every loss term and check it against the difference between launch power and receiver sensitivity, with margin held back.
Each term in dB. The total must not exceed Ptx − Prx,sens. Hold ≥ 3 dB of margin for end-of-life conditions.
Over 250 km of G.652 at 0.22 dB/km the fiber contributes 55 dB. Six connectors at 0.5 dB add 3 dB, twenty-five splices at 0.1 dB add 2.5 dB, and a mux/demux pair at 3 dB each adds 6 dB. With 3 dB of design margin the total is 69.5 dB. That figure decides how the amplifier chain is laid out — here, three spans near 83 km, each amplifier set for about 20 dB of gain to recover its span loss.
The OSNR budget
Apply the span formula across the chain and confirm the receiver OSNR clears the required value for the chosen format and FEC, with margin. Three spans of 83 km at 0.22 dB/km is about 18 dB of span loss; with 0 dBm per channel and a 4.5 dB noise figure the OSNR lands near 58 + 0 − 18 − 4.5 − 10·log10(3) ≈ 30.7 dB, comfortably above a QPSK requirement and leaving room for a higher-order format if capacity demands it later. The broader parameter set that a full design reconciles — OSNR, GOSNR, Q-margin, CD, PMD, PDL, and nonlinear noise — is laid out in the parameters of DWDM link design, and the wider engineering trade-offs in optical system design considerations.
Verification, in order
Acceptance follows the same chain the signal does. Capture an OTDR baseline and verify span and event losses; measure received power at every monitored point against the window; measure OSNR per channel against the budget; run an extended BER test and record Q; confirm pre-FEC BER sits well below the code threshold; and test protection switching before the link carries revenue traffic. The principle of continuous optical performance monitoring is to keep watching those same numbers and alarm at a fraction of the limit, not at the limit itself.
Design disciplineDesign for end-of-life, not first-light: hold at least 3 dB of OSNR margin, account for every connector and splice in the loss budget, and verify both directions independently. The cost of adequate margin at design time is far below the cost of a service outage in operation.
Interactive toolkit
Four calculators built on the formulas above. The OSNR tool uses the canonical span chain and shows margin against a format target; the dispersion tool uses a modulation-limited source width so the reach limit scales correctly as 1/B2 for direct detection; the Q-to-BER tool uses the exact erfc relation and the OOK extinction-ratio penalty; and the reach tool is a loss-limited power budget. Move the inputs and the result bands move with them.
The dispersion calculator models direct detection, where CD is a reach limit. Coherent receivers compensate CD in DSP and are not bound by these curves; the tool is there to show why the direct-detection era needed compensation and why higher baud rates hit the wall so much sooner.
In the field
An existing 60 km metro route ran four 10G channels through legacy dispersion-compensating modules, and the OSNR sat at 18 dB — marginal for any upgrade, with the DCF modules alone burning about 8 dB. Replacing the transponders with 100G coherent units removed the DCF entirely, since the DSP compensates CD: that single change recovered the 8 dB. Re-splicing three poor joints and re-terminating connectors recovered another 1.8 dB, and lower-noise-figure amplifiers (4 dB versus 5.5 dB) added more. OSNR rose to 24 dB, Q from 5.8 to 7.2, and the route now carries eight 100G channels — double the capacity — with margin to spare. The lesson is that coherent detection makes optical dispersion compensation obsolete, and the loss it removes is often worth more than the new optics cost.
A 400 km repeatered cable threw post-FEC errors several times a day on two of sixteen channels after eighteen months of clean operation. OSNR on every channel read 21–22 dB, within spec, so the noise budget was not the cause. PMD monitoring found the answer: DGD on the affected channels spiked intermittently to 8 ps against a normal 2 ps, traced to a polarization-sensitive splice near the 250 km point under mechanical stress. Re-splicing with proper alignment and strain relief stabilized DGD below 2.5 ps and the errors stopped. PMD’s intermittency and wavelength dependence are exactly why a single OSNR reading cannot clear a link — the fault lived in a parameter that looked fine on average.
Three sites needed eight 100G links under 2 ms latency at five-nines availability, with the longest hop 40 km. A power-budget check showed the spans were short enough to run unamplified: with a healthy launch power, a coherent receiver sensitivity, and 0.22 dB/km fiber, the loss-limited reach cleared 40 km with large margin, and the measured OSNR settled around 36 dB. The design used 1+1 protection on diverse routes rather than extra amplification. Over the first year it logged one protection switch — a backhoe strike, caught and switched in 45 ms — and beat its availability target. Simple designs with honest margins outperform complex ones.
Main Points
- Fiber quality metrics form one causal chain: attenuation and amplifier noise set OSNR, OSNR sets Q-factor, Q sets BER, and FEC converts a tolerable pre-FEC BER into error-free service.
- OSNR over an identical-span chain follows OSNR ≈ 58 + Pch − Lspan − NF − 10·log10(N); doubling the span count costs a fixed 3 dB regardless of amplifier quality.
- BER follows Q exactly through ½·erfc(Q/√2): Q = 6 gives about 10−9, Q = 7 about 1.3×10−12. No fitting constant is involved.
- Chromatic dispersion tolerance scales as 1/B2; chirp-free 10G NRZ runs out near 1200–1400 ps/nm (about 60–80 km), but coherent DSP compensates many thousands of ps/nm and removes CD as a reach limit.
- PMD grows with √L and is statistical: a link inside its mean PMD budget can still fail on a rare high-DGD event, which is why intermittent errors point to PMD before OSNR.
- Net coding gain, not overhead, is the FEC figure of merit: oFEC delivers about 11 dB for roughly 15% overhead, worth several spans of reach.
- Return loss is a positive magnitude you want large (> 27 dB working, ≥ 45 dB coherent); reflectance is the negative per-interface figure (APC ≤ −60 dB). They describe the same physics with opposite signs.
- Received power is a window with two walls: below sensitivity the signal is undetectable, above overload the receiver saturates and BER rises again.
- Small losses compound: a few dirty connectors and poor splices quietly erode the OSNR budget until one repair tips a link into failure.
- Design for end-of-life with at least 3 dB of margin, verify both directions, and monitor OSNR, Q, BER, CD, and PMD together — no single metric tells the whole story.
References
- ITU-T G.652 — Characteristics of a Single-Mode Optical Fibre and Cable, ITU-T Study Group 15.
- ITU-T G.709 / OTN — Interfaces for the Optical Transport Network, ITU-T Study Group 15.
- OIF — 400ZR Implementation Agreement, Optical Internetworking Forum.
- OpenZR+ MSA — OpenZR+ Multi-Source Agreement Technical Specification, OpenZR+ MSA Group.
Sanjay Yadav, "Optical Network Communications: An Engineer's Perspective" — Bridge the Gap Between Theory and Practice in Optical Networking.
Developed by MapYourTech Team
For educational purposes in Optical Networking Communications Technologies
Note: This guide is based on industry standards, best practices, and real-world implementation experiences. Specific implementations may vary based on equipment vendors, network topology, and regulatory requirements. Always consult with qualified network engineers and follow vendor documentation for actual deployments.
Feedback Welcome: If you have any suggestions, corrections, or improvements to propose, please feel free to write to us at [email protected]
Optical Communications & Network Automation Expert | Author of 3 Books for Optical Engineers | Founder, MapYourTech
Optical networking engineer with nearly two decades of experience across DWDM, OTN, coherent optics, submarine systems, and cloud infrastructure. Founder of MapYourTech. Read full bio →
Follow on LinkedIn
