Transportation agencies report that millions of crashes are caused by poor road conditions every year, which makes the localization of roadway anomalies extremely important. Common methods of road condition evaluation require special types of equipment that are usually expensive and time-consuming. Therefore, the use of smartphones has become a potential alternative. However, differences in the sensitivity of their inertial sensors and their sample rate can result in measurement inconsistencies. This study validated those inconsistencies by using three different types of smartphones to collect data from the traversal of both a paved and an unpaved road. Three calibration methods were used including the reference-mean, reference-maximum, and reference-road-type methods. Statistical testing under identical conditions of device mounting using the same vehicle revealed that the roughness indices derived from each device and road type are normally distributed with unequal means. Consequently, applying a calibration coefficient to equalize the means of the distributions of roughness indices produced from any device using the reference mean method resulted in consistent measurements for both road types.