QA benchmarks for Realtime Transit data
Knowing the quality assurance (QA) benchmarks for Realtime (RT) data improves your experience with Google Transit. Meeting these benchmarks helps you serve high-quality transit information, which creates positive user experiences.
Essential quality metrics
To help maintain reliable Transit information for Google Transit users, there are 3 key quality metrics for RT data:
- Quality: What users need
- Stability: When users need it
- Coverage: Where users need it
We work hard to maintain strict adherence to the community-driven General Transit Format Specification (GTFS-RT) to optimize user experiences. If your information deviates from the GTFS-RT, the warning and blocking errors are listed in the validation report, which include:
- Inconsistencies with the underlying static feed
- Format and correctness of entities (syntax)
- Presence of required fields
- Time ranges matching expected values from the static feed
QA benchmark - Tolerances and expectations
- Tolerance for warnings and errors for percentage of affected entities is 2% and 0%, respectively. Additional details are available in the validation report.
- A regular, stable, and cyclical distribution for feed entities over time is expected, without spikes or drops in number of feed entities (e.g. number of vehicles).
- Service alerts require proper language, consistency, and cause and effect information.
Issues with tolerances and expectations can lead to incomplete or inaccurate data shown to users, which is often launch-blocking. Possible causes and fixes are mentioned in the report. For more details, review our support docs.
We require a consistent, weekly inflow of data feeds via data pushes and pulls to provide the best on-ground experience for users. Downtime is expected during setup and system improvements, but Realtime systems must remain stable and provide parsable data through the year (including weekends and public holidays).
QA benchmark - Server uptime errors
Google monitors incoming feeds for stability over 8-16 days. Errors in data formats and corrupt protobuffers are effectively failed pushes / pulls (e.g. no ASCII in production). From the daily validation report, we expect to see instances of 1% or below for the following types of server uptime errors:
- Protobuf errors (format and parsing)
- Fetcher errors
Several failed fetches in a row trigger bad user experiences and get flagged, block launches, and blacklist the feed.
QA benchmark - Feed age
Feed timestampsare used to compute Feed age, and are measured in seconds since epoch (UTC). A feed age shouldn’t be older than 90 seconds when we receive it. Feed age for Service Alerts can be up to 10 minutes.
Note: Feed age is different from a trip update timestamp.
Transit provides users with complete, on-ground information for their trips. Specifically, Transit aims to provide:
- Complete data for areas near city centers and high-traffic areas
- Updates on most trips from the static feed, especially busy and central routes (full and partial coverage)
- Accurate and up-to-date data though service alerts and Vehicle Positions wherever possible, in addition to trip updates
Follow these guidelines to make sure you meet data quality benchmarks and create the best experience for Transit users.
Validation Report - A critical resource for every RT integration