Tag Archives: real time road traffic

Building an interactive journey time map from data.gov.hk Traffic Speed Map data with Google Maps API


Some practical applications for road traffic data processing are the subject covered in past installments here, including a traffic jam classifier using GPU deep learning on surveillance camera images and a mash-up of CCTV still image feed with Google Maps. Both are centered around the data source provided by data.gov.hk, which is an initiative to support big data development by disseminating government data of various type to the public through the Internet.

This installment will walk through another mash-up of Google Maps API for real-time interactive journey time map using Google Maps API. Simply put, it is an overlay of real-time road saturation level information on an interactive digital map.  A demo is available here. As a public demo, it is limited to routes from one of three local regions (Hong Kong Island), largely in consideration of the free daily Google API account quota. For personal use, Google’s free daily quota should be more than enough for most needs.rtm-21-colorcode

Work as well in the 3D Earth view.

Zoom in for a clean view of travel time in color-coded road sections with direction arrows of travel.

Traffic Speed Map data from data.gov.hk

The Transport Department here publishes road travel time data on data.gov.hk for selected road sections. It is not clear from that site the method of collections and collation of this average speed published. However it is envisaged that apart from traditional roadside vehicle sensors and automatic CCTV vehicle speed estimations, public transport operators like buses might have contributed with real time data from their fleet to improve the overall accuracy.

The page at data.gov.hk provided all the necessary information to consume this data feed, including URL to the XML Traffic Speed Map file (which is being updated every 2 minutes), the schema of the XML file, and some supporting documents with the coordinates of the route origin and destination.

As the XML schema suggested, the XML file is structured pretty straight forward. It includes all routes in one single file, under the element jtis_speedmap (JTIS is probably an acronym for Journey Time Indication System, which is a decade old infrastructure aiming at informing motorists, via giant dot matrix displays mounted on highway gantries, estimated travel time of certain routes). In this XML file, each route is represented by one jtis_speedmap node, and underneath this node carries data associated with that route, including the route record identifier LINK_ID.


Most of these data elements are self-explanatory. For more details, there is a separate PDF file describing the data elements in details. In addition, the table available in an Excel file included the origin and destination coordinates of each route.

Customary to the local authority’s geodetic practices here, the coordinates in this file are expressed in HK80 grid, therefore translation to Google Maps’ WGS84 system is required. Further details of the translation are explained in this previous post.

Google Maps Directions

Because the data from data.gov.hk only provided the start and end point of each route (indirectly from the accompanying documentations in PDF and Excel), it is not sufficient for color-coded plotting of road sections on digital map – a feature common to most standard GIS visualization application for road network monitoring. In fact, this plotting feature would not be possible without working with complex GIS data structure.

To address this data insufficiency issue, we are going to utilize a publicly available technology, the Google Maps Directions API. With this Google service, routing between two points on road network is as simple as providing the coordinates of the two endpoints. There are many options available from this service, but we are only interested in the “DRIVING” travel mode here for road network.

This route from Google Maps Directions, however, is not guaranteed to represent the exact route the Traffic Speed Map data meant to measure speed for, but should serve well as a reasonably accurate approximation, and can be further fine-tuned manually. For example, route 910-931 will result in a much longer route using Google’s route service than it is more likely to be, as shown in below the “as-is” route from the XML data (left) and the fine-tuned result by making judgement call after assessing the road network (right):

It should be noted that even after fine tuning, there are still some “interesting” routes. But without further information like waypoints from the data provider (the local Transport Department) to resolve these ambiguities, we would settle with this for now.

To make calls to Google Maps API, a Google API Service account is required. In addition, a secret key has to be generated via the Google API Console. By embedding this secret key in any Javascript code, the Google Maps API will be ready for action. There is a good online collection of samples for reference.

Integrating Google Maps Directions with Traffic Speed Map data at data.gov.hk

To stick with the principle of keeping things simple, as in a previous article on Google Maps integration, the target is to keep everything inside one single Javascript HTML file.

The first step is to embed the static data from data.gov.hk into Javascript code. Those data that are not likely to change often are considered static, and in this case, the coordinates of the routes available on the PDF and Excel file previously mentioned. Using simple awk scripts, the following code fragment is generated for such data. The way the Traffic Speed Map file uses hyphens in the link ID necessitated some character substitution legwork during code generation, as this is used as variable names suffix and thus have to conform with Javascript variable naming conventions which prohibits hyphens.

// start dynamic data code generation

// -- route ID 722-50059
origin722_50059= new google.maps.LatLng(22.2859946618326,114.15524436017);
destination722_50059= new google.maps.LatLng(22.28686575617,114.153536622643);

// end dynamic data code generation


Next thing is to build the method to read the XML file available on data.gov.hk site. For security restrictions to safeguard cross domain issues on modern browsers, the XML file from the remote data.gov.hk server will be mirrored to my web server using cron jobs and monkeyed in order to prevent violation of browser restrictions.

wget -q http://resource.data.one.gov.hk/td/speedmap.xml -O speedmap.xml
gawk "NR==2{print \"<jtis_speedlist>\"}NR!=2{print}" speedmap.xml > speedmap_clean.xml


Adding the entry below to the crontab to schedule running the XML file mirroring every 2 minutes for best synchronization to the data update frequency of the Traffic Speed Map at data.gov.hk.

*/2 * * * * /home/ubuntu/syncrtm.sh


Now it is time for the XML request method.

	function readXML() {
		var xhttp = new XMLHttpRequest();
		xhttp.onreadystatechange = function() {
			if (this.readyState == 4 && this.status == 200) {
				liveDataXml = xhttp.responseXML;
		// use speedmap_clean for javascript xml parser CORS issues.
		xhttp.open("GET", "speedmap_clean.xml", true);


For supporting the extraction of specific data elements from the XML data source, some helper functions are prepared like extractSpeed() and extractSaturation(). These methods used XPath expression, as exemplified below, to retrieve values of interests in the XML.

var path = "/jtis_speedlist/jtis_speedmap[LINK_ID[text()='" + id + "']]/TRAFFIC_SPEED";


The TRAFFIC_SPEED value is extracted for displaying along with the LINK_ID as tooltip on each route markers on mouse over. The ROAD_SATURATION_LEVEL value is color coded to red, amber, and green respectively to render the route on the base map. At this point both the static data (i.e. the coordinates of routes) and the dynamic data (i.e. the XML file with speed and saturation) are ready. Plotting the route is simply by calling the Google Maps API “route” method from the class DirectionsService. The callback function holds the response object and will call another method in my implementation to render the routes and markers on the map. This is where our parsed data from the Traffic Speed Map XML file come into play – setting up the color code according to saturation and label the marker’s tooltip with speed information.

  origin: originLatLng,
  destination: destinationLatLng,
  travelMode: 'DRIVING'
}, function(response, status) {
    if (status === 'OK') { // Update the route on screen
        console.debug('Route service called and updated display');


As there will be quite a lot of routes to plot on the same map, and there are overlapping sections according to the Traffic Speed Map data, it certainly make sense to present information with higher value (congested road sections) without being covered by the less valued. A simple trick for this is to set the z-Index of the route to the inverse of its speed. Adding directional arrows along the route also helps to avoid route information being overlooked for overlapping routes as a result of rendering.

var zIndexInvSpeed = 1000 - parseInt(speed); 

directionsDisplay = new google.maps.DirectionsRenderer({
polylineOptions: {
 strokeColor: color,
 strokeWeight: 8,
 zIndex: zIndexInvSpeed,


Event listeners are attached to each of the markers for a simple filtering function. On clicking of a marker, the map will hide every other routes but the one clicked. This will allow some form of interaction for viewers to focus on specific route of interest from a road network full of color-coded road sections.

Another feature is to allow viewers to cycle through overlapping endpoints during filtering. Imagine a position where two routes joined up, that is, where the end of route A is also start of route B. On the map the markers as rendered by Google Maps will certainly overlap. Allowing user to cycle through by simply clicking away greatly improve viewers’ ability to drill down by filtering without being hindered by the overlapping problem. Shown below is an example of clicking on the end of route 930-931 which coincide with the start of route 931-4650. The overall filtering effect will be filtering out 930-931 first, and then filtering out 931-4650, and finally back to no filtering (show all).

The following screen shows the tooltip of a marker, with information of the route ID and the speed from the Traffic Speed Map XML data.

Color-coded road sections.

The core of the Javascript file is a loop that will update the map at specific interval according to the Traffic Speed Map XML data. Due to restrictions on request rating imposed by Google Maps API (which is a fair restriction), there are delays deliberately set in my implementation between each DirectionsService route call, even though this has to be done only once on startup for each route.

Finally the secret API key to calling Google Maps API must be included as the parameter to its library, along with the callback method.

<!-- Your GoogleMap API key goes here -->
<script async defer src="https://maps.googleapis.com/maps/api/js?key=YOUR_SECRET_KEY&callback=initMap">


Transportation and traffic analysis is one interest I am particularly fond of.  Studies on travel time has been a serious topic in the academics as well as the transportation industry, and is a very complex cross-disciplinary problem. For the general public, it is still fun to integrate your own free solution and in the way you want it to be.

Consolidating real time traffic CCTV data from data.gov.hk with Goolge Map API

Real time CCTV snapshot images from major route and highways in Hong Kong are provided online at data.gov.hk. Considering availability of data and related technology in the last decade, we have made a huge progress from obtaining traffic jam information from the radio to real time traffic CCTV images on the web.

Advancement in web technology not only enabled access of information, but also provided information in a manner flexible enough to easily build new view of content from multiple sources. The term “mashups” generally refer to this type of web application, and one such example presented here being visualizing road traffic data in near real time images on the web. A demo page is available on this site.

As explained in a previous article demonstrating a practical neural network application, CCTV images from the Hong Kong road network are already made available online. Together with Google Map API, an integration of the two will no doubt a very interesting fusion of technology.

Firstly, visit data.gov.hk for a glimpse of the traffic data available. Each CCTV provides image data on the web via a fixed URL.

The next thing to do is to integrate these data in Google Map API. In an attempt for a quick demonstration, a simple out-of-the-box application is adapted to retrieve data from data.gov.hk. These include the camera image, and the best of all, the geographical information from each of these images, i.e. the longitude and latitude. data.gov.hk provided a complete list of geographical data of its cameras, but since they are encoded in a format not directly understood by Google Map (HK80), a translation is required.

After confirming the position of each camera, a simple CSV file is created and the resulting Javascript is generated using an AWK script. There can’t be a method possibly be simpler than this when a prototype page with the Google API has been worked out. The below is captured from the resulting web page showing a selection of location of interest from all available CCTV images from data.gov.hk.


The map below in HTML created using Google Map API showing a complete list of CCTV location available at data.gov.hk. Clicking on each marker will open the real time image from the corresponding CCTV camera.hkdatagovcctv1

Excel is used to output a CSV data file for generating the Javascript file for use in the above example. The original file is in PDF but it is fairly easy to convert into Excel and then export from local coordinates in data.gov.hk (HK80) to something Google Map API can understand and render.hkdatagovcctv3

The layout of the CSV file is largely duplicated from the PDF file, and the line below is a sample. Most fields are self-explanatory, only the last three fields are additional to the data from the PDF file. Two of these are the converted coordinates for Google Map, and the last one is the default position of the CCTV image with respect to the marker. This can be either one of C,Q,W,A,S. For Q,W,A,S the image will be displayed on the corresponding quadrant as these characters are laid out on the keyboard, while C represent the display centered below the marker.

H106F,H106F.JPG,Connaught Road Central near Exchange Square,http://tdcctv.data.one.gov.hk/H106F.JPG,816365,834060,22.2861747,114.1554513,C

The AWK script below is used to loop through the CSV file to generate the part of the HTML content for each CCTV data.


	print "y0=" $7 ";x0=" $8;
	print "y1=y0+0.04;x1=x0+0.05;"
	print "var bounds" idx " = new google.maps.LatLngBounds(new google.maps.LatLng(y0,x0),new google.maps.LatLng(y1,x1));"
    print "var srcImage" idx " = '" $4 "';"
	print "var marker" idx " = new google.maps.Marker({position:new google.maps.LatLng(y0,x0),map:map,title: '" $1 "\\n" $3 "'});"
	print "var quadrant" idx " = \"C\";"
    print "overlay" idx " = new TrafficImageOverlay(bounds" idx ", srcImage" idx ", quadrant" idx ", map);"
	print "arr.push(overlay" idx ");"
	print "marker" idx ".addListener('click', function() {overlay" idx ".toggle();});"
	if ($9=="C") {
		print "overlay" idx ".toggle();"

Finally, the HTML file.

<!DOCTYPE html>
    <meta name="viewport" content="initial-scale=1.0, user-scalable=no">
    <meta charset="utf-8">
    <title>Hong Kong Real Time Road Traffic Camera Image</title>
      /* Always set the map height explicitly to define the size of the div
       * element that contains the map. */
      #map {
        height: 100%;
      /* Optional: Makes the sample page fill the window. */
      html, body {
        height: 100%;
        margin: 0;
        padding: 0;

    <script src="https://maps.googleapis.com/maps/api/js?key=YOUR_SECRET_GOOGLEMAP_KEY"></script>

    var overlay;
	var arr = [];
    TrafficImageOverlay.prototype = new google.maps.OverlayView();
    TrafficImageOverlay.prototype.toggle = function() {
      if (this.getMap()) {
		if (this.quadrant_=="A") {this.quadrant_="Q"; this.setMap(null); this.setMap(this.map_);}
		else if (this.quadrant_=="Q") {this.quadrant_="W"; this.setMap(null); this.setMap(this.map_);}
		else if (this.quadrant_=="W") {this.quadrant_="S"; this.setMap(null); this.setMap(this.map_);}
		else if (this.quadrant_=="S") {this.quadrant_="C"; this.setMap(null); this.setMap(this.map_);}
		else if (this.quadrant_=="C") {this.quadrant_="A"; this.setMap(null);}
      } else {

	TrafficImageOverlay.prototype.refreshImg = function() {
		try {
			this.img_.src=this.image_+'?rand=' + Math.random();
		catch (err) {

	function initMap() {
        var map = new google.maps.Map(document.getElementById('map'), {
			zoom: 12,
			center: {lat: 22.308, lng: 114.1368}

		var x0,x1,y0,y1;

		<!-- code block below is generated from awk script, showing only one block. One block for each CCTV data. -->

		var bounds1 = new google.maps.LatLngBounds(new google.maps.LatLng(y0,x0),new google.maps.LatLng(y1,x1));
        var srcImage1 = 'http://tdcctv.data.one.gov.hk/TC560F.JPG';
		var marker1 = new google.maps.Marker({position:new google.maps.LatLng(y0,x0),map:map,title: 'Tsing Ma Bridge'});
		var quadrant1 = "Q";
        overlay1 = new TrafficImageOverlay(bounds1, srcImage1, quadrant1, map);
		marker1.addListener('click', function() {overlay1.toggle();});

		<!-- end code block generated from awk script -->

    function TrafficImageOverlay(bounds, image, quadrant, map) {this.bounds_ = bounds;this.image_ = image;this.quadrant_ = quadrant;this.map_ = map;this.div_ = null;this.setMap(map);}
		TrafficImageOverlay.prototype.onAdd = function() {
        var div = document.createElement('div');div.style.borderStyle = 'none';div.style.borderWidth = '0px';div.style.position = 'absolute';
        var img = document.createElement('img');img.src = this.image_;img.style.width = '100%';img.style.height = '100%';
		img.style.position = 'absolute';div.appendChild(img);this.div_ = div;
		var panes = this.getPanes();panes.overlayLayer.appendChild(div);

	TrafficImageOverlay.prototype.draw = function() {
        var overlayProjection = this.getProjection();
        var sw = overlayProjection.fromLatLngToDivPixel(this.bounds_.getSouthWest());
        var ne = overlayProjection.fromLatLngToDivPixel(this.bounds_.getNorthEast());
        var div = this.div_;div.style.left = sw.x+'px';div.style.top=ne.y+'px';div.style.width=(ne.x-sw.x)+'px';div.style.height=(sw.y-ne.y)+'px';
		if (this.quadrant_=="Q") {
			div = this.div_;div.style.left = sw.x-(ne.x-sw.x)+'px';div.style.top=ne.y+'px';div.style.width=(ne.x-sw.x)+'px';div.style.height=(sw.y-ne.y)+'px';
		}else if (this.quadrant_=="W") {
			div = this.div_;div.style.left = sw.x+'px';div.style.top=ne.y+'px';div.style.width=(ne.x-sw.x)+'px';div.style.height=(sw.y-ne.y)+'px';
		}else if (this.quadrant_=="A") {
			div = this.div_;div.style.left = sw.x-(ne.x-sw.x)+'px';div.style.top=ne.y+(sw.y-ne.y)+'px';div.style.width=(ne.x-sw.x)+'px';div.style.height=(sw.y-ne.y)+'px';
		}else if (this.quadrant_=="S") {
			div = this.div_;div.style.left = sw.x+'px';div.style.top=ne.y+(sw.y-ne.y)+'px';div.style.width=(ne.x-sw.x)+'px';div.style.height=(sw.y-ne.y)+'px';
		}else if (this.quadrant_=="C") {
			div = this.div_;div.style.left = sw.x-(ne.x-sw.x)/2+'px';div.style.top=ne.y+(sw.y-ne.y)+'px';div.style.width=(ne.x-sw.x)+'px';div.style.height=(sw.y-ne.y)+'px';

	TrafficImageOverlay.prototype.onRemove = function() {
        this.div_.parentNode.removeChild(this.div_);this.div_ = null;

    google.maps.event.addDomListener(window, 'load', initMap);

	var refreshIdx = 0;
		function() {
			if (refreshIdx >= arr.length) {
				refreshIdx = 0;
			try {
			} catch (err) {
<div id="map"></div>