%md # [SDS-2.2-360-in-525-03: Geospatial Analytics and Big Data](https://lamastex.github.io/scalable-data-science/360-in-525/2018/03/) This needs to be edited for Spark 2.2 and for tight integration with other topics like: * geomesa * twitter geo-located tweets * etc. Students need to have seen: * scala crash course * Spark * DataFrames * rdds * graphX * ?
%md This is the 2016 version for Middle Earth for a starting point to shape around.... The [html source url](https://raw.githubusercontent.com/raazesh-sainudiin/scalable-data-science/master/db/week10/035_ScalableGeoSpatialComputing.html) of this databricks notebook and its recorded Uji : [](https://www.youtube.com/watch?v=0wKxVfeBQBc?rel=0&autoplay=1&modestbranding=1&start=0&end=753)
This is the 2016 version for Middle Earth for a starting point to shape around....
The html source url of this databricks notebook and its recorded Uji 
:
Last refresh: Never
%md # What is Geospatial Analytics? **(watch now 3 minutes and 23 seconds)**: [](https://www.youtube.com/v/1lF1oSjxMT4?rel=0&autoplay=1&modestbranding=1&start=111&end=314)
What is Geospatial Analytics?
(watch now 3 minutes and 23 seconds):
Last refresh: Never
%md # Some Concrete Examples of Scalable Geospatial Analytics ## 1. Let us check out cross-domain data fusion in MSR's Urban Computing Group * lots of interesting papers to read at [http://research.microsoft.com/en-us/projects/urbancomputing/](http://research.microsoft.com/en-us/projects/urbancomputing/). Here is an excerpt from the MSR Urban Computing link form above: <p align="justify"><em>Urban computing</em> is a process of acquisition, integration, and analysis of big and heterogeneous data generated by a diversity of sources in urban spaces, such as sensors, devices, vehicles, buildings, and human, to tackle the major issues that cities face, e.g. air pollution, increased energy consumption and traffic congestion. Urban computing connects unobtrusive and ubiquitous sensing technologies, advanced data management and analytics models, and novel visualization methods, to create win-win-win solutions that improve urban <i>environment</i>, <i>human</i> life quality, and <i>city</i> operation systems. Urban computing also helps us understand the nature of urban phenomena and even predict the future of cities. A survey paper on urban computing:</p> <p align="justify"><b>Yu Zheng, </b>Licia Capra, Ouri Wolfson, Hai Yang. <a href="https://www.microsoft.com/en-us/research/publication/urban-computing-concepts-methodologies-and-applications/">Urban Computing: concepts, methodologies, and applications</a>. ACM Transaction on Intelligent Systems and Technology (<b>ACM TIST</b>). 2014.</p> <p align="justify"><strong>Urban computing</strong> is also a research project in Microsoft Research, led by Dr. <a href="https://www.microsoft.com/en-us/research/people/yuzheng/"><b>Yu Zheng</b></a> since March 2008. By analyzing the big data generated in urban spaces, a series of urban computing applications have been enabled as follows. One of core research problems is to fuse data across different domains. The other is to learn knowledge from spatio-temporal data, e.g. trajectories.</p> <p align="justify">Yu Zheng. <a href="https://www.microsoft.com/en-us/research/publication/methodologies-for-cross-domain-data-fusion-an-overview/">Methodologies for Cross-Domain Data Fusion: An Overview</a>. IEEE Transactions on Big Data, vol. 1, no. 1. 2015. (<a href="https://www.microsoft.com/en-us/research/project/cross-domain-data-fusion/">A Tutorial</a>)</p> <p align="justify">Yu Zheng. <a href="https://www.microsoft.com/en-us/research/publication/trajectory-data-mining-an-overview/">Trajectory Data Mining: An Overview</a>. ACM Transaction on Intelligent Systems and Technology. 2015, vol. 6, issue 3. (<a href="https://www.microsoft.com/en-us/research/project/trajectory-data-mining">A Tutorial</a>)</p>
Some Concrete Examples of Scalable Geospatial Analytics
1. Let us check out cross-domain data fusion in MSR's Urban Computing Group
- lots of interesting papers to read at http://research.microsoft.com/en-us/projects/urbancomputing/.
Here is an excerpt from the MSR Urban Computing link form above:
Urban computing is a process of acquisition, integration, and analysis of big and heterogeneous data generated by a diversity of sources in urban spaces, such as sensors, devices, vehicles, buildings, and human, to tackle the major issues that cities face, e.g. air pollution, increased energy consumption and traffic congestion. Urban computing connects unobtrusive and ubiquitous sensing technologies, advanced data management and analytics models, and novel visualization methods, to create win-win-win solutions that improve urban environment, human life quality, and city operation systems. Urban computing also helps us understand the nature of urban phenomena and even predict the future of cities. A survey paper on urban computing:
Yu Zheng, Licia Capra, Ouri Wolfson, Hai Yang. Urban Computing: concepts, methodologies, and applications. ACM Transaction on Intelligent Systems and Technology (ACM TIST). 2014.
Urban computing is also a research project in Microsoft Research, led by Dr. Yu Zheng since March 2008. By analyzing the big data generated in urban spaces, a series of urban computing applications have been enabled as follows. One of core research problems is to fuse data across different domains. The other is to learn knowledge from spatio-temporal data, e.g. trajectories.
Yu Zheng. Methodologies for Cross-Domain Data Fusion: An Overview. IEEE Transactions on Big Data, vol. 1, no. 1. 2015. (A Tutorial)
Yu Zheng. Trajectory Data Mining: An Overview. ACM Transaction on Intelligent Systems and Technology. 2015, vol. 6, issue 3. (A Tutorial)
Last refresh: Never
%md ## 1. Several sciences are naturally geospatial * forestry, * geography, * geology, * seismology, * etc. etc. See for example the global EQ datastreams from US geological Service below. **A bold idea: ** Imagine the non-parametric inference problem of estimating co-exciting Hawkes-like processes for modelling earth quakes on the entire planet! For a global data source, see US geological Service's Earthquake hazards Program ["http://earthquake.usgs.gov/data/](http://earthquake.usgs.gov/data/).
1. Several sciences are naturally geospatial
- forestry,
- geography,
- geology,
- seismology,
- etc. etc.
See for example the global EQ datastreams from US geological Service below.
A bold idea: Imagine the non-parametric inference problem of estimating co-exciting Hawkes-like processes for modelling earth quakes on the entire planet!
For a global data source, see US geological Service's Earthquake hazards Program "http://earthquake.usgs.gov/data/.
Last refresh: Never
SDS-2.2-360-in-525-03: Geospatial Analytics and Big Data
This needs to be edited for Spark 2.2 and for tight integration with other topics like:
Students need to have seen:
Last refresh: Never