Utilities Needed for Mass Media Data
Johannes Graner (LinkedIn), Albert Nilsson (LinkedIn) and Raazesh Sainudiin (LinkedIn)
2020, Uppsala, Sweden
This project was supported by Combient Mix AB through summer internships at:
Combient Competence Centre for Data Engineering Sciences, Department of Mathematics, Uppsala University, Uppsala, Sweden
Here, certain delta.io tables are loaded. These tables have been prepared already.
You will not be able to load them directly but libraries used in the process are all open-sourced.
object GdeltUtils {
private val gdeltV1Path = "s3a://XXXXXX/GDELT/delta/bronze/v1/"
private val eoiCheckpointPath = "s3a://XXXXXX/.../texata/"
private val poiCheckpointPath = "s3a://XXXXXX/.../person_graph/"
def getGdeltV1Path = gdeltV1Path
def getEOICheckpointPath = eoiCheckpointPath
def getPOICheckpointPath = poiCheckpointPath
}
defined object GdeltUtils
class GdeltUtils:
def getEOICheckpointPath():
return "s3a://XXXXXX/.../texata/"