Skip to content

fakhriTo/spark-iqmulus

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

61 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Spark SQL IQmulus Library

A library for reading and writing Lidar point cloud collections in PLY, LAS and XYZ formats from Spark SQL.

Build Status codecov.io

Requirements

This Spark package is for Spark 1.6.2

Usage

The spark-iqmulus library is published as a Spark package, which can be added to Spark jobs launched through spark-shell or spark-submit by using the --packages command line option. For example, to include it when starting the spark shell:

$ bin/spark-shell --packages IGNF:spark-iqmulus:0.1.0-s_2.10

Unlike using --jars, using --packages ensures that this library and its dependencies will be added to the classpath. The --packages argument can also be used with bin/spark-submit. For now this library is only published for Scala 2.10.

Scala API

Examples below require the creation of an SQLContext and the following imports, depending on which aspect of the library is being used:

val sqlContext = new SQLContext(sc)

import fr.ign.spark.iqmulus.ply._
import fr.ign.spark.iqmulus.las._
import fr.ign.spark.iqmulus.xyz._

Reading files as Spark dataframes:

val df1 = sqlContext.read.ply("myfile.ply")
val df2 = sqlContext.read.las("myfile.las")
val df3 = sqlContext.read.xyz("myfile.xyz")

Writing back Spark dataframes as files:

similarly, for XYZ and LAS :

val df = ...
df.write.ply("/tmp/ply")
df.write.las("/tmp/las")
df.write.xyz("/tmp/xyz")

Acknowledgments

IGN France IQmulus The development of this library is partly supported by:

  • the EU FP7 Project IQmulus, which leverages the information hidden in large heterogeneous geospatial data sets and make them a practical choice to support reliable decision making (N. ICT- 2011-318787).
  • IGN, the French National Mapping Agency and its MATIS research lab.

References

To our knowledge this library has been used in the following research:

About

Draft to test the lib

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Scala 100.0%