Apache Spark

The Connector for Apache Spark – Community Edition allows you to configure a connection to Spark server, and then use activities to run Spark SQL, execute spark Scala code and submit spark job.

It provides the following features:

• Spark Connection Shared Resource: You can use the Spark connection shared resource to connect Spark Livy server. The shared resource is used by the Spark activities.
• Spark SQL Activity: You can use this activity to run spark sql.
• Spark Execution Activity: You can use this activity to submit spark code and jobs.
• Wait for Completion Activity: You can use this activity to wait for spark execution completion.