River IQ

Image

Hive Integration with Spark

  Ashish Kumar      Spark January 22, 2019

Are you struggling to access hive using spark?Is your hive table is not showing in spark?No worry here I am going to show you the key changes made in HDP 3.0 for hive and how we can access hive using spark. Now in HDP 3.0 both spark and hive ha their own meta store. Hive uses the "hive" catalog, and Spark uses the "spark" catalog. With HDP 3.0 in Ambari you can find below configuration for spark.As we know before we could access hive table in spark using HiveContext/SparkSession but now in HDP 3.0 we can access hive using Hive ...

Read more
Image

Apache Hive 3.0

  Ashish Kumar      Hive September 26, 2018

Apache Hive 3.0 Workload managementManaging resources was always a concern to gain hive performance. With LLAP it improved the performance. With hive 3.0 using workload management, you can create resource pools and allocate resources to match your needs and prevent contention for those resources.  Workload management improves parallel query execution and cluster sharing for queries running on Hive LLAP, and improves performance of non-LLAP queries. Transaction processing improvementsMature versions of ACID (Atomicity, Consistency, I...

Read more