River IQ | Blogs

Hive Integration with Spark

Ashish Kumar Spark January 22, 2019

Are you struggling to access hive using spark?Is your hive table is not showing in spark?No worry here I am going to show you the key changes made in HDP 3.0 for hive and how we can access hive using spark. Now in HDP 3.0 both spark and hive ha their own meta store. Hive uses the "hive" catalog, and Spark uses the "spark" catalog. With HDP 3.0 in Ambari you can find below configuration for spark.As we know before we could access hive table in spark using HiveContext/SparkSession but now in HDP 3.0 we can access hive using Hive ...

Apache Hive 3.0

Ashish Kumar Hive September 26, 2018

Apache Hive 3.0 Workload managementManaging resources was always a concern to gain hive performance. With LLAP it improved the performance. With hive 3.0 using workload management, you can create resource pools and allocate resources to match your needs and prevent contention for those resources. Workload management improves parallel query execution and cluster sharing for queries running on Hive LLAP, and improves performance of non-LLAP queries. Transaction processing improvementsMature versions of ACID (Atomicity, Consistency, I...