Thursday, 21 July 2022

Spark Interview Questions

 

You can do it just using split and size of pyspark API functions (Below is 


example):-


from pyspark.sql import SparkSession

from pyspark.sql import SQLContext


my_spark = SparkSession \

    .builder \

    .appName("Python Spark SQL example") \

    .enableHiveSupport() \

    .getOrCreate()


sqlcontext = SQLContext(my_spark.sparkContext)

sqlContext.createDataFrame([['this is a sample address'],['another address']])\

.select(F.size(F.split(F.col("_1"), " "))).show()


Below is Output:-

+------------------+

|size(split(_1,  ))|

+------------------+

|                 5|

|                 2|

+------------------+







import sys

 

from pyspark import SparkContext, SparkConf

 

if __name__ == "__main__":

# create Spark context with necessary configuration

sc = SparkContext("local","PySpark Word Count Exmaple")

# read data from text file and split each line into words

words = sc.textFile("D:/workspace/spark/input.txt").flatMap(lambda line: 


line.split(" "))

# count the occurrence of each word

wordCounts = words.map(lambda word: (word, 1)).reduceByKey(lambda a,b:a 


+b)

# save the counts to output

wordCounts.saveAsTextFile("D:/workspace/spark/output/")







Apache Spark is an open-source cluster computing framework for real-time 


processing.


YARN is one of the key features in Spark, providing a central and resource 


management platform to deliver scalable operations across the cluster. 


YARN is a distributed container manager, like Mesos for example, whereas 

Spark is a data processing tool. 

Spark can run on YARN, the same way Hadoop Map Reduce can run on YARN. 

Running Spark on YARN necessitates a binary distribution of Spark as built on 


YARN support. 



Explain the concept of Resilient Distributed Dataset (RDD).


Resilient Distributed Datasets are the fundamental data structure of Apache 


Spark. 

It is embedded in Spark Core. 

RDDs are immutable, fault-tolerant, distributed collections of objects that can 


be operated on in parallel.RDD’s are split into partitions and can be executed 


on different nodes of a cluster.


RDDs are created by either transformation of existing RDDs or by loading an 


external dataset from stable storage like HDFS or HBase.


RDD stands for Resilient Distribution Datasets. An RDD is a fault-tolerant 


collection of operational elements that run in parallel. The partitioned data in 


RDD is immutable and distributed in nature. There are primarily two types of 


RDD:


Parallelized Collections: Here, the existing RDDs running parallel with one 


another.

Hadoop Datasets: They perform functions on each file record in HDFS or other 


storage systems.

RDDs are basically parts of data that are stored in the memory distributed 


across many nodes. RDDs are lazily evaluated in Spark. This lazy evaluation is 


what contributes to Spark’s speed



The executor memory is basically a measure on how much memory of the worker node 


will the application utilize


Spark manages data using partitions that help parallelize distributed data 


processing with minimal network traffic for sending data between executors


park tries to read data into an RDD from the nodes that are close to it. Since 


Spark usually accesses distributed partitioned data, to optimize transformation 


operations it creates partitions to hold the data chunks. Everything in Spark is 


a partitioned RDD.



RDD has distributed a collection of objects. Distributed means, each RDD is 


divided into multiple partitions. Each of these partitions can reside in memory 


or stored on the disk of different machines in a cluster. RDDs are immutable 


(Read Only) data structure. You can’t change original RDD, but you can always 


transform it into different RDD with all changes you want.



Spark applications run as independent processes that are coordinated by the 


SparkSession object in the driver program. The resource manager or cluster 


manager assigns tasks to the worker nodes with one task per partition. Iterative 


algorithms apply operations repeatedly to the data so they can benefit from 


caching datasets across iterations. A task applies its unit of work to the 


dataset in its partition and outputs a new partition dataset. Finally, the 


results are sent back to the driver application or can be saved to the disk.







 What is a Parquet file and what are its advantages?

Parquet is a columnar format that is supported by several data processing 


systems. With the Parquet file, Spark can perform both read and write 


operations. 


Some of the advantages of having a Parquet file are:


It enables you to fetch specific columns for access.

It consumes less space

It follows the type-specific encoding

It supports limited I/O operations




11. What is shuffling in Spark? When does it occur?

Shuffling is the process of redistributing data across partitions that may lead 


to data movement across the executors. The shuffle operation is implemented 


differently in Spark compared to Hadoop. 


Shuffling has 2 important compression parameters:


spark.shuffle.compress – checks whether the engine would compress shuffle 


outputs or not spark.shuffle.spill.compress – decides whether to compress 


intermediate shuffle spill files or not


It occurs while joining two tables or while performing byKey operations such as 


GroupByKey or ReduceByKey





13. How can you calculate the executor memory?

Consider the following cluster information:


cluster

   node=10

   each node 

     core =16 (-1 OS)

     RAM = 61GB (-1 for OS)



Here is the number of core identification:

  Number of core is - number of concurrent task an executor can run in parallel.

  rule of tumb = 5 


To calculate the number of executor identification:


no of execuotr = no. of core/concurrent task

               = 15 / 5 =3


number of executor = no of node * no of executore on each node 

10 * 3 = 30


executor.






Top 40 Apache Spark Interview Questions and Answers

By Shivam Arora

Last updated on Jul 1, 202161364

Top 40 Apache Spark Interview Questions and Answers

Apache Spark is a unified analytics engine for processing large volumes of data. 


It can run workloads 100 times faster and offers over 80 high-level operators 


that make it easy to build parallel apps. Spark can run on Hadoop, Apache Mesos, 


Kubernetes, standalone, or in the cloud, and can access data from multiple 


sources.


And this article covers the most important Apache Spark Interview questions that 


you might face in a Spark interview. The Spark interview questions have been 


segregated into different sections based on the various components of Apache 


Spark and surely after going through this article you will be able to answer 


most of the questions asked in your next Spark interview.


Apache Spark Interview Questions

The Apache Spark interview questions have been divided into two parts:


Apache Spark Interview Questions for Beginners

Apache Spark Interview Questions for Experienced 

Let us begin with a few basic Apache Spark interview questions!


Apache Spark Interview Questions for Beginners

1. How is Apache Spark different from MapReduce?

Apache Spark


MapReduce


Spark processes data in batches as well as in real-time


MapReduce processes data in batches only


Spark runs almost 100 times faster than Hadoop MapReduce


Hadoop MapReduce is slower when it comes to large scale data processing


Spark stores data in the RAM i.e. in-memory. So, it is easier to retrieve it


Hadoop MapReduce data is stored in HDFS and hence takes a long time to retrieve 


the data


Spark provides caching and in-memory data storage


Hadoop is highly disk-dependent


2. What are the important components of the Spark ecosystem?

apache-spark


Apache Spark has 3 main categories that comprise its ecosystem. Those are:


Language support: Spark can integrate with different languages to applications 


and perform analytics. These languages are Java, Python, Scala, and R.

Core Components: Spark supports 5 main core components. There are Spark Core, 


Spark SQL, Spark Streaming, Spark MLlib, and GraphX.

Cluster Management: Spark can be run in 3 environments. Those are the Standalone 


cluster, Apache Mesos, and YARN.

3. Explain how Spark runs applications with the help of its architecture.

This is one of the most frequently asked spark interview questions, and the 


interviewer will expect you to give a thorough answer to it.


worker-node


Spark applications run as independent processes that are coordinated by the 


SparkSession object in the driver program. The resource manager or cluster 


manager assigns tasks to the worker nodes with one task per partition. Iterative 


algorithms apply operations repeatedly to the data so they can benefit from 


caching datasets across iterations. A task applies its unit of work to the 


dataset in its partition and outputs a new partition dataset. Finally, the 


results are sent back to the driver application or can be saved to the disk.


4. What are the different cluster managers available in Apache Spark?

Standalone Mode: By default, applications submitted to the standalone mode 


cluster will run in FIFO order, and each application will try to use all 


available nodes. You can launch a standalone cluster either manually, by 


starting a master and workers by hand, or use our provided launch scripts. It is 


also possible to run these daemons on a single machine for testing.

Apache Mesos: Apache Mesos is an open-source project to manage computer 


clusters, and can also run Hadoop applications. The advantages of deploying 


Spark with Mesos include dynamic partitioning between Spark and other frameworks 


as well as scalable partitioning between multiple instances of Spark.

Hadoop YARN: Apache YARN is the cluster resource manager of Hadoop 2. Spark can 


be run on YARN as well.

Kubernetes: Kubernetes is an open-source system for automating deployment, 


scaling, and management of containerized applications.

5. What is the significance of Resilient Distributed Datasets in Spark?

Resilient Distributed Datasets are the fundamental data structure of Apache 


Spark. It is embedded in Spark Core. RDDs are immutable, fault-tolerant, 


distributed collections of objects that can be operated on in parallel.RDD’s are 


split into partitions and can be executed on different nodes of a cluster.


RDDs are created by either transformation of existing RDDs or by loading an 


external dataset from stable storage like HDFS or HBase.


Here is how the architecture of RDD looks like:


create-rdd


6. What is a lazy evaluation in Spark?

When Spark operates on any dataset, it remembers the instructions. When a 


transformation such as a map() is called on an RDD, the operation is not 


performed instantly. Transformations in Spark are not evaluated until you 


perform an action, which aids in optimizing the overall data processing 


workflow, known as lazy evaluation.


7. What makes Spark good at low latency workloads like graph processing and 


Machine Learning?

Apache Spark stores data in-memory for faster processing and building machine 


learning models. Machine Learning algorithms require multiple iterations and 


different conceptual steps to create an optimal model. Graph algorithms traverse 


through all the nodes and edges to generate a graph. These low latency workloads 


that need multiple iterations can lead to increased performance.


8. How can you trigger automatic clean-ups in Spark to handle accumulated 


metadata?

To trigger the clean-ups, you need to set the parameter spark.cleaner.ttlx.


job


9. How can you connect Spark to Apache Mesos?

There are a total of 4 steps that can help you connect Spark to Apache Mesos.


Configure the Spark Driver program to connect with Apache Mesos

Put the Spark binary package in a location accessible by Mesos

Install Spark in the same location as that of the Apache Mesos

Configure the spark.mesos.executor.home property for pointing to the location 


where Spark is installed

10. What is a Parquet file and what are its advantages?

Parquet is a columnar format that is supported by several data processing 


systems. With the Parquet file, Spark can perform both read and write 


operations. 


Some of the advantages of having a Parquet file are:


It enables you to fetch specific columns for access.

It consumes less space

It follows the type-specific encoding

It supports limited I/O operations

Learn open-source framework and scala programming languages with the Apache 


Spark and Scala Certification training course.

11. What is shuffling in Spark? When does it occur?

Shuffling is the process of redistributing data across partitions that may lead 


to data movement across the executors. The shuffle operation is implemented 


differently in Spark compared to Hadoop. 


Shuffling has 2 important compression parameters:


spark.shuffle.compress – checks whether the engine would compress shuffle 


outputs or not spark.shuffle.spill.compress – decides whether to compress 


intermediate shuffle spill files or not


It occurs while joining two tables or while performing byKey operations such as 


GroupByKey or ReduceByKey


12. What is the use of coalesce in Spark?

Spark uses a coalesce method to reduce the number of partitions in a DataFrame.


Suppose you want to read data from a CSV file into an RDD having four 


partitions.


partition


This is how a filter operation is performed to remove all the multiple of 10 


from the data.




The RDD has some empty partitions. It makes sense to reduce the number of 


partitions, which can be achieved by using coalesce.




This is how the resultant RDD would look like after applying to coalesce.


13. How can you calculate the executor memory?

Consider the following cluster information:


cluster


Here is the number of core identification:


core-iden


To calculate the number of executor identification:


executor.


14. What are the various functionalities supported by Spark Core?

Spark Core is the engine for parallel and distributed processing of large data 


sets. The various functionalities supported by Spark Core include:


Scheduling and monitoring jobs

Memory management

Fault recovery

Task dispatching

15. How do you convert a Spark RDD into a DataFrame?

There are 2 ways to convert a Spark RDD into a DataFrame:


Using the helper function - toDF

import com.mapr.db.spark.sql._


val df = sc.loadFromMapRDB(<table-name>)


.where(field(“first_name”) === “Peter”)


.select(“_id”, “first_name”).toDF()


Using SparkSession.createDataFrame

You can convert an RDD[Row] to a DataFrame by


calling createDataFrame on a SparkSession object


def createDataFrame(RDD, schema:StructType)


16. Explain the types of operations supported by RDDs.

RDDs support 2 types of operation:


Transformations: Transformations are operations that are performed on an RDD to 


create a new RDD containing the results (Example: map, filter, join, union)


Actions: Actions are operations that return a value after running a computation 


on an RDD (Example: reduce, first, count) 


17. What is a Lineage Graph?

This is another frequently asked spark interview question. A Lineage Graph is a 


dependencies graph between the existing RDD and the new RDD. It means that all 


the dependencies between the RDD will be recorded in a graph,  rather than the 


original data.


The need for an RDD lineage graph happens when we want to compute a new RDD or 


if we want to recover the lost data from the lost persisted RDD. Spark does not 


support data replication in memory. So, if any data is lost, it can be rebuilt 


using RDD lineage. It is also called an RDD operator graph or RDD dependency 


graph.


18. What do you understand about DStreams in Spark?

Discretized Streams is the basic abstraction provided by Spark Streaming. 


It represents a continuous stream of data that is either in the form of an input 


source or processed data stream generated by transforming the input stream.


dstream.


19. Explain Caching in Spark Streaming.

Caching also known as Persistence is an optimization technique for Spark 


computations. Similar to RDDs, DStreams also allow developers to persist the 


stream’s data in memory. That is, using the persist() method on a DStream will 


automatically persist every RDD of that DStream in memory. It helps to save 


interim partial results so they can be reused in subsequent stages.


The default persistence level is set to replicate the data to two nodes for 


fault-tolerance, and for input streams that receive data over the network.


kafka


20. What is the need for broadcast variables in Spark?

Broadcast variables allow the programmer to keep a read-only variable cached on 


each machine rather than shipping a copy of it with tasks. They can be used to 


give every node a copy of a large input dataset in an efficient manner. Spark 


distributes broadcast variables using efficient broadcast algorithms to reduce 


communication costs.


scala


scala> val broadcastVar = sc.broadcast(Array(1, 2, 3))


broadcastVar: org.apache.spark.broadcast.Broadcast[Array[Int]] = Broadcast(0)


scala> broadcastVar.value


res0: Array[Int] = Array(1, 2, 3)


Apache Spark Beginners Free Course

Learn the Fundamentals of Apache SparkENROLL NOWApache Spark Beginners Free 


Course

Apache Spark Interview Questions for Experienced

21. How to programmatically specify a schema for DataFrame?

DataFrame can be created programmatically with three steps:


Create an RDD of Rows from the original RDD;

Create the schema represented by a StructType matching the structure of Rows in 


the RDD created in Step 1.

Apply the schema to the RDD of Rows via createDataFrame method provided by 


SparkSession.

spark-session


22. Which transformation returns a new DStream by selecting only those records 


of the source DStream for which the function returns true?

1. map(func)


2. transform(func)


3. filter(func)


4. count()


The correct answer is c) filter(func).


23. Does Apache Spark provide checkpoints?

This is one of the most frequently asked spark interview questions where the 


interviewer expects a detailed answer (and not just a yes or no!). Give as 


detailed an answer as possible here.


Yes, Apache Spark provides an API for adding and managing checkpoints. 


Checkpointing is the process of making streaming applications resilient to 


failures. It allows you to save the data and metadata into a checkpointing 


directory. In case of a failure, the spark can recover this data and start from 


wherever it has stopped.


There are 2 types of data for which we can use checkpointing in Spark.


Metadata Checkpointing: Metadata means the data about data. It refers to saving 


the metadata to fault-tolerant storage like HDFS. Metadata includes 


configurations, DStream operations, and incomplete batches.


Data Checkpointing: Here, we save the RDD to reliable storage because its need 


arises in some of the stateful transformations. In this case, the upcoming RDD 


depends on the RDDs of previous batches. 


24. What do you mean by sliding window operation?

Controlling the transmission of data packets between multiple computer networks 


is done by the sliding window. Spark Streaming library provides windowed 


computations where the transformations on RDDs are applied over a sliding window 


of data.


originald-stream


25. What are the different levels of persistence in Spark?

DISK_ONLY - Stores the RDD partitions only on the disk


MEMORY_ONLY_SER - Stores the RDD as serialized Java objects with a one-byte 


array per partition


MEMORY_ONLY - Stores the RDD as deserialized Java objects in the JVM. If the RDD 


is not able to fit in the memory available, some partitions won’t be cached


OFF_HEAP - Works like MEMORY_ONLY_SER but stores the data in off-heap memory


MEMORY_AND_DISK - Stores RDD as deserialized Java objects in the JVM. In case 


the RDD is not able to fit in the memory, additional partitions are stored on 


the disk


MEMORY_AND_DISK_SER - Identical to MEMORY_ONLY_SER with the exception of storing 


partitions not able to fit in the memory to the disk


26. What is the difference between map and flatMap transformation in Spark 


Streaming?

map()


flatMap()


A map function returns a new DStream by passing each element of the source 


DStream through a function func


It is similar to the map function and applies to each element of RDD and it 


returns the result as a new RDD


Spark Map function takes one element as an input process it according to custom 


code (specified by the developer) and returns one element at a time


FlatMap allows returning 0, 1, or more elements from the map function. In the 


FlatMap operation


27. How would you compute the total count of unique words in Spark?

1. Load the text file as RDD:


sc.textFile(“hdfs://Hadoop/user/test_file.txt”);


2. Function that breaks each line into words:


def toWords(line):


return line.split();


3. Run the toWords function on each element of RDD in Spark as flatMap 


transformation:


words = line.flatMap(toWords);


4. Convert each word into (key,value) pair:


def toTuple(word):


return (word, 1);


wordTuple = words.map(toTuple);


5. Perform reduceByKey() action:


def sum(x, y):


return x+y:


counts = wordsTuple.reduceByKey(sum) 


6. Print:


counts.collect()


=========================================================



1)

paired RDD


2) 

coalesce and repartition

repartion can increase and decrease the partition and does full shuffle. 


Coalesce avoids full shuffle and better for reduced number of partitions

Spark works fine when the partitions sizes are even, Coalesce might give an 


uneven sized partitions and that would impact the performance of the job


3)

Different level of persistence

memory only

disk only

memory and disk


4)

difference static and dynamic partition in Hive



5)

Parquet - 

columer store

good for compression

analytical functions


6)

Bucketting with dynamic partition - is it possible ?




7)

second higest salary



8)

Can we use hive as metadata DB instead of mysql



9)

what is sqoop




10)

expalin map reduce flow

mapper => key-value pair => shuffle =>reducer


Assume 10GB data.how may mapper and reducer will be created

10GB / 128MB = number of mapper

by default reducer =1 , we an specify number fo reducer 2 or 3


how does data get transfered from mapper to reducer

shuffle phase automatically handle it


if you have more then 1 reducer how data get into reducr


in shuffle phase the key - value pair goes through partitioner,

partitioner decide where KV should go. partitioner is hash function.

for each key it generate hash which goes to fixed reducer 



11)


when to use Hive and when to use Spark


hive - is client tool - any query will be executed as Map reducer on hadoop 


cluster.

query get translated into MR


spark is execution framework.it creates lineage, form pysicl and logical plan 


and execute.




12)

does hive support hive and curd


CURD (update delete) and ACId (Transcation) is experimental feature which i only

available for ORC (optimise RC file format)

available from Apache 0.1.3



13) 

what is diff between RDD and dataframe

In spark when we write instruction, those instruction are transformed into RDD

and spark will form the association between RDD based on instr

then those instruction are converted to logical and pyhiscal plan


Dataframe is improvement for RDD

Dataframe give facility to view at RDD with schema

as if we are looking to it as table


pulgable memory management. like project tungston

and better optimiser (catalyst optiiser)



14)

Oozie


scheduler tool

it helps to form workflow

help to maintain dependency.


15) 

Flume and Kafka

are data ingestion tool

Kafla is cluster in itslef. It is used for streaming.

kafka is distributed. can get stream from multiple producer and give data to 


multiple consumer

data can be pushed to hdfs


16)

Sentry

Its authorization tool.



17)

https://www.youtube.com/watch?v=haPwh4m_jq0



Spark is memory intensive distributed system. With access t o several GB of 


memory. 


But yet job fails with out of memory issue.


Driver and Executor are the JVM process.


they have fixed memory allocated




when spark job fails, identify what is failing.


Spark job is made of drive and set of executor. They both use large amount of 


memory


they both are allocated certain finite amount of memory


When they need more then what is allocated we get OOM




Check whether driver failed or executor failed or Task inside executor failed.



Spark job consist of numbe rof task.

Task run insdie Executor container


Sevral spark job can run inside container (executor)

Executor are the containers with CPU and memory allocated.


like this multiple executor (container) can exists for the job


Error like OOM or container killed will be showing up when memory issue happens.




Driver is orchestrating the execution between executors


both memory can be controlled by 


spark.executor.memory


spark.driver.memory




If 4 GB is allocated to Executor


            300 MB is reserved


           60 % of (4GB - 300M ) - will go to Spark = spark memory


          rest to user memory




out of 60 % of (4GB - 300M )  (Spark Memory)


some is used for storage(cached obj) and rest execution




spark memory is dynamically managed (available after 1.16 version .)


Initially storage and execution get 50 -50 % of spark memory (60% of 4gb - 


300mb)




4gb = 300MB   reserved


      1.5     user memory (40% of 4gb - 300M)


      2.2     spark Memory (60% of 4gb - 300M)


         1.1  storage


         1.1  Execution




when memory for execution is not enough spark with take memory from storage .


If storage have object, it will move to Disk.


Storage memory inside spark memory - is managed by spark.memory.storageFraction  


which is 50%






How Many cores are allocated to the executor




number of cores mean number of task we can run in parallel.


If we have 2 CPU core then we can run 2 task in parallel.


# CPU Core = # Task in parallel




rule of thumb = for each CPU 2or 3 task


Number of partition decides the number of Task






so if we have big data set which have 100 partition 


then spark will create 100 task.


If each task have to deal with lot of data then potential problem to get OOM




few partition mean - Each task is overload,






Q)

How to optimise Spark


Choose file format - JSON and XML are slow. No optimization

                     PARQUET or RC will give better performance




Partition Technique

 rdd1 = sc.parallelize(Array((a,3), (a,1), (b,7), (a,5), (c,5)),3 ) 


.partitionBy(new ashpartition(2))

 rdd2 = sc.parallelize(Array((c,5), (d,1), (b,5), (d,5)),2 ) .partitionBy(new 


ashpartition(2))



BroadCast :- broad cst certain data to all executor


Kryo Seriallizer:-

spark serialize data. for all operation

default java serailzer are bulky

 jrdd = rdd1.join(rdd2)

 jrdd.collect().foreach(println)





Q)


The convergence of SQL and NoSQL

Both SQL and NoSQL databases have their pros and cons. As such, there has been a 


movement to take the best characteristics of both types of databases and 


integrate them so users can realize the best of both worlds.


For instance, MySQL, the most popular open-source relational database, offers 


MySQL Document Store. This provides the structure of a MySQL database combined 


with the flexibility and high availability of NoSQL without having to implement 


a separate NoSQL database.


MongoDB, one of the most popular NoSQL databases, offers multi-document ACID 


transactions.


AWS’ managed NoSQL database, DynamoDB, also provides ACID-compliant transaction 


functionality.


And with the easy database setup that cloud service providers offer, you have 


the ability to use both SQL and NoSQL databases in your cloud data architecture 


to meet your data storage needs.


Now you have much more flexibility regardless of whether you choose a SQL or 


NoSQL database, and there are sure to be more flexible options in the future.


Database options

Regardless of whether you go with a SQL or NoSQL database (or both!), there are 


plenty of options to choose from.


On-premise SQL database offerings include:


MySQL – as mentioned prior, the most popular open-source relational database

Microsoft SQL server – Microsoft’s enterprise version of SQL

PostgreSQL – and enterprise-level, open-source database focused on extensibility

Oracle – full-service (and expensive) SQL option

MariaDB – an enhanced version of MySQL, built by MySQL’s original developers

And many more

The major cloud service platforms have their own SQL options:


AWS has:

RDS, their standard cloud SQL database

Aurora, which focuses on increased throughput and scalability

Microsoft Azure has:

Azure SQL Database, their managed database-as-a-service

Azure Database for MySQL, PostgreSQL, and MariaDB

Google Cloud Platform (GCP) has:

Cloud SQL, which you can use for MySQL and PostgreSQL

Cloud Spanner, which combines elements of SQL and NoSQL

On-premise NoSQL database options include:


MongoDB – by far the most popular NoSQL database

Redis – an open source, distributed, in-memory key-value database that is super 


fast

Cassandra – free, open-source NoSQL database created by Facebook that focuses on 


scalability and high availability

Many others

Cloud service providers offer plenty of NoSQL options as well:


AWS has:

DynamoDB, its managed NoSQL database

DocumentDB, a fast, scalable, highly-available MongoDB-compatible database

Microsoft Azure offers:

CosmosDB, its globally distributed, multi-model database

Google Cloud has:

Bigtable, its NoSQL wide-column database service

Cloud Datastore, its NoSQL document database service

Cloud Firestore, a cloud-native NoSQL document database that helps store and 


query app data

There is no shortage of database options to choose from!



Q)

nodes 54


Vcore 32

Memorty 32

CSH 5.16

Impala 2.12


impala no of deamon = 250

overall mem  1TB

No comments:

Post a Comment