Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit

Exam: Cluster and Cloud Computing

(COMP90024_2021_SM1)

Question 1

A) Grid computing aimed to support virtual organisations (VOs). Explain what is meant by this italicised term. [2]

B) Cloud-based Infrastructure-as-a-Service (IaaS) platforms such as the Melbourne Research Cloud (MRC) do not support VOs. Discuss some of the missing features of platforms like MRC that would be required to support them? [3]

C) More generally, describe some of the open challenges in realising inter-operable Clouds. Your answer may include technical and non-technical considerations. [5]

Question 2

A) Explain the role of a job scheduler on a high-performance computing system like the University of Melbourne SPARTAN compute cluster. What commands can be used to influence the behaviour of the job scheduler in supporting parallel jobs running on multiple nodes (servers)? [3]

B) A researcher has written Python code (myCode.py) using MPI libraries to process a large data file (bigdata.csv). They wish to run the code on SPARTAN and use four nodes with two processes per node through the following script (myScript.sh).

SBATCH -p cloud

SBATCH --nodes=4

SBATCH --ntasks-per-node=2

myCode.py bigdata.csv

Identify the various problems with this script. [5]

C) Assume that the script above has been fixed. How should it be executed on SPARTAN and why should care be taken in this step? [2]

Question 3

A) Discuss the advantages and disadvantages of Apache Spark compared to CouchDB for big data analytics. [4]

B) You are executing a long-running job on an Apache Spark cluster composed of six worker nodes, and, while the cluster is busy doing an RDD transformation, one worker node crashes. What do you do?

1) Nothing, knowing data are safe

2) Stop and restart the job, fearing some data may be lost

You should write a few sentences motivating your selection. [3]

C) You are developing a Python driver program for Apache Spark using pySpark (i.e., Spark is in client deploy mode), and you realize you need some classes from the Python pandas package (pandas is not included in the base Python libraries). What do you do?

1) you install the pandas package just on the client machine (the one with pySpark on it)

2) you install pandas on all the nodes of the cluster (i.e., both master and worker nodes) and on the client machine

3) you install pandas on all the worker nodes of the cluster and the client machine, but not on the master node

You should write a few sentences motivating your selection. [3]

Question 4

A) Which aspects of big data are most suited to serverless solutions? [2]

B) You are running an Fn FaaS cluster and observe 500 requests directed at the same function coming all at once, what do you expect your cluster to do?

1) Spin up new Docker containers of the function until 500 have been created

2) Spin up new Docker containers of the function until a given number of requests/number of containers ratio is reached

You should write a few sentences motivating your selection. [3]

C) Which aspects of big data can be optimally tackled by the noSQL solution couchDB? [2]

D) You are adding documents to a CouchDB cluster of five nodes using a replica factor of two by directing HTTP requests to the master node of the cluster. Whilst the process keeps adding documents, the master node suddenly crashes. What do you do?

1) Wait until the node restarts, or another node is added to the CouchDB cluster

2) Wait until a new master node is elected by the remaining nodes

3) Just redirect the document-adding process to a different node of the cluster

You should write a few sentences motivating your selection. [3]

Question 5

A) A user wants to create an instance in the “melbourne-qh2-uom” availability zone of the Melbourne Research Cloud using Ansible through interacting with the OpenStack APIs (i.e. not the Melbourne Research Cloud Dashboard). The instance is required to have a volume storage “vol-5523625” created and mounted. The user wrote some Ansible scripts to automate this task. Here is the code the user wrote in one such script:

---

- os_server:

name: 'COMP90024'

flavor: 'uom.mse.2c9g'

availability_zone: 'melbourne-qh2-uom'

security_groups: 'COMP90024'

volumes: 'vol-5523625'

auto_ip: yes

network: {{ network }}

state: absent

wait: yes

1) Will this code work? If not, why not? [2]

2) Describe the interactions that take place with the OpenStack components and their associated APIs in executing this task. [2]

3) To run this code, a file needs to be obtained from the Melbourne Research Cloud? What is this file and why is it needed? [2]

4) What else is needed to run this file? You can assume Ansible, and the relevant SDK are installed. [2]

B) After fixing the problems and adding the missing files, the user gets the following errors when they tried to run the playbook. What is the possible cause of these errors?

1) fatal: [xx.xx.xx.xx] UNREACHABLE! => {“changed”: false, “msg”: “Failed to connect to the host via ssh: [email protected]: Permission denied (publickey).”, “unreachable”: true} [1]

2) fatal: [xx.xx.xx.xx] UNREACHABLE! => {“changed”: false, “msg”: “Failed to connect to the host via ssh: ssh: connect to host xx.xx.xx.xx port 22: Operation timed out”, “unreachable”: true} [1]

Question 6

A) What is the main difference between a container and a virtual machine? [1]

B) What is the main difference between Docker Compose and Docker SWARM? [1]

C) What is the main difference between a volume mount and a bind mount? [1]

D) What do the following Docker commands do?

1) docker service create --replicas 2 --name nginx nginx [1]

2) docker service update --image=nginx:alpine nginx [1]

E) The commands in question D) require a specific Docker feature to be enabled. What is the name of this feature? What does that feature do? What are the benefits of using it? [3]

F) A user wants to run a three-node CouchDB cluster using Docker containers on three virtual instances. They were not able to complete the cluster setup. The containers running on each instance are shown below. What are the possible causes of the issue? [2]

Question 7

A) The Melbourne Research Cloud (MRC) requires users to authenticate using their institutional identities through the Australian Access Federation (AAF). The AAF and its use of the Internet2 Shibboleth technology supports single sign-on.

1) Explain what is meant by the italicized terms. [3]

2) Having logged in to the MRC through the AAF, explain the steps for logging into the AURIN platform (within the same browser session)? [1]

3) Describe the barriers to single sign-on in multiple, heterogeneous Infrastructure-as-a-Service Cloud settings. [4]

4) List four security features that should ideally be supported by the MRC to address some of the security requirements of the biomedical community. [2]